CN113753041A - Early warning method and early warning device for mobile camera ranging - Google Patents
Early warning method and early warning device for mobile camera ranging Download PDFInfo
- Publication number
- CN113753041A CN113753041A CN202111150015.2A CN202111150015A CN113753041A CN 113753041 A CN113753041 A CN 113753041A CN 202111150015 A CN202111150015 A CN 202111150015A CN 113753041 A CN113753041 A CN 113753041A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- early warning
- value
- probability
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000003384 imaging method Methods 0.000 claims abstract description 26
- 238000004364 calculation method Methods 0.000 claims abstract description 16
- 230000004888 barrier function Effects 0.000 claims abstract description 12
- 230000015572 biosynthetic process Effects 0.000 claims description 13
- 238000003786 synthesis reaction Methods 0.000 claims description 13
- 230000001133 acceleration Effects 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000013461 design Methods 0.000 claims description 2
- 230000035484 reaction time Effects 0.000 claims description 2
- 230000035945 sensitivity Effects 0.000 abstract description 4
- 230000006872 improvement Effects 0.000 description 9
- 230000008569 process Effects 0.000 description 9
- 238000001914 filtration Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000007781 pre-processing Methods 0.000 description 4
- 238000006073 displacement reaction Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
- B60W30/08—Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
- B60W30/095—Predicting travel path or likelihood of collision
- B60W30/0953—Predicting travel path or likelihood of collision the prediction being responsive to vehicle dynamic parameters
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
- B60W30/08—Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
- B60W30/095—Predicting travel path or likelihood of collision
- B60W30/0956—Predicting travel path or likelihood of collision the prediction being responsive to traffic or environmental parameters
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W2050/0001—Details of the control system
- B60W2050/0019—Control system elements or transfer functions
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
- B60W2050/143—Alarm means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
- B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
- B60W2420/403—Image sensing, e.g. optical camera
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2552/00—Input parameters relating to infrastructure
- B60W2552/50—Barriers
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Human Computer Interaction (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses a mobile camera ranging early warning method and an early warning device. The method comprises the following steps: calculating a current estimated value and a subsequent moment initial value; firstly, dividing outline polygons and angular points of the barrier, then projecting and converting the angular points to a normalized imaging plane, then selecting two points with the minimum and the maximum pixel horizontal coordinates in pixel coordinates as feature points of the barrier, and finally obtaining a geodetic coordinate value of the barrier; firstly, determining the value range of a true value of a course angle of a vehicle, then determining the driving safety distance of the vehicle, and finally calculating a probable collision area; judging whether the characteristic line segments are in the approximate probability collision area, calculating the numerical value of the center distance, and finally calculating the early warning probability; and early warning the vehicle according to the early warning probability. The method and the device have the advantages that the collision probability is predicted by combining the vehicle probable collision area, the accurate early warning function is realized on the safe driving of the vehicle, the application to the automatic driving of the vehicle can be used for reference, the collision and friction probability of the vehicle is reduced, the calculation is accurate, the sensitivity is good, and the early warning capability is strong.
Description
Technical Field
The invention relates to an early warning method in the technical field of distance measurement, in particular to a mobile camera distance measurement early warning method and an early warning device.
Background
At present, most of vehicle-mounted anti-collision systems select a distance sensor to be mounted in one direction or four directions, the working visual angle is small, the conditions of side-to-side front collision and the like cannot be well predicted and warned, and most of the vehicle-mounted anti-collision systems are only suitable for low-speed conditions. The standard embedded binocular vision device does not depend on the historical track of the vehicle, but cannot realize a larger observation base line due to the size of the vehicle body, and is not suitable for the condition with higher real-time requirement without simplified processing. The state information such as the vehicle running track can be acquired by using a vehicle-mounted running recorder, but the internal relation among data of vehicle-mounted sensors is unclear, and a large random error exists. The anti-collision early warning system based on satellite image and track prediction cannot process emergencies such as temporarily set obstacles. The existing vehicle-mounted anti-collision system has the problems of inaccurate vehicle running state estimation, inflexible collision early warning and high accident probability, so that great potential safety hazards exist.
Disclosure of Invention
The invention provides a mobile camera ranging early warning method and an early warning device, which aim to solve the technical problems of inaccurate estimation of vehicle running states and inflexible early warning of collision of the conventional vehicle-mounted anti-collision system.
The invention is realized by adopting the following technical scheme: a mobile camera ranging early warning method comprises the following steps:
s1: extracting historical driving state estimation data and current driving state observation data of the vehicle, and calculating a current estimation value and a subsequent moment initial value of a vehicle driving state variable;
s2: firstly, segmenting an outline polygon and an angular point which are recognized as the same obstacle from a video frame image shot by the vehicle, then projecting and converting the angular point to a normalized imaging plane according to an offline calibration result of a video camera of the vehicle to obtain an image point and a pixel coordinate of the angular point in the normalized imaging plane, then selecting two points with the minimum and the maximum pixel horizontal coordinates in the pixel coordinate as feature points of the obstacle, and finally obtaining a ground coordinate value of the obstacle according to the position relationship between the obstacle and the vehicle;
s3: firstly, determining the value range of the true value of the course angle of the vehicle, then determining the safe driving distance of the vehicle, and finally calculating the probable collision area of the vehicle according to the current estimated value, the initial value of the subsequent moment and the safe driving distance;
s4: judging whether a characteristic line segment formed by the barrier geodetic coordinate values has a point falling in the high-probability collision region or not, calculating a central distance value of a point which is in the high-probability collision region and is closest to the circle center of the high-probability collision region, and finally calculating an early warning probability according to the central distance value; and
s5: and early warning the vehicle according to the early warning probability.
According to the method, the position of the obstacle shot by the mobile camera is calculated by a vehicle state information optimization estimation method, and the collision probability is predicted by combining the vehicle general probability collision area, so that the method has an accurate early warning function on the safe driving of the vehicle; moreover, the method can prevent the vehicle from colliding with obstacles and the like in the driving process, reduce the collision probability of the vehicle, can be used for reference in automatic driving of the vehicle, solve the technical problems of inaccurate estimation of the driving state of the vehicle and inflexible collision early warning of the existing vehicle-mounted anti-collision system, and obtain the technical effects of accurate calculation, good sensitivity and strong early warning capability.
As a further improvement of the above scheme, in step S5, a low-risk early warning probability and a high-risk early warning probability are preset according to actual driving data of the vehicle, and then the early warning probabilities are compared with the low-risk early warning probability and the high-risk early warning probability, respectively, so as to obtain corresponding early warning levels according to a preset scheme.
As a further improvement of the above solution, in step S1, the current estimated value and the subsequent time initial value are calculated by a system equation of equal longitudinal acceleration-equal angular velocity-piecewise linear motion vehicle motion model design state variables.
As a further improvement of the above solution, the system equation of the current estimation value is:
in the formula (x)t,yt) Is the vehicle geodetic coordinate value v in the current estimation valuetFor the vehicle speed in said current estimate, atFor the vehicle longitudinal acceleration, ω, in said current estimatetThe state estimation value of the last time t-delta t is x for the vehicle yaw angular velocity in the current estimation valuet-Δt,yt-Δt,θt-Δt,vt-Δt,at-Δt,ωt-ΔtEpsilon is system noise, and the current estimated value is the sum of the current initial value and the system noise;
the observation equation of the current observation value is as follows:
in the formula (X)t,Yt) Is the vehicle geodetic coordinate value theta in the current observation valuetIs the heading angle, V, of the vehicletIs the vehicle speed in the current observed value, AtIs the longitudinal acceleration, omega, of the vehicle in the current observed valuetAnd delta is the observation noise, and the current observation value is the sum of the current estimation value and the observation noise.
As a further improvement of the above solution, the corner point is defined as a point Ci(i ═ 1,2, …), and point C is addediThe projection is transformed to the normalized imaging plane with the column direction perpendicular to the automobile chassis of the vehicle and the row direction parallel to the automobile wheel axle of the vehicle, and the projection is subjected to feature point grabbing and one-dimensional processing; wherein the pixel coordinate is defined as a mark Di(uDi,vDi) The two feature points are defined as L (u)L,vL),R(uR,vR) L, R point original image L0,R0Position L in the geodetic coordinate system0(xL0,yL0),R0(xR0,yR0)。
As a further improvement of the above solution, the calculation formula of the obstacle geodetic coordinate value is:
wherein θ is θt-θt-Δt，
up1,up2The displacement of the pixel point relative to the principal point of the imaging plane is shown, and f is the distance between the imaging planes of the cameras.
As a further improvement of the above, in step S3,
Δθt＝ξ(θt-θt-Δt-ωt-Δtdelta t) is used as an estimation limit error value of a course angle at the time t, and xi is an error coefficient;
defining a maximum braking deceleration a of said vehiclemax(ii) a A minimum safe running distance L of the vehicleminThe calculation formula of (2) is as follows:
defining the brake reaction time as T0Then reaction distance L0：
L0＝vtT0
The driving safety distance is as follows:
Ls＝η(L0+Lmin)
wherein L issEta is the safety factor for the safe driving distance.
As a further improvement of the above scheme, in step S4, the method for calculating the warning probability includes the following steps:
if the high probability collision region does not coincide with the line segment L0R0Intersecting, so that the collision probability of the vehicle is 0;
if the high probability of colliding the region with L0R0And intersecting, calculating the center distance R of the intersection part closest to the fan-shaped center point N:
the early warning probability is as follows:
and P is the early warning probability.
As a further improvement of the above scheme, in step S5, a low risk early warning probability P is defined0And a high risk early warning probability P1；
If P is less than or equal to P0If so, not prompting collision and prompting normal driving;
if P0＜P≤P1If so, carrying out low-level risk collision prompt and prompting prudent driving;
if P > P1And then, carrying out high-grade risk collision prompt and prompting to brake and stop.
The invention also provides an early warning device applying any one of the mobile camera ranging early warning methods, which comprises the following steps:
a camera unit for continuously imaging a range directly in front of a vehicle to obtain front traveling state observation data of the vehicle;
the logic synthesis unit is used for calculating a current estimation value and a subsequent moment initial value of the vehicle running state variable according to the current running state observation data; the logic synthesis unit is further configured to segment an outline polygon and an angular point which are recognized as the same obstacle from a video frame image shot by the vehicle, project and transform the angular point to a normalized imaging plane according to an offline calibration result of a video camera of the vehicle to obtain an image point and a pixel coordinate of the angular point in the normalized imaging plane, select two points with the minimum and the maximum pixel abscissa in the pixel coordinate as a feature point of the obstacle, and finally obtain a ground coordinate value of the obstacle according to a position relationship between the obstacle and the vehicle; the logic synthesis unit is further used for firstly determining the value range of the real heading angle value of the vehicle, then determining the safe driving distance of the vehicle, and finally calculating the probable collision area of the vehicle according to the current estimated value, the initial value at the subsequent moment and the safe driving distance; the logic synthesis unit is also used for judging whether a characteristic line segment formed by the barrier geodetic coordinate values is a point in the high-probability collision region, calculating a central distance value which is in the high-probability collision region and is closest to the center of the high-probability collision region, and finally calculating the early warning probability according to the central distance value; and
and the early warning prompting unit is used for early warning the vehicle according to the early warning probability.
The mobile camera ranging early warning method and the early warning device have the following beneficial effects:
the mobile camera ranging early warning method comprises the steps of obtaining observation data of a current driving state and estimation data of a historical driving state through a vehicle-mounted camera, carrying out extended Kalman filtering calculation to obtain an estimation value, and calculating the estimation value to obtain an initial value; extracting barrier feature points from the video frame image and preprocessing the barrier feature points; calculating and obtaining a geodetic coordinate value of the obstacle according to the vehicle running track and the course angle data; calculating a vehicle safety distance according to the current vehicle data; calculating a vehicle dangerous area according to the vehicle safety distance and the heading angle error predicted value; obtaining early warning probability through the intersection relation between the geodetic coordinate value of the obstacle and the dangerous area of the vehicle; and comparing the early warning probability with the preset probability to obtain a corresponding early warning grade, and issuing early warning information when the early warning grade is higher. The estimated value and the initial value are calculated through the extended Kalman filtering, the vehicle state can be estimated in advance, the response sensitivity is improved, the earth coordinate value of the obstacle is calculated according to the vehicle running track and the navigation angle, the obstacle relative judgment is facilitated, the judgment accuracy is improved, the vehicle danger area is calculated according to the vehicle safety distance and the course angle error predicted value, and the judgment accuracy is high.
The invention calculates the position of the obstacle shot by the mobile camera, predicts the collision probability by combining the vehicle with the approximate collision area, and has accurate early warning function for the safe driving of the vehicle; moreover, the method can reduce the probability of collision with the obstacle in the driving process of the vehicle, can be used for reference in automatic driving of the vehicle, and fills the blank of the prior art.
The beneficial effect of the early warning device of the invention is the same as that of the above-mentioned mobile camera ranging early warning method, and the detailed description is omitted here.
Drawings
Fig. 1 is a flowchart of a mobile camera ranging early warning method in embodiment 1 of the present invention;
fig. 2 is a schematic view of a vehicle motion model in a mobile camera ranging early warning method in embodiment 1 of the present invention;
fig. 3 is a schematic diagram of a process of preprocessing an obstacle feature point in the mobile camera ranging early warning method in embodiment 1 of the present invention; the method comprises the following steps of (a) original image, (b) foreground separation, (c) regularized contour, (d) normalization projection, (e) end point selection, (f) one-dimensional processing;
fig. 4 is a schematic diagram of a process of shooting feature points of an obstacle in the mobile camera ranging early warning method in embodiment 1 of the present invention;
fig. 5 is a schematic diagram illustrating a relationship between an object point P and an image point thereof and a motion trajectory of a vehicle in the mobile camera ranging early warning method in embodiment 1 of the present invention;
fig. 6 is a schematic diagram illustrating calculation of a high probability collision region and a collision probability in the mobile camera ranging early warning method in embodiment 1 of the present invention;
fig. 7 is a block diagram of the early warning apparatus in embodiment 2 of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Example 1
Referring to fig. 1-6, the present embodiment provides a mobile camera ranging early warning method, which first extracts historical driving state estimation data and current driving state observation data from a storage unit and performs extended kalman filtering to obtain a current estimation value, and a subsequent time initial value (i.e., a predicted value) obtained according to the current estimation value. Meanwhile, the pixel coordinate values of the feature points of the obstacles are extracted from the vehicle-mounted video target frame. And then substituting the vehicle running track and the course angle data into a binocular vision model to calculate and solve the geodetic coordinate value of the obstacle. And calculating the safe distance of the vehicle according to the current vehicle data. And calculating the dangerous area and the collision probability of the vehicle according to the safe distance, the error-containing predicted value of the course angle and the position of the obstacle, and issuing early warning information. In this embodiment, the moving camera ranging early warning method is mainly implemented by the following steps, specifically steps S1-S5.
Step S1: and extracting historical driving state estimation data and current driving state observation data of the vehicle, and calculating a current estimation value and a subsequent moment initial value of the vehicle driving state variable. The vehicle driving state observed value at the current time t is read by the vehicle sensing unit. The historical values of the observed value and the estimated value of the running state can be read from the storage unit, and the state estimated value of the last time t-delta t is xt-θt,yt-Δt,θt-Δt,vt-Δt,at-Δt,ωt-ΔtAnd corresponds to the observed values one by one.
In order to achieve both accuracy and real-time performance, a system equation of state variables can be designed by using a constant longitudinal acceleration-constant angular velocity-piecewise linear motion vehicle motion model, and the vehicle motion model corresponding to the time from t-delta t to t is shown in fig. 2.
Thus, the system equation for the current estimate is:
in the formula (x)t,yt) For the geodetic coordinate value, v, of the vehicle in the current estimatetFor the vehicle speed in the current estimate, atFor the vehicle longitudinal acceleration, ω, in the current estimatetDetermining the vehicle yaw angular velocity in the current estimation value, wherein epsilon is system noise, and the current estimation value is the sum of the current initial value and the system noise;
the observation equation of the current observation value is:
in the formula (X)t,Yt) Is the vehicle geodetic coordinate value theta in the current observed valuetIs the heading angle, V, of the vehicletFor the vehicle speed in the current observation, AtFor the longitudinal acceleration, Ω, of the vehicle in the current observationtAnd d, calculating the vehicle yaw angular velocity in the current observed value, wherein delta is the observation noise, and the current observed value is the sum of the current estimated value and the observation noise. In this embodiment, both epsilon and delta are noise, and are random variables with fixed gaussian distribution, and the standard deviation is determined according to experience and actual performance of the sensor respectively. And (4) obtaining an estimated value of the vehicle running state variable at the time t through an extended Kalman filtering process.
Step S2: the method comprises the steps of firstly segmenting an outline polygon and an angular point which are recognized as the same obstacle from a video frame image shot by a vehicle, then projecting and converting the angular point to a normalized imaging plane according to an off-line calibration result of a video camera of the vehicle to obtain an image point and a pixel coordinate of the angular point in the normalized imaging plane, then selecting two points with the minimum and the maximum pixel abscissa in the pixel coordinate as a feature point of the obstacle, and finally obtaining a ground coordinate value of the obstacle according to the position relationship between the obstacle and the vehicle.
In this embodiment, to obtain the feature points capable of accurately replacing the obstacles, it is first necessary to use a foreground-background separation algorithm to view the feature pointsContour polygon divided into same obstacles in frequency frame image and its corner Ci(i ═ 1,2, …). Then according to the offline calibration result of the video camera, C is obtainediTransforming the projection into a normalized imaging plane with the column direction perpendicular to the automobile chassis and the row direction parallel to the automobile wheel axle to obtain CiImage point in normalized imaging plane and its pixel coordinate Di(uDi,vDi). Then, in order to ensure the real-time performance of the calculation, the calculation model is simplified by only selecting DiTwo points L (u) with minimum and maximum horizontal coordinates of middle pixelL,vL),R(uR,vR) As the characteristic point of the obstacle, change of the original image of the R point at time t- Δ t and time t L is ignored, and only the relative positional relationship between the obstacle and the vehicle is further discussed in a two-dimensional plane (ground plane). Fig. 3 illustrates the above image preprocessing process, and fig. 4 shows a two-dimensional process in which the camera takes the same obstacle twice before and after the moving country.
And (3) imaging a static point P in the geodetic coordinate system in the vehicle-mounted video frame at the time t-delta t and the time t respectively, and calculating the displacement of the point P at the time t relative to the optical center of the camera or the geometric center of the vehicle and the geodetic coordinate value thereof according to the pixel coordinates of the image point and the estimated value of the vehicle trajectory at the time t-delta t and the time t. The geometrical relationship is shown in fig. 5.
FIG. 5 illustrates a vehicle coordinate system at time t- Δ t, which may be related to the geodetic coordinate system by xt-Δt,yt-Δt,θt-ΔtCoordinate transformation is carried out, and theta in the graph is equal to thetat-θt-Δt，
up1,up2The displacement (unit: pixel length) of the pixel point relative to the principal point of the imaging plane is shown, and f is the camera image plane distance (unit: pixel length). Therefore, the calculation formula of the obstacle geodetic coordinate value is as follows:
the coordinates of the point P in the vehicle coordinate system and further in the geodetic coordinate system at the time t-delta t can be obtained through the formula.
The calculation method is applied to the one-dimensional characteristic points L and R to obtain the original image L of the points L and R0,R0(i.e., the line segment into which the obstacle is abstracted) in the geodetic coordinate system0(xL0,yL0),R0(xR0,yR0)。
Step S3: the method comprises the steps of firstly determining the value range of a real value of a course angle of a vehicle, then determining the driving safety distance of the vehicle, and finally calculating the probable collision area of the vehicle according to a current estimated value, a preliminary calculated value at a subsequent moment and the driving safety distance.
In this embodiment, in order to calculate the area where a collision may still occur when the vehicle is currently and immediately applying an emergency brake, a value range of a true value of the heading angle needs to be determined first. Because the standard deviation and the mean deviation of the probability distribution of the course angle estimated value are unknown, the error range is estimated by considering the difference value of the initial value and the estimated value, specifically, the difference value of delta theta is usedt＝ξ(θt-θt-Δt-ωt-ΔtΔ t) as the estimated limit error value of the heading angle at the time t, where ξ is an error coefficient, and may be taken as a certain value empirically, for example, ξ is 3.
Secondly, a safe distance needs to be determined. An estimated value of the vehicle running speed at time t is known as vtAccording to the vehicle performance parameters, the maximum braking deceleration a can be obtainedmaxThen the minimum safe driving distance LminComprises the following steps:
assuming a human braking response time of T0Then reaction distance L0：
L0＝vtT0
Safety distance LsComprises the following steps:
Ls＝η(L0+Lmin)
wherein L issFor driving a safe distance, η is a safety factor, and appropriate values are empirically selected, such as: η is 1.2.
From the vehicle geodetic coordinate estimate xt,ytCourse angle estimate θtEstimate the limit error Δ θtA safety distance LsThe approximate probability collision zone can be calculated: is defined by (x)t,yt) Centered on LsIs a radius, symmetrical to thetatDirectional ray, angular range 2 delta thetatIs the approximate impact area, i.e., the sector area in fig. 6.
Step S4: judging whether a feature line segment formed by the geodetic coordinate values of the obstacle has a point falling in the high probability collision area or not, calculating a central distance numerical value of a point falling in the high probability collision area and closest to the center of the high probability collision area, and finally calculating the early warning probability according to the central distance numerical value. In the present embodiment, the geodetic coordinates L are determined from the feature points of the obstacle0(xL0,yL0),R0(xR0,yR0) The characteristic line segment L can be determined0R0Whether a point is in the approximate collision area and a point N (x) closest to the circle center of the fan-shaped areaN,yN) And calculating the collision probability. This is done:
if the high probability collision area does not contact the line segment L0R0Intersecting, the collision probability of the vehicle is 0;
if the high probability of colliding the region with L0R0And intersecting, calculating the center distance R of the intersection part closest to the fan-shaped center point N:
the early warning probability is as follows:
and P is the early warning probability.
Step S5: and early warning the vehicle according to the early warning probability. In this embodiment, a low risk early warning probability P is preset according to the actual driving data of the vehicle0And a high risk early warning probability P1And comparing the early warning probability with the low risk early warning probability and the high risk early warning probability respectively, and obtaining corresponding early warning levels according to a preset scheme. Therefore, according to the calculated early warning probability P, compared with the preset probability, the corresponding early warning grade can be obtained:
if P is less than or equal to P0If so, not prompting collision and prompting normal driving;
if P0＜P≤P1If so, carrying out low-level risk collision prompt and prompting prudent driving;
if P > P1And then, carrying out high-grade risk collision prompt and prompting to brake and stop.
In summary, compared with the existing vehicle collision prediction technology, the mobile camera ranging early warning method of the embodiment has the following advantages:
the mobile camera ranging early warning method comprises the steps of obtaining observation data of a current driving state and estimation data of a historical driving state through a vehicle-mounted camera, carrying out extended Kalman filtering calculation to obtain an estimation value, and calculating the estimation value to obtain an initial value; extracting barrier feature points from the video frame image and preprocessing the barrier feature points; calculating and obtaining a geodetic coordinate value of the obstacle according to the vehicle running track and the course angle data; calculating a vehicle safety distance according to the current vehicle data; calculating a vehicle dangerous area according to the vehicle safety distance and the heading angle error predicted value; obtaining early warning probability through the intersection relation between the geodetic coordinate value of the obstacle and the dangerous area of the vehicle; and comparing the early warning probability with the preset probability to obtain a corresponding early warning grade, and issuing early warning information when the early warning grade is higher. The estimated value and the initial value are calculated through the extended Kalman filtering, the vehicle state can be estimated in advance, the response sensitivity is improved, the earth coordinate value of the obstacle is calculated according to the vehicle running track and the navigation angle, the obstacle relative judgment is facilitated, the judgment accuracy is improved, the vehicle danger area is calculated according to the vehicle safety distance and the course angle error predicted value, and the judgment accuracy is high.
The method calculates the position of the obstacle shot by the mobile camera, predicts the collision probability by combining the vehicle with the approximate collision area, and has an accurate early warning function on the safe driving of the vehicle; moreover, the method can reduce the probability of collision with the obstacle in the driving process of the vehicle, can be used for reference in automatic driving of the vehicle, and fills the blank of the prior art.
Example 2
Referring to fig. 7, the present embodiment provides an early warning device, which can apply the distance measuring and early warning method in embodiment 1, and specifically includes a camera unit, a control unit, a wireless communication module, a storage unit, a logic synthesis unit, and an early warning module. The camera shooting unit realizes continuous imaging of a range right in front of the device, the control unit realizes a control algorithm and interacts data with each unit, the wireless communication module realizes a communication function between the devices, the storage unit stores historical data of target variables, the logic integration unit realizes running state calculation, video ranging calculation and running early warning probability calculation, and the early warning prompting module realizes early warning.
The camera unit is used for continuously imaging the range right in front of the vehicle to obtain the observation data of the front running state of the vehicle. And the logic synthesis unit is used for calculating the current estimated value and the initial value of the subsequent moment of the vehicle running state variable according to the current running state observation data. The logic synthesis unit is also used for segmenting an outline polygon and an angular point which are recognized as the same obstacle from a video frame image shot by a vehicle, then projecting and converting the angular point to a normalized imaging plane according to an offline calibration result of a video camera of the vehicle to obtain an image point and a pixel coordinate of the angular point in the normalized imaging plane, then selecting two points with the minimum and the maximum pixel horizontal coordinates in the pixel coordinate as feature points of the obstacle, and finally obtaining a ground coordinate value of the obstacle according to the position relationship between the obstacle and the vehicle. The logic synthesis unit is also used for firstly determining the value range of the true value of the course angle of the vehicle, then determining the safe driving distance of the vehicle, and finally calculating the probable collision area of the vehicle according to the current estimated value, the initial value at the subsequent moment and the safe driving distance. The logic synthesis unit is also used for judging whether a characteristic line segment formed by the barrier geodetic coordinate values is a point in the high probability collision area or not, calculating a circle center distance value which is in the high probability collision area and is closest to the circle center of the high probability collision area, and finally calculating the early warning probability according to the circle center distance value. And the early warning prompting unit is used for early warning the vehicle according to the early warning probability. The wireless communication module is mainly used for receiving and transmitting wireless signals, can perform a communication function with a traffic command center, and can also perform signal transmission with alarm equipment, a mobile phone and the like, which is not described herein in detail.
Example 3
The present embodiments provide a computer terminal comprising a memory, a processor, and a computer program stored on the memory and executable on the processor. The steps of the moving camera ranging early warning method in embodiment 1 can be realized when the processor executes the program.
When the vehicle mobile camera ranging early warning method is applied, the vehicle mobile camera ranging early warning method can be applied in a software mode, for example, a program designed to run independently is installed on a computer terminal, and the computer terminal can be a computer, a smart phone, a control system and other Internet of things equipment. The vehicle moving camera shooting distance measuring early warning method can also be designed into an embedded running program and installed on a computer terminal, such as a single chip microcomputer.
Example 4
The present embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by the processor, the steps of the moving camera ranging early warning method in embodiment 1 can be realized. When the vehicle moving camera ranging early warning method is applied, the vehicle moving camera ranging early warning method can be applied in a software mode, for example, a program which is designed to be a computer readable storage medium and can run independently is designed, the computer readable storage medium can be a U disk which is designed to be a U shield, and the U disk is designed to be a program which starts the whole method through external triggering.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (10)
1. A mobile camera ranging early warning method is characterized by comprising the following steps:
s1: extracting historical driving state estimation data and current driving state observation data of the vehicle, and calculating a current estimation value and a subsequent moment initial value of a vehicle driving state variable;
s2: firstly, segmenting an outline polygon and an angular point which are recognized as the same obstacle from a video frame image shot by the vehicle, then projecting and converting the angular point to a normalized imaging plane according to an offline calibration result of a video camera of the vehicle to obtain an image point and a pixel coordinate of the angular point in the normalized imaging plane, then selecting two points with the minimum and the maximum pixel horizontal coordinates in the pixel coordinate as feature points of the obstacle, and finally obtaining a ground coordinate value of the obstacle according to the position relationship between the obstacle and the vehicle;
s3: firstly, determining the value range of the true value of the course angle of the vehicle, then determining the safe driving distance of the vehicle, and finally calculating the probable collision area of the vehicle according to the current estimated value, the initial value of the subsequent moment and the safe driving distance;
s4: judging whether a characteristic line segment formed by the barrier geodetic coordinate values has a point falling in the high-probability collision region or not, calculating a central distance value of a point which is in the high-probability collision region and is closest to the circle center of the high-probability collision region, and finally calculating an early warning probability according to the central distance value; and
s5: and early warning the vehicle according to the early warning probability.
2. The mobile camera ranging early warning method as claimed in claim 1, wherein in the step S5, a low risk early warning probability and a high risk early warning probability are preset according to actual driving data of the vehicle, and then the early warning probabilities are respectively compared with the low risk early warning probability and the high risk early warning probability to obtain corresponding early warning levels according to a preset scheme.
3. The mobile camera ranging pre-warning method as claimed in claim 1, wherein in the step S1, the current estimated value and the subsequent time initial value are calculated by a system equation of constant longitudinal acceleration-constant angular velocity-piecewise linear motion vehicle motion model design state variables.
4. A mobile camera ranging pre-warning method as claimed in claim 3, wherein the system equation of the current estimation value is:
in the formula (x)t,yt) Is the vehicle geodetic coordinate value v in the current estimation valuetFor the vehicle speed in said current estimate, atFor the vehicle longitudinal acceleration, ω, in said current estimatetThe state estimation value of the last time t-delta t is x for the vehicle yaw angular velocity in the current estimation valuet-Δt,yt-Δt,θt-Δt,vt-Δt,at-Δt,ωt-ΔtEpsilon is system noise, and the current estimated value is the sum of the current initial value and the system noise;
the observation equation of the current observation value is as follows:
in the formula (X)t,Yt) Is as described inThe vehicle geodetic coordinate value theta in the front observation valuetIs the heading angle, V, of the vehicletIs the vehicle speed in the current observed value, AtIs the longitudinal acceleration, omega, of the vehicle in the current observed valuetAnd d, calculating the vehicle yaw angular velocity in the current observed value, wherein delta is the observation noise, and the current observed value is the sum of the current estimated value and the observation noise.
5. The mobile camera ranging pre-warning method as claimed in claim 4, wherein the corner point is defined as a point Ci(i ═ 1,2, …), and point C is addediThe projection is transformed to the normalized imaging plane with the column direction perpendicular to the automobile chassis of the vehicle and the row direction parallel to the automobile wheel axle of the vehicle, and the projection is subjected to feature point grabbing and one-dimensional processing; wherein the pixel coordinate is defined as a mark Di(uDi,vDi) The two feature points are defined as L (u)L,vL),R(uR,vR) L, R point original image L0,R0Position L in the geodetic coordinate system0(xL0,yL0),R0(xR0,yR0)。
6. The mobile camera ranging early warning method as claimed in claim 5, wherein the calculation formula of the obstacle geodetic coordinate value is as follows:
wherein θ is θt-θt-Δt，
up1,up2Is a principal point of a pixel point relative to an imaging planeF is the camera image plane distance.
7. The mobile camera ranging pre-warning method as claimed in claim 6, wherein in the step S3,
Δθt＝ξ(θt-θt-Δt-ωt-Δtdelta t) is used as an estimation limit error value of a course angle at the time t, and xi is an error coefficient;
defining a maximum braking deceleration a of said vehiclemax(ii) a A minimum safe running distance L of the vehicleminThe calculation formula of (2) is as follows:
defining the brake reaction time as T0Then reaction distance L0：
L0＝vtT0
The driving safety distance is as follows:
Ls＝η(L0+Lmin)
wherein L issEta is the safety factor for the safe driving distance.
8. The mobile camera ranging early warning method as claimed in claim 7, wherein in the step S4, the method for calculating the early warning probability comprises the following steps:
if the high probability collision region does not coincide with the line segment L0R0Intersecting, so that the collision probability of the vehicle is 0;
if the high probability of colliding the region with L0R0And intersecting, calculating the center distance R of the intersection part closest to the fan-shaped center point N:
the early warning probability is as follows:
and P is the early warning probability.
9. The mobile camera ranging pre-warning method as claimed in claim 8, wherein in step S5, a low risk pre-warning probability P is defined0And a high risk early warning probability P1；
If P is less than or equal to P0If so, not prompting collision and prompting normal driving;
if P0＜P≤P1If so, carrying out low-level risk collision prompt and prompting prudent driving;
if P > P1And then, carrying out high-grade risk collision prompt and prompting to brake and stop.
10. An early warning device applying the mobile camera ranging early warning method according to any one of claims 1 to 9, characterized by comprising:
a camera unit for continuously imaging a range directly in front of a vehicle to obtain front traveling state observation data of the vehicle;
the logic synthesis unit is used for calculating a current estimation value and a subsequent moment initial value of the vehicle running state variable according to the current running state observation data; the logic synthesis unit is further configured to segment an outline polygon and an angular point which are recognized as the same obstacle from a video frame image shot by the vehicle, project and transform the angular point to a normalized imaging plane according to an offline calibration result of a video camera of the vehicle to obtain an image point and a pixel coordinate of the angular point in the normalized imaging plane, select two points with the minimum and the maximum pixel abscissa in the pixel coordinate as a feature point of the obstacle, and finally obtain a ground coordinate value of the obstacle according to a position relationship between the obstacle and the vehicle; the logic synthesis unit is further used for firstly determining the value range of the real heading angle value of the vehicle, then determining the safe driving distance of the vehicle, and finally calculating the probable collision area of the vehicle according to the current estimated value, the initial value at the subsequent moment and the safe driving distance; the logic synthesis unit is also used for judging whether a characteristic line segment formed by the barrier geodetic coordinate values is a point in the high-probability collision region, calculating a central distance value which is in the high-probability collision region and is closest to the center of the high-probability collision region, and finally calculating the early warning probability according to the central distance value; and
and the early warning prompting unit is used for early warning the vehicle according to the early warning probability.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111150015.2A CN113753041B (en) | 2021-09-29 | 2021-09-29 | Mobile camera ranging early warning method and early warning device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111150015.2A CN113753041B (en) | 2021-09-29 | 2021-09-29 | Mobile camera ranging early warning method and early warning device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113753041A true CN113753041A (en) | 2021-12-07 |
| CN113753041B CN113753041B (en) | 2023-06-23 |
Family
ID=78798145
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111150015.2A Active CN113753041B (en) | 2021-09-29 | 2021-09-29 | Mobile camera ranging early warning method and early warning device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113753041B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117922559A (en) * | 2024-03-07 | 2024-04-26 | 南通职业大学 | Intelligent road early warning system for automobile |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112349144A (en) * | 2020-11-10 | 2021-02-09 | 中科海微(北京)科技有限公司 | Monocular vision-based vehicle collision early warning method and system |
| US20210188263A1 (en) * | 2019-12-23 | 2021-06-24 | Baidu International Technology (Shenzhen) Co., Ltd. | Collision detection method, and device, as well as electronic device and storage medium |
-
2021
- 2021-09-29 CN CN202111150015.2A patent/CN113753041B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20210188263A1 (en) * | 2019-12-23 | 2021-06-24 | Baidu International Technology (Shenzhen) Co., Ltd. | Collision detection method, and device, as well as electronic device and storage medium |
| CN112349144A (en) * | 2020-11-10 | 2021-02-09 | 中科海微(北京)科技有限公司 | Monocular vision-based vehicle collision early warning method and system |
Non-Patent Citations (1)
| Title |
|---|
| 陈友荣;万锦昊;陈俊洁;任条娟;: "基于车车通信的车辆防碰撞算法", 电信科学, no. 11, pages 38 - 47 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117922559A (en) * | 2024-03-07 | 2024-04-26 | 南通职业大学 | Intelligent road early warning system for automobile |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113753041B (en) | 2023-06-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3779922B1 (en) | Method for estimating driving road and driving road estimation system | |
| CN109927719B (en) | Auxiliary driving method and system based on obstacle trajectory prediction | |
| CN112154455B (en) | Data processing method, equipment and movable platform | |
| US9714034B2 (en) | Vehicle control device | |
| JP3822770B2 (en) | Vehicle front monitoring device | |
| US11898855B2 (en) | Assistance control system that prioritizes route candidates based on unsuitable sections thereof | |
| WO2019213287A1 (en) | Trailer detection and autonomous hitching | |
| CN110884490B (en) | Method and system for judging vehicle intrusion and assisting driving, vehicle and storage medium | |
| JP2003536096A (en) | Tracking map generator | |
| US20070043502A1 (en) | System for and method of detecting a collision and predicting a vehicle path | |
| CN112106065A (en) | Predicting the state and position of an observed vehicle using optical tracking of wheel rotation | |
| CN114442101B (en) | Vehicle navigation method, device, equipment and medium based on imaging millimeter wave radar | |
| CN112498347A (en) | Method and apparatus for real-time lateral control and steering actuation evaluation | |
| US11042160B2 (en) | Autonomous driving trajectory determination device | |
| US11326889B2 (en) | Driver assistance system and control method for the same | |
| Valldorf et al. | Advanced Microsystems for Automotive Applications 2007 | |
| CN111959482B (en) | Autonomous driving apparatus and method | |
| JP2005258941A (en) | Device for detecting obstacle | |
| CN113900443B (en) | Unmanned aerial vehicle obstacle avoidance early warning method and device based on binocular vision | |
| CN113753041B (en) | Mobile camera ranging early warning method and early warning device | |
| KR20200140979A (en) | Method, Apparatus for controlling vehicle, and system including it | |
| US20240067218A1 (en) | Lane change determination for vehicle on shoulder | |
| CN113022593B (en) | Obstacle processing method and device and traveling equipment | |
| CN114084133B (en) | Method and related device for determining following target | |
| CN115082562A (en) | External parameter calibration method, device, equipment, server and vehicle-mounted computing equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240627 Address after: 230051, intersection of Jinxiu Avenue and Susong Road, Economic Development Zone, Hefei City, Anhui Province, Tsinghua Qidi Science and Technology City. Area 2, 1st Floor, Annex Building 1, Robot Industry Base Patentee after: Hefei Tianlu Aviation Technology Co.,Ltd. Country or region after: China Address before: Tunxi road in Baohe District of Hefei city of Anhui Province, No. 193 230009 Patentee before: Hefei University of Technology Country or region before: China |
|
| TR01 | Transfer of patent right |