CN113778045A

CN113778045A - Method and device for evaluating automatic driving vehicle controller

Info

Publication number: CN113778045A
Application number: CN202011242385.4A
Authority: CN
Inventors: 边学鹏; 张亮亮
Original assignee: Beijing Jingdong Qianshi Technology Co Ltd
Current assignee: Beijing Jingdong Qianshi Technology Co Ltd
Priority date: 2020-11-09
Filing date: 2020-11-09
Publication date: 2021-12-10
Anticipated expiration: 2040-11-09
Also published as: CN113778045B

Abstract

The invention discloses a method and a device for evaluating an automatic driving vehicle controller, and relates to the technical field of computers. One embodiment of the method comprises: the controller receives the planned track, the real-time running parameters of the vehicle and a control instruction generated by the controller according to the planned track and the real-time running parameters of the vehicle, and the control instruction is used for controlling the instruction execution unit to drive the vehicle to run; analyzing according to the planned track, the control instruction generated by the controller and the real-time driving parameters of the vehicle to obtain an evaluation index of the controller; and evaluating the control effect of the controller in the test scene according to the evaluation index. The implementation mode can evaluate the control effect of the upper layer controller and the lower layer controller for tracking the reference track, provides a set of standard evaluation system of the automatic driving controller for the field of automatic driving vehicles, makes up for the deficiency of relevant aspects of the market, and can be used as an efficient tool for debugging the controller or a clear index requirement for the controller.

Description

Method and device for evaluating automatic driving vehicle controller

Technical Field

The invention relates to the technical field of computers, in particular to a method and a device for evaluating an automatic driving vehicle controller.

Background

The automatic driving vehicle controller consists of an upper layer and a lower layer, wherein the upper layer controller is used for tracking a track and generating a front wheel corner and longitudinal command (a speed command, an acceleration command and a brake-accelerator percentage), and the lower layer controller is used for responding to command information output by the upper layer controller to drive a vehicle to run, so that complete closed-loop control of the vehicle is realized.

In the process of implementing the invention, the inventor finds that at least the following problems exist in the prior art: at present, a set of standard evaluation system for a controller of an automatic driving vehicle does not exist in the prior art, so that the debugging and optimization of the controller cannot be better performed by combining the control effect in the implementation process of the controller, and the efficiency of function development and debugging of the controller is reduced.

Disclosure of Invention

In view of this, embodiments of the present invention provide a method and an apparatus for evaluating an autonomous vehicle controller, where a vehicle runs in multiple scenes and related data are collected for analysis and processing, and a control effect of the controller is evaluated by determining whether various indicators of the vehicle meet requirements in each scene, so as to provide a set of standard evaluation systems for autonomous controllers in the field of autonomous vehicles, make up for deficiencies in relevant aspects of the market, and be used as an efficient tool for debugging the controller or as a clear indicator requirement for the controller.

To achieve the above object, according to one aspect of an embodiment of the present invention, there is provided a method of evaluating an autonomous vehicle controller.

A method of evaluating an autonomous vehicle controller, comprising: acquiring a planning track received by a controller, real-time running parameters of a vehicle and a control instruction generated by the controller according to the planning track and the real-time running parameters of the vehicle in the running process of an automatic driving vehicle in a preset test scene, wherein the control instruction is used for controlling an instruction execution unit to drive the vehicle to run; analyzing according to the planned track, the control instruction generated by the controller and the real-time running parameters of the vehicle to obtain an evaluation index of the controller; and evaluating the control effect of the controller in the test scene according to the evaluation index.

Optionally, the test scenario includes one or more of: the method comprises the following steps of a straight-going uniform speed scene, a small-amplitude circumambulation and turning scene of a flat road surface and a transverse slope road surface, a starting acceleration and parking deceleration scene and a common acceleration and deceleration scene under a full-load state, a slope starting and descending scene under the full-load state, an emergency stop scene, a reversing scene and a low-adhesion road driving scene.

Optionally, the control instruction generated by the controller includes: the method comprises the steps that a front wheel steering angle instruction and a speed instruction are carried out, wherein the speed instruction comprises an acceleration instruction and a brake accelerator instruction; the real-time driving parameters of the vehicle comprise: the current position, heading angle, speed, acceleration of the vehicle, and the front wheel steering angle, braking and throttle values that the vehicle actually performs in response to the control commands.

Optionally, analyzing according to the planned trajectory, the control instruction generated by the controller, and the real-time driving parameter of the vehicle includes: obtaining track parameters of each track point according to the planned track, wherein the track parameters comprise position coordinates, course angles, speeds and accelerations of the track points; determining a reference track point according to the current position of the vehicle and the reference position coordinate of the track point; and analyzing according to the track parameters of the reference track points, the control instructions generated by the controller and the real-time running parameters of the vehicle.

Optionally, the reference track point includes a time reference track point and a position reference track point, and the time reference track point is a track point whose timestamp is closest to the current vehicle driving time in the track points included in the planned track; and the position reference track point is the track point which is closest to the current position of the vehicle in the track points included in the planning track.

Optionally, the evaluation index comprises one or more of the following: the method comprises the following steps of generating a vehicle transverse and longitudinal stability, generating a vehicle transverse deviation and steady state deviation, generating a course angle following property, generating a front wheel rotation angle increment and a vehicle following property of a front wheel rotation angle, generating a longitudinal deviation and steady state deviation, generating a speed command and a vehicle response to a brake accelerator command, wherein the vehicle following property is the following property of an acceleration command relative to a reference acceleration and the following property of an actual vehicle acceleration relative to an acceleration command.

Optionally, the vehicle lateral and longitudinal stability is calculated by the following formula:

wherein, V_maxThe maximum non-side-slip velocity of the vehicle, g is the gravitational acceleration,

is the ground adhesion coefficient, R is the turning radius, alpha_maxMaximum grade at which backward turning of the vehicle does not occur, L_bIs the vehicle track width and H is the vehicle center of mass to ground height.

Optionally, the lateral deviation and the steady state deviation are calculated by the following formulas:

lat_error＝(y_vehicle-y_ref)cosθ_ref-(x_vehicle-x_ref)sinθ_ref；

wherein, lat_errorIs a lateral deviation, x_vehicle、y_vehicleIndicating the current position, x, of the vehicle_ref、y_refIndicating the position of the reference track point, theta_refThe course angle of the reference track point is shown, and the transverse deviation is the steady-state deviation after the straight line driving is stable.

Optionally, the heading angle followability is analyzed by calculating an overshoot amount and a heading angle increment, and the calculation formula is as follows:

Δθ＝θ_k-θ_k-1；

wherein, theta_maxIs the instantaneous maximum deviation value of course angle, theta_∞Indicating course angle stabilityThe state value, Delta theta, is the course angle increment, theta_k、θ_k-1Respectively, the heading angles at the time k and the time (k-1).

Optionally, the front wheel steering angle increment and the following performance of the vehicle to the front wheel steering angle are calculated by the following formula:

Δδ＝δ_k-δ_k-1；

wherein, delta_maxThe instantaneous maximum deviation value, delta, of the front wheel turning angle_∞Representing steady state values of front wheel steering angle, Δ δ being the front wheel steering angle increment, δ_k、δ_k-1The front wheel rotation angles at the time k and (k-1), respectively.

Optionally, the longitudinal deviation and the steady state deviation are calculated by the following formula:

lon_error＝(y_vehicle-y_ref)sinθ_ref+(x_vehicle-x_ref)cosθ_ref+S₂-S₁；

wherein x is_vehicle、y_vehicleIndicating the current position, x, of the vehicle_ref、y_refIndicating the position of the reference track point, theta_refIndicating the course angle of the reference track point, S₁、S₂The position coordinate S of the nearest reference track point searched by time and position is respectively referred, and S is the course value of the track point.

Optionally, the following property of the acceleration command with respect to the reference acceleration and the following property of the actual acceleration of the vehicle with respect to the acceleration command are calculated by the following formulas:

Δa＝a_k-a_k-1；

wherein, a_maxIs the instantaneous maximum deviation value of the acceleration, a_∞Indicating steady state of accelerationThe value, Δ a, is the acceleration increment, a_k、a_k-1The acceleration at times k and (k-1).

Optionally, the following performance of the speed command relative to the reference speed and the following performance of the actual speed of the vehicle relative to the speed command are calculated by the following formulas:

Δv＝v_k-v_k-1；

wherein v is_maxIs the instantaneous maximum deviation value of the velocity, v_∞Representing steady-state values of speed, Δ v being the increment of speed, v_k、v_k-1The velocities at times k and (k-1), respectively.

According to another aspect of an embodiment of the present invention, an apparatus for evaluating an autonomous vehicle controller is provided.

An apparatus for evaluating an autonomous vehicle controller, comprising: the data acquisition module is used for acquiring a planning track received by a controller, real-time running parameters of the vehicle and a control instruction generated by the controller according to the planning track and the real-time running parameters of the vehicle in the running process of the automatic driving vehicle in a preset test scene, wherein the control instruction is used for controlling an instruction execution unit to drive the vehicle to run; the index acquisition module is used for analyzing according to the planned track, the control instruction generated by the controller and the real-time running parameters of the vehicle to obtain the evaluation index of the controller; and the effect evaluation module is used for evaluating the control effect of the controller in the test scene according to the evaluation index.

Optionally, the index obtaining module is further configured to: obtaining track parameters of each track point according to the planned track, wherein the track parameters comprise position coordinates, course angles, speeds and accelerations of the track points; determining a reference track point according to the current position of the vehicle and the reference position coordinate of the track point; and analyzing according to the track parameters of the reference track points, the control instruction generated by the controller and the real-time running parameters of the vehicle.

Optionally, the reference track point comprises a time reference track point and a position reference track point, wherein the time reference track point is a track point of which the timestamp of the track point is closest to the current running time of the vehicle in the track points included in the planned track; and the position reference track point is the track point which is closest to the current position of the vehicle in the track points included in the planning track.

lat_error＝(y_vehicle-y_ref)cosθ_ref-(x_vehicle-x_ref)sinθ_ref；

Δθ＝θ_k-θ_k-1；

wherein, theta_maxIs the instantaneous maximum deviation value of course angle, theta_∞Representing a steady state value of the course angle, delta theta being the course angle increment, theta_k、θ_k-1Respectively, the heading angles at the time k and the time (k-1).

Δδ＝δ_k-δ_k-1；

lon_error＝(y_vehicle-y_ref)sinθ_ref+(x_vehicle-x_ref)cosθ_ref+S₂-S₁；

Δa＝a_k-a_k-1；

wherein, a_maxIs the instantaneous maximum deviation value of the acceleration, a_∞Representing steady-state values of acceleration, Δ a being the acceleration increment, a_k、a_k-1The acceleration at times k and (k-1).

Δv＝v_k-v_k-1；

According to yet another aspect of an embodiment of the present invention, an electronic device for evaluating an autonomous vehicle controller is provided.

An electronic device for evaluating an autonomous vehicle controller, comprising: one or more processors; a storage device to store one or more programs that, when executed by the one or more processors, cause the one or more processors to implement the method of evaluating an autonomous vehicle controller provided by embodiments of the invention.

According to yet another aspect of embodiments of the present invention, a computer-readable medium is provided.

A computer readable medium having stored thereon a computer program which, when executed by a processor, implements a method of evaluating an autonomous vehicle controller as provided by embodiments of the invention.

One embodiment of the above invention has the following advantages or benefits: the method comprises the steps that in the process that an automatic driving vehicle runs in a preset test scene, a planning track received by a controller, real-time running parameters of the vehicle and a control instruction generated by the controller according to the planning track and the real-time running parameters of the vehicle are collected, and the control instruction is used for controlling an instruction execution unit to drive the vehicle to run; analyzing according to the planned track, the control instruction generated by the controller and the real-time driving parameters of the vehicle to obtain an evaluation index of the controller; according to the technical means for evaluating the control effect of the controller in the test scene according to the evaluation indexes, the control effect of tracking the reference track of the upper layer controller and the lower layer controller can be evaluated, so that the debugging and optimization of the controller can be better carried out by combining the evaluation result of the control effect, and the efficiency of function development and debugging of the controller is reduced; the standard evaluation system of the automatic driving controller is provided for the field of automatic driving vehicles, the defects of market related aspects are made up, and the standard evaluation system can also be used as an efficient tool for debugging the controller or the clear index requirements for the controller.

Further effects of the above-mentioned non-conventional alternatives will be described below in connection with the embodiments.

Drawings

The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:

FIG. 1 is a schematic diagram of an implementation of a method for evaluating an autonomous vehicle controller according to an embodiment of the invention;

FIG. 2 is a block flow diagram of an evaluation of an autonomous vehicle controller according to an embodiment of the invention;

FIG. 3 is a schematic diagram of the main steps of a method of evaluating an autonomous vehicle controller according to an embodiment of the invention;

FIG. 4 is a schematic diagram of the main blocks of an apparatus for evaluating an autonomous vehicle controller according to an embodiment of the present invention;

FIG. 5 is an exemplary system architecture diagram in which embodiments of the present invention may be employed;

fig. 6 is a schematic block diagram of a computer system suitable for use in implementing a terminal device or server of an embodiment of the invention.

Detailed Description

Exemplary embodiments of the present invention are described below with reference to the accompanying drawings, in which various details of embodiments of the invention are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Fig. 1 is a schematic diagram of an implementation principle of a method for evaluating an autonomous vehicle controller according to an embodiment of the invention. As shown in fig. 1, the embodiment of the present invention is applicable to a scenario in which the autonomous driving vehicle evaluates the control effect of the vehicle controller when operating in a preset test scenario, and can also be used as an efficient tool for debugging the controller or a clear indicator requirement for the controller.

As shown in fig. 1, an upper controller of an automatic driving trajectory tracking control system (disposed in a controller) receives a planned trajectory transmitted by a planning module (an upstream system module of the controller), and obtains real-time driving parameters of a vehicle, including, for example, current position coordinates, heading angle, speed, and the like of the vehicle. Then, control commands for the vehicle are generated according to the planned track and the real-time running parameters of the vehicle and output to the lower layer controller, wherein the control commands comprise front wheel turning angle and longitudinal commands (speed commands or acceleration commands, or brake throttle percentage) for example. The planned track is a track to be followed in the running process of the vehicle, the track is composed of track points, and each track point contains information such as position coordinates, course angles and speeds and the like expected to be realized by the vehicle.

The input of the lower layer controller is the front wheel turning angle and longitudinal command sent by the upper layer controller, the output is a torque signal, and the torque signal is directly acted on a vehicle mechanical mechanism to control the vehicle to run according to the torque signal. And then, vehicle response information such as the front wheel rotation angle, the brake value and the throttle value which are actually responded by the vehicle is returned to the upper-layer controller so as to update the real-time running parameters of the vehicle, and the upper-layer controller can control the vehicle according to the planned track and the updated real-time running parameters of the vehicle, so that a control closed loop is formed.

During the running process of the vehicle, the automatic driving track tracking control system carries out data acquisition and data analysis so as to evaluate the control effect of the controller. The collected data includes a planned track and real-time driving parameters of the vehicle, control instructions including front wheel rotation angles and longitudinal instructions (speed instructions or acceleration instructions, or brake and accelerator percentages) generated by an upper layer controller, and vehicle response information including front wheel rotation angles, brakes and accelerator values of the actual response of the vehicle. And then analyzing the collected data to obtain preset evaluation indexes, and evaluating the control effect of the controller according to the evaluation indexes.

Fig. 2 is a block diagram of a process for evaluating an autonomous vehicle controller according to an embodiment of the present invention, and as shown in fig. 2, the evaluation process mainly includes the following two steps:

1. the method comprises the following steps of data acquisition, wherein the data acquisition is integrated in an automatic driving program and is started only when a vehicle evaluates an automatic driving vehicle controller, and the acquired data comprises the current position, the course angle, the speed and the acceleration of the vehicle, the position coordinates of a reference track point, a reference course angle, a reference speed and a reference acceleration, a front wheel rotation angle instruction, a speed instruction (an acceleration instruction and a brake accelerator instruction), the actual response front wheel rotation angle of the vehicle, the actual response brake and accelerator value of the vehicle and the like;

2. and data analysis can be carried out on line and is carried out on the basis of the collected data, the purpose is to make evaluation indexes for analyzing the driving effect of the vehicle in a specific scene, obtain the evaluation indexes according to the collected data analysis and then evaluate the control effect of the controller according to the evaluation indexes. The data analysis content mainly comprises: the method comprises the following steps of transverse deviation and steady state deviation analysis, course angle analysis, front wheel rotation angle analysis, transverse stability and longitudinal stability analysis, longitudinal deviation and steady state deviation analysis, acceleration analysis, speed analysis and brake and accelerator analysis. The specific test scenario includes, for example: the road surface evaluation method comprises the following steps of straight running and uniform speed running, small-amplitude detour running, turning (flat road surface and transverse slope road surface), starting acceleration capability (full load), parking, hill starting (full load), downhill, ordinary acceleration and deceleration, sudden stop, reversing, low-adhesion-rate road running and the like, wherein specific evaluation indexes are evaluation indexes obtained through data analysis.

In general, for unmanned vehicles or other automatic driving vehicles, lower controllers in controllers of the vehicles are integrally provided by a chassis supplier, so that for tracking control of the driving track of the vehicles, upper controllers are mainly designed, and accordingly, the invention also mainly evaluates the upper controllers. Besides the most basic requirements for the controller to be stable, accurate and fast, the robustness of the controller is also required to be strong. The evaluation method can artificially add interference to the vehicle in each scene based on the evaluation mode of the invention, and analyze the recovery effect to evaluate. And secondly, self-adaptive control can be realized for different vehicles with the same model. All vehicles with the same model and the same batch (with better vehicle consistency) can achieve the expected control effect through the same algorithm, and the evaluation method is to evaluate scenes of all vehicles in the invention, then compare the data, analyze whether the control effect is consistent and evaluate. Thirdly, the controller can realize self-adaptive control aiming at different scenes, and the following description focuses on the content, and the content is also the basis of the first two items.

Fig. 3 is a schematic diagram of main steps of a method for evaluating an autonomous vehicle controller according to an embodiment of the present invention, and as shown in fig. 3, the method for evaluating an autonomous vehicle controller according to an embodiment of the present invention mainly includes the following steps S301 to S303.

Step S301: acquiring a planning track received by a controller, real-time running parameters of a vehicle and a control instruction generated by the controller according to the planning track and the real-time running parameters of the vehicle in the running process of the automatic driving vehicle in a preset test scene, wherein the control instruction is used for controlling an instruction execution unit to drive the vehicle to run;

step S302: analyzing according to the planned track, the control instruction generated by the controller and the real-time driving parameters of the vehicle to obtain an evaluation index of the controller;

step S303: and evaluating the control effect of the controller in the test scene according to the evaluation index.

According to the steps S301 to S303, the vehicle runs in various scenes, relevant data are collected for analysis and processing, whether various indexes are met in the scenes is judged, and therefore the control effect of tracking the reference track by the upper layer controller and the lower layer controller is evaluated, a set of standard evaluation system of the automatic driving controller is provided for the field of automatic driving vehicles, the defects in the relevant aspects of the market are made up, and the standard evaluation system can be used as an efficient tool for debugging the controller or a clear index requirement for the controller.

According to one embodiment of the invention, the test scenario includes one or more of the following: the method comprises the following steps of a straight-going uniform speed scene, a small-amplitude circumambulation and turning scene of a flat road surface and a transverse slope road surface, a starting acceleration and parking deceleration scene and a common acceleration and deceleration scene under a full-load state, a slope starting and descending scene under the full-load state, an emergency stop scene, a reversing scene and a low-adhesion road driving scene.

According to another embodiment of the present invention, the control instructions generated by the controller include: the method comprises the steps that a front wheel steering angle instruction and a speed instruction are carried out, wherein the speed instruction comprises an acceleration instruction and a brake accelerator instruction; the real-time driving parameters of the vehicle comprise: the current position, heading angle, speed, acceleration of the vehicle, and the front wheel steering angle, braking and throttle values that the vehicle actually performs in response to the control commands.

According to yet another embodiment of the present invention, analyzing according to the planned trajectory, the control command generated by the controller, and the real-time driving parameter of the vehicle comprises: obtaining track parameters of each track point according to the planned track, wherein the track parameters comprise position coordinates, course angles, speeds and accelerations of the track points; determining a reference track point according to the current position of the vehicle and the reference position coordinate of the track point; and analyzing according to the track parameters of the reference track points, the control instruction generated by the controller and the real-time running parameters of the vehicle.

In the embodiment of the invention, the reference track points comprise time reference track points and position reference track points, and the time reference track points are the track points with the time stamps closest to the current running time of the vehicle in the track points included in the planned track; the position reference track point is the track point which is closest to the current position of the vehicle in the track points included in the planning track.

In an embodiment of the invention, the evaluation index comprises one or more of the following: the method comprises the following steps of generating a vehicle transverse and longitudinal stability, generating a vehicle transverse deviation and steady state deviation, generating a course angle following property, generating a front wheel rotation angle increment and a vehicle following property of a front wheel rotation angle, generating a longitudinal deviation and steady state deviation, generating a speed command and a vehicle response to a brake accelerator command, wherein the vehicle following property is the following property of an acceleration command relative to a reference acceleration and the following property of an actual vehicle acceleration relative to an acceleration command.

In some embodiments of the invention, vehicle lateral and longitudinal stability is calculated by the following formula:

is the ground adhesion coefficient, R is the turning radius, alpha_maxMaximum grade at which backward turning of the vehicle does not occur, L_bThe vehicle wheel track is defined as the vehicle wheel track, and H is the height from the vehicle center of mass to the ground;

the lateral deviation and the steady state deviation are calculated by the following formulas:

lat_error＝(y_vehicle-y_ref)cosθ_ref-(x_vehicle-x_ref)sinθ_ref；

wherein, lat_errorIs a lateral deviation, x_vehicle、y_vehicleIndicating the current position, x, of the vehicle_ref、y_refIndicating the position of the reference track point, theta_refRepresenting a reference track point course angle, and obtaining a transverse deviation as a steady-state deviation after the straight line driving is stable;

the course angle following performance is analyzed by calculating overshoot and course angle increment, and the calculation formula is as follows:

Δθ＝θ_k-θ_k-1；

wherein, theta_maxIs the instantaneous maximum deviation value of course angle, theta_∞Representing a steady state value of the course angle, delta theta being the course angle increment, theta_k、θ_k-1Respectively are the course angles at the time of k and (k-1);

the front wheel steering angle increment and the following performance of the vehicle to the front wheel steering angle are calculated by the following formulas:

Δδ＝δ_k-δ_k-1；

wherein, delta_maxThe instantaneous maximum deviation value, delta, of the front wheel turning angle_∞Representing steady state values of front wheel steering angle, Δ δ being the front wheel steering angle increment, δ_k、δ_k-1The front wheel rotation angles at the time k and the time (k-1) are respectively;

the longitudinal deviation and the steady state deviation are calculated by the following formula:

lon_error＝(y_vehicle-y_ref)sinθ_ref+(x_vehicle-x_ref)cosθ_ref+S₂-S₁；

wherein x is_vehicle、y_vehicleIndicating the current position, x, of the vehicle_ref、y_refIndicating the position of the reference track point, theta_refIndicating the course angle of the reference track point, S₁、S₂Respectively indicating the position coordinate S value of the nearest reference track point searched by time and position, wherein S is the course value of the track point;

the followability of the acceleration command with respect to the reference acceleration and the followability of the actual acceleration of the vehicle with respect to the acceleration command are calculated by the following formulas:

Δa＝a_k-a_k-1；

wherein, a_maxIs the instantaneous maximum deviation value of the acceleration, a_∞Representing steady-state values of acceleration, Δ a being the acceleration increment, a_k、a_k-1Acceleration at time k and (k-1);

the following property of the speed command with respect to the reference speed and the following property of the actual speed of the vehicle with respect to the speed command are calculated by the following formulas:

Δv＝v_k-v_k-1；

The following specifically describes the technical index analysis in each scenario of the present invention with reference to the embodiment shown in fig. 2, and the process of analyzing each index is described in detail as follows.

1. Vehicle lateral stability and longitudinal stability analysis

The invention evaluates the lateral stability, directly judges whether the vehicle carries out speed limit to prevent the sideslip phenomenon when turning, if the turning radius R is 15, the maximum non-sideslip speed is as follows:

wherein g is the acceleration of gravity, L_bIs the vehicle wheel track, H is the height from the vehicle center of mass to the ground,

is the coefficient of ground adhesion, here order

Longitudinal stability: this criterion is primarily intended to prevent the vehicle from tipping over around the rear axle or the front axle when ascending or descending a slope. This phenomenon is of course very unlikely to occur. The maximum gradient calculation formula of the automobile without backward turning is as follows:

2. lateral deviation and steady state deviation analysis

The transverse deviation refers to the transverse distance between the current position of the vehicle and the closest point of the reference track, and the index represents the accuracy degree of following the reference path, and the calculation mode is as follows:

lat_error＝(y_vehicle-y_ref)cosθ_ref-(x_vehicle-x_ref)sinθ_ref；

wherein x is_vehicle、y_vehicleIndicating the current position, x, of the vehicle_ref、y_refIndicating the position of the reference track point, theta_refRepresenting a reference track point course angle;

steady state deviation: the transverse deviation is the steady state deviation after the straight line running is stable, and the smaller the deviation and the deviation parameter is, the better.

3. Course angle analysis

The following performance of the course angle is an important index of the following performance of the posture in the running process of the vehicle, and the reference course angle and the actual course angle of the vehicle are mainly analyzed. The analysis content includes overshoot, delay, heading angle increment (smoothness), etc., and the smaller the above parameters, the better. The overshoot calculation formula is as follows:

the heading angle increment calculation formula is as follows:

Δθ＝θ_k-θ_k-1；

where θ is the heading angle, and k and (k-1) represent two adjacent time instants.

4. Front wheel cornering analysis

The front wheel turning angle is the control quantity of an upper layer controller, is the main control variable of transverse control, and the important points to be paid attention to are the increment size and smoothness of the front wheel turning angle and the following performance of a chassis to the front wheel turning angle. The smaller the increment value of the turning angle is, the better, the calculation formula is as follows:

Δδ＝δ_k-δ_k-1；

the specific indexes of the chassis for the instruction following of the front wheel turning angle include overshoot, delay and the like. Wherein,

where δ is the front wheel angle, and k and (k-1) represent two adjacent time instants.

5. Longitudinal deviation and steady state deviation analysis

The longitudinal deviation refers to the longitudinal distance between the current position of the vehicle and the closest point of the reference track, and the index represents the accuracy degree of following the reference track (including time and speed), and the calculation method is as follows:

lon_error＝(y_vehicle-y_ref)sinθ_ref+(x_vehicle-x_ref)cosθ_ref+S₂-S₁；

wherein x is_vehicle、y_vehicleIndicating the current position, x, of the vehicle_ref、y_refIndicating the position of the reference track point, theta_refIndicating the course angle of the reference track point, S₁、S₂Respectively, the position coordinate s value of the nearest reference track point found by time and position.

The longitudinal deviation deviates too much from the zero value, including both the case where the planned trajectory is not followed in the longitudinal direction and the case where the vehicle has traveled more than the desired distance, i.e. the position reached at the predetermined time is ahead of the planned one. For the condition that the vehicle does not follow the planned track, the phenomenon that the vehicle cannot reach a preset place and the like often occurs; in the case where the vehicle travels over a desired distance, it often occurs that the vehicle exceeds a predetermined place, causing the vehicle to detour in a non-timely manner, and the like. The closer to 0 both the dynamic longitudinal deviation and the post-steady-state deviation are, the better.

6. Acceleration analysis

The acceleration command is a control quantity of an upper layer controller and is a main control variable of longitudinal control, and the acceleration command is directly related to the magnitude of a chassis accelerator and a brake command and directly determines the smoothness of the vehicle during running, so the acceleration command is extremely important for controlling the acceleration. The method mainly comprises two aspects of analyzing the acceleration signal, wherein one aspect is the followability of the acceleration command of the upper layer controller relative to the reference acceleration, and the other aspect is the followability of the actual acceleration of the chassis relative to the acceleration command. The delay, adjustment time, overshoot, and increment (smoothness) of these two aspects were analyzed separately. In particular, the amount of the solvent to be used,

the overshoot is calculated by the formula:

adjusting time: the adjustment time refers to the time required for the response to reach and keep within the final value, for example, if the oscillation time is too long and the oscillation amplitude is too large, the index does not reach the standard;

the calculation formula of the acceleration increment is as follows:

Δa＝a_k-a_k-1；

where a is acceleration, k and (k-1) represent two adjacent time instants, the smaller the acceleration analysis related parameter, the better.

7. Velocity analysis

If the chassis control is in the speed control mode, the speed signal is a control quantity of the upper controller, and the analysis of the speed signal is the following performance of the speed command of the upper controller relative to the reference speed and the following performance of the actual speed of the chassis relative to the speed command. While for vehicles that are directly controlled by acceleration or torque, there is no speed command signal, the present invention directly analyzes the followability of the vehicle speed with respect to a reference speed. The main technical indexes are similar to the acceleration signal analysis, and delay, adjusting time, overshoot and increment (smoothness) are respectively analyzed. In particular, the amount of the solvent to be used,

the overshoot is calculated by the formula:

the velocity increment is calculated as:

Δv＝v_k-v_k-1；

where v is velocity, and k and (k-1) represent two adjacent time instants, the smaller the parameters involved in the velocity analysis, the better.

8. Brake throttle analysis

If the chassis control is in speed or acceleration control mode, then there is no index analysis. The analysis content relates to the generation of the brake-throttle instruction of the upper layer controller and the response of the actual brake-throttle of the chassis. The main analysis content of the generation of the brake and accelerator instruction comprises the following steps: mutual exclusivity of brake and accelerator command signals, and accuracy and reasonability of an accelerator command and a brake command. The main analysis contents of the actual response of the chassis comprise: delay of the accelerator, overshoot of the accelerator, adjustment time of the accelerator, delay of the brake, overshoot of the brake, adjustment time of the brake, and the like.

In the following, with reference to the embodiment shown in fig. 2 and the above technical index analysis, the driving effect of the vehicle in a specific scene is evaluated, and the specific contents are as follows:

1. straight-going uniform-speed scene

For a scene of straight line constant speed driving, the indexes which need to be focused mainly comprise transverse static deviation and dynamic deviation analysis, course angle signal analysis, front wheel corner analysis, longitudinal deviation and steady state deviation analysis, and evaluation is made according to the analysis results.

2. Small-amplitude circumambulation and turning scene of flat road surface and transverse slope road surface

The scene comprises a scene of small-amplitude circumambulation, a scene of turning on a flat road surface and a scene of turning on a transverse slope road surface, the indexes needing to be focused on comprise transverse static deviation and dynamic deviation analysis, course angle signal analysis, front wheel corner analysis, longitudinal deviation and steady state deviation analysis and transverse stability analysis, and evaluation is made according to the analysis results.

3. Starting acceleration and parking deceleration scene and common acceleration and deceleration scene under full-load state

For the scene, the indexes which need to be focused on are longitudinal stability analysis, longitudinal deviation and steady state deviation analysis, acceleration signal analysis, speed signal analysis, brake and accelerator signal analysis and the like, and evaluation is made according to the analysis results.

4. Hill start and down slope scenes under full load condition

For the scene, the indexes which need to be focused on are longitudinal deviation and steady state deviation analysis, acceleration signal analysis, speed signal analysis, brake and accelerator signal analysis and the like, and evaluation is made according to the analysis results.

5. Emergency stop scene

When a module has a problem or needs emergency parking, the scene can be used for evaluation. For the scene, all the technical indexes need to be analyzed, and evaluation is made according to the analysis result.

6. Scene of backing a car

For the scene, the indexes which need to pay important attention to include transverse static deviation and dynamic deviation analysis and speed signal analysis, and evaluation is made according to the analysis results.

7. Low-adhesion road driving scene

The low adhesion road is, for example, an ice road or a snow road. For the scene, all the technical indexes need to be analyzed, and evaluation is made according to the analysis result.

Fig. 4 is a schematic diagram of main blocks of an apparatus for evaluating an autonomous vehicle controller according to an embodiment of the present invention, and as shown in fig. 4, the apparatus 400 for evaluating an autonomous vehicle controller according to an embodiment of the present invention mainly includes a data acquisition module 401, an index acquisition module 402, and an effect evaluation module 403.

The data acquisition module 401 is configured to acquire a planned track received by the controller, a real-time driving parameter of the vehicle, and a control instruction generated by the controller according to the planned track and the real-time driving parameter of the vehicle in the driving process of the automatic driving vehicle in a preset test scene, where the control instruction is used for controlling the instruction execution unit to drive the vehicle to drive;

the index acquisition module 402 is configured to analyze the planned trajectory, the control instruction generated by the controller, and the real-time driving parameter of the vehicle to obtain an evaluation index of the controller;

and an effect evaluation module 403, configured to evaluate a control effect of the controller in the test scenario according to the evaluation index.

According to one embodiment of the invention, the test scenario comprises one or more of the following: the method comprises the following steps of a straight-going uniform speed scene, a small-amplitude circumambulation and turning scene of a flat road surface and a transverse slope road surface, a starting acceleration and parking deceleration scene and a common acceleration and deceleration scene under a full-load state, a slope starting and descending scene under the full-load state, an emergency stop scene, a reversing scene and a low-adhesion road driving scene.

According to still another embodiment of the present invention, the index obtaining module 402 may further be configured to: obtaining track parameters of each track point according to the planned track, wherein the track parameters comprise position coordinates, course angles, speeds and accelerations of the track points; determining a reference track point according to the current position of the vehicle and the reference position coordinate of the track point; and analyzing according to the track parameters of the reference track points, the control instruction generated by the controller and the real-time running parameters of the vehicle.

According to another embodiment of the invention, the reference track points comprise time reference track points and position reference track points, and the time reference track points are the track points with the timestamp closest to the current driving time of the vehicle in the track points included in the planned track; the position reference track point is the track point which is closest to the current position of the vehicle in the track points included in the planning track.

According to an embodiment of the invention, the evaluation index comprises one or more of the following: the method comprises the following steps of generating a vehicle transverse and longitudinal stability, generating a vehicle transverse deviation and steady state deviation, generating a course angle following property, generating a front wheel rotation angle increment and a vehicle following property of a front wheel rotation angle, generating a longitudinal deviation and steady state deviation, generating a speed command and a vehicle response to a brake accelerator command, wherein the vehicle following property is the following property of an acceleration command relative to a reference acceleration and the following property of an actual vehicle acceleration relative to an acceleration command.

lat_error＝(y_vehicle-y_ref)cosθ_ref-(x_vehicle-x_ref)sinθ_ref；

Δθ＝θ_k-θ_k-1；

Δδ＝δ_k-δ_k-1；

lon_error＝(y_vehicle-y_ref)sinθ_ref+(x_vehicle-x_ref)cosθ_ref+S₂-S₁；

Δa＝a_k-a_k-1；

Δv＝v_k-v_k-1；

Fig. 5 illustrates an exemplary system architecture 500 to which the method of evaluating an autonomous vehicle controller or the apparatus for evaluating an autonomous vehicle controller of embodiments of the invention may be applied.

As shown in fig. 5, the system architecture 500 may include

terminal devices

501, 502, 503, a network 504, and a server 505. The network 504 serves to provide a medium for communication links between the

terminal devices

501, 502, 503 and the server 505. Network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The user may use the

terminal devices

501, 502, 503 to interact with a server 505 over a network 504 to receive or send messages or the like. Various client applications, autopilot-like applications, evaluation-like applications for autopilot vehicle controllers, controller commissioning tools, etc. (by way of example only) may be installed on the

terminal devices

501, 502, 503.

The

terminal devices

501, 502, 503 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and the like.

Server 505 may be a server that provides various services, such as a back-office management server (for example only) that supports an evaluation-type application for autonomous vehicle controllers that a user browses using

terminal devices

501, 502, 503. The backend management server may perform processing such as analysis on the received data, and feed back a processing result (e.g., an index evaluation result — just an example) to the terminal device.

It should be noted that the method for evaluating the autonomous vehicle controller according to the embodiment of the present invention is generally executed by the server 505, and accordingly, the device for evaluating the autonomous vehicle controller is generally provided in the server 505.

It should be understood that the number of terminal devices, networks, and servers in fig. 5 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

Referring now to FIG. 6, a block diagram of a computer system 600 suitable for use with a terminal device or server implementing an embodiment of the invention is shown. The terminal device or the server shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.

As shown in fig. 6, the computer system 600 includes a Central Processing Unit (CPU)601 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.

In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611. The computer program performs the above-described functions defined in the system of the present invention when executed by the Central Processing Unit (CPU) 601.

It should be noted that the computer readable medium shown in the present invention can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present invention, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, a computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units or modules described in the embodiments of the present invention may be implemented by software, or may be implemented by hardware. The described units or modules may also be provided in a processor, and may be described as: a processor comprises a data acquisition module, an index acquisition module and an effect evaluation module. For example, the data acquisition module may also be described as "a module for acquiring a planned track received by the controller, real-time driving parameters of the vehicle, and control instructions generated by the controller according to the planned track and the real-time driving parameters of the vehicle during driving of the autonomous vehicle in a preset test scene, wherein the control instructions are used for controlling the instruction execution unit to drive the vehicle to drive".

As another aspect, the present invention also provides a computer-readable medium that may be contained in the apparatus described in the above embodiments; or may be separate and not incorporated into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to comprise: acquiring a planning track received by a controller, real-time running parameters of a vehicle and a control instruction generated by the controller according to the planning track and the real-time running parameters of the vehicle in the running process of the automatic driving vehicle in a preset test scene, wherein the control instruction is used for controlling an instruction execution unit to drive the vehicle to run; analyzing according to the planned track, the control instruction generated by the controller and the real-time running parameters of the vehicle to obtain an evaluation index of the controller; and evaluating the control effect of the controller in the test scene according to the evaluation index.

According to the technical scheme of the embodiment of the invention, in the process of acquiring the running of the automatic driving vehicle in the preset test scene, the controller receives the planning track, the real-time running parameters of the vehicle and the control instruction generated by the controller according to the planning track and the real-time running parameters of the vehicle, and the control instruction is used for controlling the instruction execution unit to drive the vehicle to run; analyzing according to the planned track, the control instruction generated by the controller and the real-time driving parameters of the vehicle to obtain an evaluation index of the controller; according to the technical means for evaluating the control effect of the controller in the test scene according to the evaluation indexes, the vehicle runs in various scenes and relevant data are collected for analysis and processing, whether each index in the scene is met or not is judged, and therefore the control effect of tracking the reference track by the upper layer controller and the lower layer controller is evaluated, the controller can be debugged and optimized better by combining the evaluation result of the control effect, and the efficiency of function development and debugging of the controller is reduced; the standard evaluation system of the automatic driving controller is provided for the field of automatic driving vehicles, the defects of market related aspects are made up, and the standard evaluation system can also be used as an efficient tool for debugging the controller or the clear index requirements for the controller.

The above-described embodiments should not be construed as limiting the scope of the invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions can occur, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method of evaluating an autonomous vehicle controller, comprising:

acquiring a planning track received by a controller, real-time running parameters of a vehicle and a control instruction generated by the controller according to the planning track and the real-time running parameters of the vehicle in the running process of an automatic driving vehicle in a preset test scene, wherein the control instruction is used for controlling an instruction execution unit to drive the vehicle to run;

analyzing according to the planned track, the control instruction generated by the controller and the real-time running parameters of the vehicle to obtain an evaluation index of the controller;

and evaluating the control effect of the controller in the test scene according to the evaluation index.

2. The method of claim 1, wherein the test scenario comprises one or more of:

the method comprises the following steps of a straight-going uniform speed scene, a small-amplitude circumambulation and turning scene of a flat road surface and a transverse slope road surface, a starting acceleration and parking deceleration scene and a common acceleration and deceleration scene under a full-load state, a slope starting and descending scene under the full-load state, an emergency stop scene, a reversing scene and a low-adhesion road driving scene.

3. The method of claim 1, wherein the controller generated control instructions comprise: the method comprises the steps that a front wheel steering angle instruction and a speed instruction are carried out, wherein the speed instruction comprises an acceleration instruction and a brake accelerator instruction;

the real-time driving parameters of the vehicle comprise: the current position, heading angle, speed, acceleration of the vehicle, and the front wheel steering angle, braking and throttle values that the vehicle actually performs in response to the control commands.

4. The method of claim 3, wherein analyzing according to the planned trajectory, the control instructions generated by the controller, and the real-time driving parameters of the vehicle comprises:

obtaining track parameters of each track point according to the planned track, wherein the track parameters comprise position coordinates, course angles, speeds and accelerations of the track points;

determining a reference track point according to the current position of the vehicle and the reference position coordinate of the track point;

and analyzing according to the track parameters of the reference track points, the control instructions generated by the controller and the real-time running parameters of the vehicle.

5. The method according to claim 4, wherein the reference track points comprise time reference track points and position reference track points, and the time reference track points are track points of which the timestamp is closest to the current running time of the vehicle from among the track points included in the planned track; and the position reference track point is the track point which is closest to the current position of the vehicle in the track points included in the planning track.

6. The method of claim 1, wherein the evaluation index comprises one or more of:

the method comprises the following steps of generating a vehicle transverse and longitudinal stability, generating a vehicle transverse deviation and steady state deviation, generating a course angle following property, generating a front wheel rotation angle increment and a vehicle following property of a front wheel rotation angle, generating a longitudinal deviation and steady state deviation, generating a speed command and a vehicle response to a brake accelerator command, wherein the vehicle following property is the following property of an acceleration command relative to a reference acceleration and the following property of an actual vehicle acceleration relative to an acceleration command.

7. The method of claim 6, wherein the vehicle lateral and longitudinal stability is calculated by the formula:

8. The method of claim 6, wherein the lateral deviation and steady state deviation are calculated by the following equations:

lat_error＝(y_vehicle-y_ref)cosθ_ref-(x_vehicle-x_ref)sinθ_ref；

9. The method of claim 6, wherein the course angle followability is analyzed by calculating an overshoot and a course angle delta according to the following formula:

Δθ＝θ_k-θ_k-1；

10. The method of claim 6, wherein the front wheel steer angle increment and the vehicle's following of the front wheel steer angle are calculated by the following equations:

Δδ＝δ_k-δ_k-1；

11. The method of claim 6, wherein the longitudinal deviation from steady state deviation is calculated by the following equation:

lon_error＝(y_vehicle-y_ref)sinθ_ref+(x_vehicle-x_ref)cosθ_ref+S₂-S₁；

12. The method according to claim 6, wherein the followability of the acceleration command with respect to the reference acceleration and the followability of the actual acceleration of the vehicle with respect to the acceleration command are calculated by the following formulas:

Δa＝a_k-a_k-1；

13. The method of claim 6, wherein the following of the speed command with respect to the reference speed and the following of the actual speed of the vehicle with respect to the speed command are calculated by the following equations:

Δv＝v_k-v_k-1；

wherein v is_maxIs the instantaneous maximum deviation value of the speed,v_∞representing steady-state values of speed, Δ v being the increment of speed, v_k、v_k-1The velocities at times k and (k-1), respectively.

14. An apparatus for evaluating an autonomous vehicle controller, comprising:

the data acquisition module is used for acquiring a planning track received by a controller, real-time running parameters of the vehicle and a control instruction generated by the controller according to the planning track and the real-time running parameters of the vehicle in the running process of the automatic driving vehicle in a preset test scene, wherein the control instruction is used for controlling an instruction execution unit to drive the vehicle to run;

the index acquisition module is used for analyzing according to the planned track, the control instruction generated by the controller and the real-time running parameters of the vehicle to obtain the evaluation index of the controller;

and the effect evaluation module is used for evaluating the control effect of the controller in the test scene according to the evaluation index.

15. An electronic device for evaluating an autonomous vehicle controller, comprising:

one or more processors;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-13.

16. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-13.