WO2023082850A1

WO2023082850A1 - Pedestrian trajectory prediction method and apparatus, and storage medium

Info

Publication number: WO2023082850A1
Application number: PCT/CN2022/120380
Authority: WO
Inventors: 韩佳琪; 崔茂源; 吕颖; 高延熹; 孔德宝; 吕铮; 董小瑜
Original assignee: 中国第一汽车股份有限公司
Priority date: 2021-11-11
Filing date: 2022-09-22
Publication date: 2023-05-19
Also published as: CN113895460B; CN113895460A

Abstract

A pedestrian trajectory prediction method and apparatus, and a storage medium. The method comprises: acquiring the current position coordinates of a target pedestrian in a vehicle coordinate system of each target vehicle, and the current relative speed of the target pedestrian and each target vehicle; determining, according to the current position coordinates, the current relative speed and a pre-constructed risk feature function, a collision risk value of the target pedestrian corresponding to each target vehicle at the next moment; determining, according to the collision risk value, an avoidance probability distribution corresponding to the target pedestrian; determining an avoidance movement speed of the target pedestrian according to the current relative speed, the current position coordinates and a pre-established human-vehicle interaction model; and determining a target predicted position of the target pedestrian according to the current relative speed, the current position coordinates, the avoidance movement speed and the avoidance probability distribution.

Description

Pedestrian trajectory prediction method, device and storage medium

This application claims priority to a Chinese patent application with application number 202111333664.6 filed with the China Patent Office on November 11, 2021, the entire contents of which are incorporated herein by reference.

technical field

The embodiments of the present application relate to the technical field of intelligent vehicles, for example, to a pedestrian trajectory prediction method, device and storage medium.

Background technique

A typical automatic driving system consists of three parts: environment perception, planning and decision-making, and executive control. Pedestrian trajectory prediction is a basic task in planning and decision-making. In mixed traffic scenarios, accurate and fast prediction of the future trajectory of pedestrians will help improve the planning effectiveness and decision-making accuracy of autonomous vehicles.

In the urban road traffic environment, a variety of traffic signs and signal lights regulate the interaction between pedestrians and vehicles. Model-based prediction methods in the related art usually use this feature to predict the future trajectory of pedestrians. However, this method It needs to rely on the traffic equipment in the related technology, and the application range is limited. Also, in the absence of traffic equipment, the interaction between pedestrians and vehicles cannot be reasonably predicted.

Since pedestrian trajectory prediction is essentially a time-series problem, neural network-based prediction methods have been gradually applied in the field of autonomous driving in recent years. At present, the use of large data sets for training and learning can achieve accurate prediction of pedestrian trajectories in any scene. However, the performance superiority and versatility of this method are at the expense of model interpretability and computational speed.

Contents of the invention

Embodiments of the present application provide a method, device, and storage medium for predicting pedestrian trajectories, which can quickly and accurately predict pedestrian trajectories.

In the first aspect, the embodiment of the present application provides a pedestrian trajectory prediction method, the method comprising:

Obtaining the current position coordinates of the target pedestrian in the vehicle coordinate system of each of the multiple target vehicles, and the current relative speed between the target pedestrian and each target vehicle;

According to the current position coordinates, the current relative speed and the pre-built risk characteristic function, determine the collision risk value corresponding to the target pedestrian and each target vehicle at the next moment; wherein, the risk characteristic function is based on the sample pedestrian and The minimum distance of the sample vehicle corresponding to the sample pedestrian and the remaining time to reach the minimum distance are constructed;

determining the avoidance probability distribution corresponding to the target pedestrian according to the collision risk value;

According to the current relative speed, the current position coordinates and the pre-established human-vehicle interaction model, determine the avoidance movement speed of the target pedestrian; wherein the pre-established human-vehicle interaction model is based on the sample pedestrian's Establishing the sample position coordinates in the vehicle coordinate system of the sample vehicle corresponding to the sample pedestrian and the sample relative speed between the sample pedestrian and the sample vehicle;

A target predicted position of the target pedestrian is determined according to the current relative speed, the current position coordinates, the avoidance motion speed, and the avoidance probability distribution.

In the second aspect, the embodiment of the present application also provides a pedestrian trajectory prediction device, which includes:

An information acquisition module configured to acquire the current position coordinates of the target pedestrian in the vehicle coordinate system of each of the multiple target vehicles, and the current relative speed between the target pedestrian and each target vehicle;

The collision risk value determination module is configured to determine the collision risk value corresponding to the target pedestrian and each target vehicle at the next moment according to the current position coordinates, the current relative speed and the pre-built risk characteristic function; wherein, The risk characteristic function is constructed according to the minimum distance between the sample pedestrian and the sample vehicle corresponding to the sample pedestrian and the remaining time to reach the minimum distance;

The avoidance probability determination module is configured to determine the avoidance probability distribution corresponding to the target pedestrian according to the collision risk value;

The movement speed prediction module is configured to determine the avoidance movement speed of the target pedestrian according to the current relative speed, the current position coordinates and the pre-established human-vehicle interaction model; wherein, the pre-established human-vehicle interaction model is based on Establishing the sample position coordinates of the sample pedestrian in the vehicle coordinate system of the sample vehicle corresponding to the sample pedestrian and the sample relative speed between the sample pedestrian and the sample vehicle;

The position prediction module is configured to determine the target predicted position of the target pedestrian according to the current relative speed, the current position coordinates, the avoidance motion speed and the avoidance probability distribution.

In a third aspect, the embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the pedestrian trajectory prediction method as described in any one of the embodiments of the present application is implemented.

Description of drawings

FIG. 1 is a schematic flow chart of a pedestrian trajectory prediction method provided by an embodiment of the present application;

FIG. 2 is a schematic flowchart of a pedestrian trajectory prediction method provided by another embodiment of the present application;

Fig. 3 is a schematic diagram of a vehicle coordinate system provided by an embodiment of the present application;

FIG. 4 is a schematic flowchart of a pedestrian trajectory prediction method provided by another embodiment of the present application;

FIG. 5 is a schematic structural diagram of a pedestrian trajectory prediction device provided by an embodiment of the present application;

FIG. 6 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

Figure 1 is a schematic flow chart of a method for predicting pedestrian trajectories provided by an embodiment of the present application. This embodiment can be applied to the situation of predicting the future movement trajectories of pedestrians in the scene of automatic driving. device, the device may be implemented in the form of software and/or hardware, and the hardware may be an electronic device, for example, the electronic device may be a mobile terminal or the like.

As shown in Figure 1, the method of this embodiment includes the following steps:

S110. Obtain the current position coordinates of the target pedestrian in the vehicle coordinate system of each target vehicle, and the current relative speed between the target pedestrian and each target vehicle.

Wherein, the target pedestrian may be a pedestrian whose trajectory is to be predicted. The target vehicle may be a vehicle that can affect the change of the motion state of the target pedestrian. The vehicle coordinate system may be a planar Cartesian coordinate system constructed according to the target vehicle. The current position coordinates may be the coordinate information of the target pedestrian in the vehicle coordinate system. The current relative speed may be the relative speed of the target pedestrian relative to the target vehicle at the current moment.

It should be noted that if there is no target vehicle for the target pedestrian, the target pedestrian will move according to the original motion state. If there is at least one target vehicle for the target pedestrian, one or more target vehicles will change the motion state of the target pedestrian.

For example, in the vehicle coordinate system of each target vehicle, the current position coordinates of the target pedestrian in the vehicle coordinate system are determined. And, for each target vehicle, determine the relative speed of the target pedestrian and the target vehicle at the current moment.

S120. Determine the collision risk value corresponding to the target pedestrian and each target vehicle at the next moment according to the current position coordinates, the current relative speed and the pre-built risk characteristic function.

Wherein, the risk feature function is constructed according to the minimum distance between the sample pedestrian and the sample vehicle corresponding to the sample pedestrian and the remaining time to reach the minimum distance. The collision risk value may be a data value for a collision between the target pedestrian and the target vehicle. The sample pedestrians and sample vehicles can be samples collected from big data and used to analyze the movement changes of pedestrians.

For example, after determining the current position coordinates and the current relative speed, the current position coordinates and the current relative speed can be input into the pre-built risk characteristic function, and the collision between the target pedestrian and each target vehicle at the next moment can be obtained by calculation value at risk.

It should be noted that a risk boundary can be constructed according to the risk characteristic function, for example: within the risk boundary, the collision risk value is large, and the target pedestrian will avoid the target vehicle; outside the risk boundary, the collision risk value is small, and the target pedestrian The target vehicle will not be evasive.

S130. Determine the avoidance probability distribution corresponding to the target pedestrian according to the collision risk value.

Wherein, the avoidance probability distribution can be a binary probability distribution, which can be understood as the probability value of the target pedestrian choosing to avoid the target vehicle and the probability value of the target pedestrian choosing not to avoid the target vehicle.

For example, after determining the collision risk value of the target pedestrian with respect to each target vehicle, the probability value of the target pedestrian choosing to avoid or not avoiding each target vehicle can be calculated to form an avoidance probability distribution.

It should be noted that according to the actual situation, the greater the collision risk value, the greater the probability value of the target pedestrian choosing to avoid. Therefore, the avoidance probability distribution can be calculated according to the collision risk value.

S140. Determine the avoidance movement speed of the target pedestrian according to the current relative speed, the current position coordinates and the pre-established human-vehicle interaction model.

Among them, the pre-established human-vehicle interaction model can be a model used to predict the movement state change of the target pedestrian affected by the target vehicle, and the human-vehicle interaction model can be based on the vehicle coordinate system of the sample vehicle corresponding to the sample pedestrian The sample position coordinates in and the sample relative velocity of the sample pedestrian and the sample vehicle are established. The evasive movement speed may be the movement speed of the target pedestrian after the current relative speed has been changed after being affected by the target vehicle. If there are multiple target vehicles corresponding to the target pedestrian, the evasive movement speed may be multiple.

For example, by inputting the current relative speed and current position coordinates of the target pedestrian into the pre-established human-vehicle interaction model, the avoidance speed of the target pedestrian relative to each target vehicle can be determined.

S150. Determine a target predicted position of the target pedestrian according to the current relative speed, the current position coordinates, the avoidance motion speed, and the avoidance probability distribution.

Wherein, the predicted position of the target may be the position information of the target pedestrian at the next moment, which may be described by the coordinate information of the vehicle coordinate system corresponding to the current relative speed.

For example, according to the avoidance distribution probability, it can be determined which target vehicles the target pedestrian avoids and which target vehicles do not avoid. Therefore, the use of the avoidance motion speed can be determined through the avoidance probability distribution, and then the target predicted position of the target pedestrian can be determined using the current position coordinates as the starting point.

According to the technical solution of the embodiment of the present application, by obtaining the current position coordinates of the target pedestrian in the vehicle coordinate system of each target vehicle, and the current relative speed between the target pedestrian and each target vehicle, according to the current position coordinates, current relative speed and preset The risk characteristic function constructed determines the collision risk value corresponding to the target pedestrian and each target vehicle at the next moment to judge the collision risk, and according to the collision risk value, determines the avoidance probability distribution corresponding to the target pedestrian to judge whether the target pedestrian chooses to avoid , and furthermore, according to the current relative speed, current position coordinates and the pre-established human-vehicle interaction model, determine the avoidance movement speed of the target pedestrian, and determine the target pedestrian’s avoidance speed according to the current relative speed, current position coordinates, avoidance movement speed and avoidance probability distribution The predicted position of the target avoids the low interpretability and slow calculation speed of the model, and can quickly and accurately predict the trajectory of pedestrians.

Fig. 2 is a schematic flowchart of a pedestrian trajectory prediction method provided by another embodiment of the present application. On the basis of the above embodiments, this embodiment adds the steps of establishing a human-vehicle interaction model and constructing a risk characteristic function. The specific method Refer to the technical solution of this embodiment. Wherein, explanations of terms that are the same as or corresponding to those in the foregoing embodiments will not be repeated here.

As shown in Figure 2, the method of this embodiment includes the following steps:

S210, establishing a human-vehicle interaction model.

For example, before pedestrian trajectory prediction, a human-vehicle interaction model needs to be established in advance.

The way to establish the human-vehicle interaction model can be: according to the predefined pedestrian speed influence function, the ordinate value of the sample pedestrian in the vehicle coordinate system of each sample vehicle corresponding to the sample pedestrian and the sample relative speed of the sample pedestrian and the sample vehicle , to establish a human-vehicle interaction model.

Among them, the influence function of pedestrian speed is a piecewise linear function, which is used to describe the influence of vehicles on pedestrian speed. The sample pedestrians may be pedestrian samples collected through big data for analysis. The sample vehicle may be a vehicle corresponding to a sample pedestrian in a pedestrian avoidance scene. It should be noted that the sample vehicles may have an impact on the future movement of the sample pedestrians. The sample relative speed may be the moving speed of the sample pedestrian relative to the sample vehicle.

For example, for a sample pedestrian, determine the time t, and the position coordinates of the sample pedestrian are

The position coordinates of the sample pedestrians corresponding to the time t in the future are X _t ∈ R ² . The time series of sample pedestrian motion can be divided into observation frames and prediction frames, where t=1,...,T _obs are observation frames, and t=T _obs+1 ,...,T _perd are prediction frames. The position of the sample pedestrian at the future moment is the sampling of the motion trajectory of the sample pedestrian according to the predicted frame, specifically expressed as:

Assuming that the positions and speeds of the sample vehicles are consistent at each moment, the set of sample vehicles that can attract the attention of the sample pedestrians and affect the movement of the sample pedestrians is defined as r _t ∈ {1,…,n _v }, where n _v represents the number of sample vehicles. Then, the set of sample vehicles that may attract the attention of sample pedestrians at time t is defined as R _t ∈ {1,…,n _v }. Whether the sample pedestrian avoids the sample vehicle can be expressed by the binary variable q _t , when q _t = 0, it means that the sample pedestrian avoids the sample vehicle, otherwise, when q _t = 1, it means that the sample pedestrian does not avoid Sample the vehicle and continue to move with the current speed and direction. Use separately

Indicates the position and velocity of the i-th sample vehicle at time t, respectively expressed by

Indicates the abscissa and ordinate of the sample pedestrian in the vehicle coordinate system of the i-th sample vehicle at time t.

According to the actual scene, the closer the relative distance between the pedestrian and the vehicle, the smaller the pedestrian's speed should be, so the pedestrian speed influence function can be defined: f _u : R→R, this function can be regarded as a line about

A piecewise linear function of symmetry centered at zero. The function is obtained through learning, and in the first parameter

is linear, where n _u represents the number of grid points in the piecewise linear function, the function grid is parameterized by the maximum distance u _max , and the grid points are uniformly distributed on [0, u _max ]. The maximum distance u _max represents the maximum distance of the pedestrian's observation field of view. If the sample pedestrian avoids the i-th sample vehicle, then the speed of the sample pedestrian can be expressed as:

Among them, v _t,yield represents the movement velocity of the sample pedestrian after avoiding the sample vehicle. If the sample pedestrian does not avoid the sample vehicle, it will continue to move at the current speed.

It should be noted that the above representation of the speed of the sample pedestrians can be used as a human-vehicle interaction model.

It should also be noted that, assuming that the target pedestrian observation

Subject to a mathematical expectation X _t , the variance is

The normal distribution of , which is expressed as:

Assume that the velocity model of the sample pedestrian is a drift-free random walk model, which obeys a mathematical expectation v _t-1 and a variance of

The normal distribution of , which is expressed as:

S220. Construct a risk characteristic function.

For example, the risk feature function needs to be pre-built before pedestrian trajectory prediction.

A manner of constructing the risk characteristic function may be: constructing the risk characteristic function according to the minimum distance between the sample pedestrian and the sample vehicle corresponding to the sample pedestrian and the remaining time to reach the minimum distance.

Wherein, the minimum distance may be the minimum distance between the sample pedestrian and the sample vehicle under safe conditions. The remaining time may be the time required when the distance between the sample pedestrian and the sample vehicle reaches the minimum distance.

For example, whether the sample vehicle attracts the attention of the sample pedestrian and whether the sample pedestrian yields depends on the risk feature, and the risk feature function is determined by the minimum distance and the remaining time. When the values of the remaining time and the minimum distance are small, the collision risk between pedestrians and vehicles will be high, otherwise, the collision risk will be relatively small, so a risk characteristic function can be defined.

For example, the remaining time and the minimum distance can be determined according to the following steps:

Step 1. For each sample vehicle, determine the remaining time according to the relative distance between the sample vehicle and the sample pedestrian, and the sample relative speed between the sample vehicle and the sample pedestrian.

Wherein, the relative distance may be the distance between the sample vehicle and the sample pedestrian at the current moment. The sample relative speed may be the relative speed between the sample pedestrian and the sample vehicle at the current moment.

For example, when the relative speed of the sample is constant, the relative distance between the sample pedestrian and the sample vehicle is defined as the remaining time when the sample pedestrian starts to notice the sample vehicle until reaching the minimum distance:

It is represented as follows:

in,

is the remaining time between the sample pedestrian and the i-th sample vehicle at time t, X _t is the position coordinate of the sample pedestrian at time t,

are the position and velocity of the i-th sample vehicle at time t, respectively,

is the movement speed of the sample pedestrian. ‖·‖ ₂ means the second norm, that is, the square root of the absolute value.

Step 2: Determine the minimum distance according to the relative distance, the relative speed of the sample and the remaining time.

For example, the minimum distance between a sample pedestrian and the i-th sample vehicle is defined as

It is represented as follows:

Therefore, the risk characteristic function can be defined as follows:

in,

Indicates the remaining time at time t-1,

Indicates the minimum distance at time t-1.

In order to make the risk more variable and closer to the actual traffic situation, according to the nature of the logarithmic function, the risk characteristic function

The parameters of are in logarithmic form, and the degree of change is greater when the minimum distance and remaining time are small (less than 1), and the degree of risk is also higher. The risk characteristic function is defined based on [b ₀ ,b ₁ ]∈R ² with

A piecewise function f _β on a regular grid of equally spaced points: R ² →R, the second parameter β is a real vector and each grid point has an element and a bias term, therefore, in β The total number of elements is

Through the collected data sets of vehicles and pedestrians, the value of β is learned.

For example, the first parameter in the pedestrian speed influence function and the second parameter in the risk characteristic function are determined according to the joint distribution of pedestrian trajectory prediction.

Wherein, the first parameter may be the hyperparameter u in the pedestrian speed influence function. The second parameter may be a hyperparameter β in the risk characteristic function.

For example, according to a large number of sampled pedestrian trajectories, the joint distribution of pedestrian trajectory prediction can be obtained, and then the first parameter in the pedestrian speed influence function and the second parameter in the risk characteristic function can be calculated to optimize the pedestrian speed influence function and risk characteristic function.

For example, the joint distribution of pedestrian trajectory predictions can be determined according to the following steps:

Step 1: Obtain the movement trajectory data of the sample pedestrians.

Wherein, the motion trajectory data includes the position coordinates of the sample pedestrians at multiple moments, and the position coordinates of the sample vehicles corresponding to the sample pedestrians at multiple moments.

Step 2. According to the distribution probability of the first parameter, the distribution probability of the second parameter, the probability distribution of the position coordinates of the sample pedestrian at the next moment, the probability distribution of the sample pedestrian's choice to avoid, and the maximum relative speed of the sample pedestrian at the next moment Probabilities to determine the joint distribution for pedestrian trajectory predictions.

For example, when no sample vehicle appears within the attention range of the sample pedestrian, R _t is an empty set that is

In this case, the sample pedestrian will keep moving at the current speed without avoiding. At this time, q _t = 1; The degree of attention is proportional to the degree of risk. For i∈R _t , the distribution of r _t is expressed as follows:

Among them, p(r _t ＝i|X _t-1 ,v _t-1 ,β) means that by adjusting the position coordinate X _t-1 of the sample pedestrian at the last moment, the relative velocity v _t-1 and the second parameter β to obtain the current At time r _t = the maximum probability of i, exp(·) represents an exponential function with the natural constant e as the base, and ∑(·) represents the addition of multiple parameters in parentheses.

When the sample vehicle attracts the attention of the sample pedestrian and makes the sample pedestrian give way, that is, q _t =0, its distribution is expressed as follows:

Among them, p(q _t ＝0|X _t-1 ,v _t-1 ,r _t ,β) means that by setting the position coordinate X _t-1 of the sample pedestrian at the last moment, the relative velocity v _t-1 , the set of sample vehicles r _t and the second parameter β yield the maximum probability of q _t =0.

Compared with the selection of evasive vehicles based on relative risk characteristics, the decision of whether to give way is based on absolute risk. In order to ensure the accuracy of the first parameter u and the second parameter β, it is assumed that its prior probability is a Gaussian with zero mean and unit variance For the convenience of subsequent calculations, through reasonable derivation and simplification, the negative logarithmic form of the probability distribution is expressed as the linear relationship between the first parameter u and the second parameter β, which is expressed as follows:

Among them, α _u and α _β are two hyperparameters. By setting appropriate values, the values of multiple elements in u can be controlled between [-1,1], and the predicted results can be made more accurate. .

The first parameter u and the second parameter β can be estimated by optimizing a likelihood function. In order to make the expression of the formula more concise and clear, X, v, r, and q are respectively used to represent X _t , v _t , r t _, q _t The entire time series of these variables, such as

It represents the entire observation sequence value of X _t from t=1 to t=T _obs , let S _t =(X _t ,v _t ), using the Markov structure of the model, the joint distribution of pedestrian trajectory prediction is expressed as follows:

Among them, L(·) represents the joint probability distribution of multiple parameters in brackets, p(u) is the probability distribution of the first parameter u, p(β) is the probability distribution of the second parameter β, ∏(·) represents the parentheses Multiplying multiple parameters in ,

Indicates that the position, relative speed, sample vehicle set, binary variable q _t and first parameter u of the previous moment are adjusted to obtain the current moment

The maximum probability of , p(q _t |S _t-1 ,r _t ,β) indicates the maximum probability of obtaining q _t by setting the position, relative speed, sample vehicle set and the second parameter β at the previous moment, p(r _t | S _t-1 ,β) represents the maximum probability of obtaining r _t by adjusting the position, relative velocity and second parameter β at the previous moment,

Indicates the maximum probability of obtaining the relative speed at the current moment by adjusting the relationship between the relative speed and the speed change at the previous moment.

In order to avoid local optimization in the optimization process, the above formula is simplified, and the time series of a group of sample vehicles that do not attract the attention of sample pedestrians is defined as

when

When q _t = 1, then p(q _t = 1|t∈Q) = 1, and the speed of the sample pedestrian remains unchanged. Therefore, the likelihood term in the above formula can be simplified and expressed as follows:

Then the joint distribution of pedestrian trajectory prediction can be expressed as follows:

in,

The relative speed in is measured by the sensor, and the relative speed at the current moment can be obtained according to the relative speed and speed change at the previous moment. Therefore, the probability value of this item is 1. In the t∈Q time series,

The position coordinates and relative speed of the sample pedestrians in are measured by the sensor, because there are no sample vehicles that attract attention, and the sample pedestrians keep moving at the original speed, so the position coordinates at the next moment can be determined, so the probability of this item is 1, represented by q _c

Then the final likelihood estimate can be written as

Among them, L _c ( ) is the simplified joint probability distribution, and the model parameters can be obtained by minimizing its negative logarithm according to the principle of maximum likelihood estimation, and its negative logarithm form is expressed as follows:

Among them, l _c represents the loss function based on conditional probability, and p(q _t |S _t-1 , r _t , β) represents the method of obtaining q _t by adjusting the position, relative speed, sample vehicle set and the second parameter β at the previous moment. maximum probability.

Accordingly, the above formula can be minimized to obtain the first parameter u and the second parameter β through learning. It should be noted that the physical meaning of the first parameter u can be the reflection of pedestrians at different distances from the vehicle, and the physical meaning of the second parameter β can be the boundary value of the impact of the vehicle on pedestrians.

S230. Obtain the current position coordinates of the target pedestrian in the vehicle coordinate system of each target vehicle, and the current relative speed between the target pedestrian and each target vehicle.

S240. Determine the collision risk value corresponding to the target pedestrian and each target vehicle at the next moment according to the current position coordinates, the current relative speed and the pre-built risk characteristic function.

S250. Determine the avoidance probability distribution corresponding to the target pedestrian according to the collision risk value.

S260. Determine an avoidance movement speed of the target pedestrian according to the current relative speed, the current position coordinates, and the pre-established human-vehicle interaction model.

It should be noted that the determination of the collision risk value, avoidance probability distribution, and avoidance movement speed of the target pedestrian can be determined according to the analysis method for the sample pedestrian in S210-S220, which will not be repeated here.

S270. Determine the target predicted position of the target pedestrian according to the current relative speed, the current position coordinates, the avoidance motion speed, and the avoidance probability distribution.

For example, the target predicted location of a target pedestrian can be expressed as:

Among them, X _t represents the position of the target pedestrian at time t, X _t-1 represents the position of the target pedestrian at time t-1, q _t is a binary variable of whether the target pedestrian avoids the target vehicle, and f _u ( ) is Pedestrian speed influence function,

is the vertical coordinate of the target pedestrian in the vehicle coordinate system of each target vehicle in the target vehicle set at time t, and Δt is the step size of the sampling time.

For example, the vehicle coordinate system of the target vehicle can be established in the following way:

For each target vehicle, take the center of mass of the target vehicle as the origin of the coordinate system, take the forward direction of the target vehicle as the positive direction of the horizontal axis, and rotate the horizontal axis 90 degrees counterclockwise as the vertical axis to construct the vehicle coordinate system of the target vehicle.

For example, the target vehicle can be determined by:

Determine the vehicle within the target range centered on the target pedestrian as the vehicle to be screened; for each vehicle to be screened, if the angle between the horizontal axis direction of the vehicle coordinate system of the vehicle to be screened and the direction of travel of the target pedestrian is greater than the preset angle, Then the vehicle to be screened is determined as the target vehicle.

Wherein, the target range may be the maximum distance determined according to the pedestrian's field of vision, and vehicles exceeding the target range are not within the observation range of the target pedestrian. In addition, vehicles in front of the target pedestrian and vehicles far away from the target pedestrian are not considered. The vehicles to be screened can be all vehicles around the target pedestrian.

For example, in the vehicle coordinate system shown in Figure 3, the specific expression of T _t is as follows:

in,

Indicates the horizontal coordinates of the target pedestrian in the reference coordinate system of the i-th vehicle to be screened at time t, l is the distance from the center of mass of the vehicle to be screened to the rear of the vehicle,

Indicates the longitudinal coordinates of the target pedestrian in the vehicle coordinate system of the i-th vehicle to be screened at time t, v _t ^T indicates the transposition of the relative velocity of the target pedestrian in the vehicle coordinate system, and z is the unit of the horizontal axis in the vehicle coordinate system vector.

As an exemplary implementation of the above-mentioned embodiment, FIG. 4 is a schematic flowchart of a pedestrian trajectory prediction method provided in another embodiment of the present application.

As shown in Figure 4, the method of this embodiment includes:

Obtain the historical trajectories of the sample pedestrians, establish the human-vehicle interaction model, construct the risk characteristic function of the human-vehicle interaction, and establish the joint distribution of pedestrian trajectory prediction. According to the joint distribution of human-vehicle interaction model, risk characteristic function and pedestrian trajectory prediction, a pedestrian trajectory prediction model is established. Obtain the position and velocity information of the target pedestrian and the target vehicle, and predict the trajectory of the target pedestrian.

By establishing a risk-based human-vehicle interaction model and risk characteristic function, the target vehicle that can attract the attention of the target pedestrian and the impact on the target pedestrian are judged, and the trajectory is fitted through data-driven and the first parameter and the second parameter of the model are adjusted. Parameters, design the Markov structure to obtain the joint distribution of pedestrian trajectory prediction, use the maximum likelihood estimation method to calculate the maximum probability behavior and convert it into a predicted future trajectory, where the prediction model can be used in scenarios without traffic equipment in related technologies Predicting pedestrian trajectories is not limited by the traffic equipment information in the prediction methods in related technologies. It considers the correlation between target vehicles and target pedestrian trajectory predictions, establishes a conditional probability model, and completes the prediction of target pedestrians step by step, improving the accuracy of the prediction process. Efficiency, while increasing trajectory prediction accuracy, by combining with a data-driven approach avoids the need for auxiliary simulations or manually specified parameters in previous model-based prediction methods. Moreover, the measurement data used in this method can be obtained by the laser radar or camera mounted on the smart car. These sensors are widely mounted on the smart car and have good realizability.

In the technical solution of the embodiment of the present application, by establishing a human-vehicle interaction model and constructing a risk characteristic function, the current position coordinates of the target pedestrian in the vehicle coordinate system of each target vehicle and the current relative speed between the target pedestrian and each target vehicle are obtained , according to the current position coordinates, current relative velocity and the pre-built risk characteristic function, determine the collision risk value corresponding to the target pedestrian and each target vehicle at the next moment to judge the collision risk, and determine the corresponding collision risk value of the target pedestrian according to the collision risk value avoidance probability distribution to determine whether the target pedestrian chooses to avoid, and then, according to the current relative speed, current position coordinates and the pre-established human-vehicle interaction model, determine the avoidance movement speed of the target pedestrian, and according to the current relative speed, current position coordinates, avoidance The movement speed and avoidance probability distribution determine the target predicted position of the target pedestrian, avoiding the low interpretability of the model and the slow calculation speed, and can quickly and accurately predict the pedestrian trajectory.

5 is a schematic structural diagram of a pedestrian trajectory prediction device provided by an embodiment of the present application, which includes: an information acquisition module 310, a collision risk value determination module 320, an avoidance probability determination module 330, a movement speed prediction module 340 and a position prediction module. Module 350.

Wherein, the information acquisition module 310 is set to acquire the current position coordinates of the target pedestrian in the vehicle coordinate system of each target vehicle, and the current relative speed between the target pedestrian and each target vehicle; the collision risk value determination module 320 is set To determine the collision risk value corresponding to the target pedestrian and each target vehicle at the next moment according to the current position coordinates, the current relative speed and the pre-built risk feature function; wherein, the risk feature function is based on the sample pedestrian Constructed with the minimum distance of the sample vehicle corresponding to the sample pedestrian and the remaining time to reach the minimum distance; the avoidance probability determination module 330 is configured to determine the avoidance probability corresponding to the target pedestrian according to the collision risk value distribution; the motion speed prediction module 340 is configured to determine the avoidance motion speed of the target pedestrian according to the current relative speed, the current position coordinates and the pre-established human-vehicle interaction model; wherein, the pre-established pedestrian-vehicle The interaction model is established according to the sample position coordinates of the sample pedestrian in the vehicle coordinate system of the sample vehicle corresponding to the sample pedestrian and the sample relative speed between the sample pedestrian and the sample vehicle; the position prediction module 350 sets To determine the target predicted position of the target pedestrian according to the current relative speed, the current position coordinates, the avoidance motion speed and the avoidance probability distribution.

For example, the device further includes: a human-vehicle interaction model establishment module, configured to establish the human-vehicle interaction model; The ordinate value in the vehicle coordinate system of each sample vehicle corresponding to the sample pedestrian and the sample relative speed between the sample pedestrian and the sample vehicle are used to establish the human-vehicle interaction model; wherein, the pedestrian speed affects The function is a piecewise linear function.

For example, the device further includes: a risk characteristic function building module, configured to construct the risk characteristic function; the risk characteristic function building module, configured to The minimum distance and the remaining time to reach the minimum distance construct a risk characteristic function.

For example, the risk characteristic function building module is further configured to determine the remaining Time: determining the minimum distance according to the relative distance, the relative speed of the sample and the remaining time.

For example, the device further includes: a parameter determination module configured to determine the first parameter in the pedestrian speed influence function and the second parameter in the risk characteristic function according to the joint distribution of pedestrian trajectory prediction.

For example, the parameter determination module is also configured to acquire the motion trajectory data of the sample pedestrian, wherein the motion trajectory data includes the position coordinates of the sample pedestrian at multiple moments, and the sample vehicle corresponding to the sample pedestrian at multiple times. position coordinates at a moment; according to the distribution probability of the first parameter, the distribution probability of the second parameter, the probability distribution of the position coordinates of the sample pedestrian at the next moment, the probability distribution of the sample pedestrian's choice to avoid, and the maximum probability of the relative velocity of the sample pedestrian at the next moment, to determine the joint distribution of the trajectory prediction of the pedestrian.

For example, the device further includes: a coordinate system establishment module, configured to, for each target vehicle, take the vehicle center of mass of the target vehicle as the origin of the coordinate system, and take the forward direction of the target vehicle as the positive direction of the horizontal axis, and set The horizontal axis is rotated 90 degrees counterclockwise as the vertical axis to construct the vehicle coordinate system of the target vehicle.

For example, the device further includes: a target vehicle determination module, configured to determine vehicles within a target range centered on the target pedestrian as vehicles to be screened; for each vehicle to be screened, if the vehicle coordinates of the vehicle to be screened If the included angle between the direction of the horizontal axis of the system and the traveling direction of the target pedestrian is greater than a preset angle, the vehicle to be screened is determined as the target vehicle.

According to the technical solution of the embodiment of the present application, by obtaining the current position coordinates of the target pedestrian in the vehicle coordinate system of each target vehicle, and the current relative speed between the target pedestrian and each target vehicle, according to the current position coordinates, current relative speed and preset The risk characteristic function constructed determines the collision risk value corresponding to the target pedestrian and each target vehicle at the next moment to judge the collision risk, and according to the collision risk value, determines the avoidance probability distribution corresponding to the target pedestrian to judge whether the target pedestrian chooses to avoid , and furthermore, according to the current relative speed, current position coordinates and the pre-established human-vehicle interaction model, determine the avoidance movement speed of the target pedestrian, and determine the target pedestrian’s avoidance speed according to the current relative speed, current position coordinates, avoidance movement speed and avoidance probability distribution The predicted position of the target avoids the low interpretability of the model and the slow calculation speed, and can quickly and accurately predict the trajectory of pedestrians.

The pedestrian trajectory prediction device provided in the embodiments of the present application can execute the pedestrian trajectory prediction method provided in any embodiment of the present application, and has corresponding functional modules and beneficial effects for executing the method.

It is worth noting that the multiple units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the specific names of multiple functional units It is only for the convenience of distinguishing each other, and is not used to limit the protection scope of the embodiment of the present application.

FIG. 6 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. FIG. 6 shows a block diagram of an exemplary electronic device 40 suitable for implementing the embodiments of the present application. The electronic device 40 shown in FIG. 6 is only an example, and should not limit the functions and scope of use of this embodiment of the present application.

As shown in FIG. 6, electronic device 40 takes the form of a general-purpose computing device. Components of the electronic device 40 may include, but are not limited to: one or more processors or processing units 401, a system memory 402, and a bus 403 connecting different system components (including the system memory 402 and the processing unit 401).

Bus 403 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus structures. These architectures include, by way of example, but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect ( PCI) bus.

Electronic device 40 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 40 and include both volatile and nonvolatile media, removable and non-removable media.

System memory 402 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 404 and/or cache memory 405 . Electronic device 40 may include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 406 may be configured to read from and write to non-removable, non-volatile magnetic media (not shown in FIG. 6, commonly referred to as a "hard drive"). Although not shown in FIG. 6, a disk drive for reading and writing to removable non-volatile disks (such as "floppy disks") may be provided, as well as for removable non-volatile optical disks (such as CD-ROM, DVD-ROM or other optical media) CD-ROM drive. In these cases, each drive may be connected to bus 403 via one or more data media interfaces. System memory 402 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of various embodiments of the present application.

A program/utility 408 having a set (at least one) of program modules 407, such as but not limited to, an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include the realization of the network environment. The program module 407 generally executes the functions and/or methods in the embodiments described in this application.

The electronic device 40 may also communicate with one or more external devices 409 (such as a keyboard, pointing device, display 410, etc.), communicate with one or more devices that enable a user to interact with the electronic device 40, and/or communicate with Any device (eg, network card, modem, etc.) that enables the electronic device 40 to communicate with one or more other computing devices. Such communication may occur through input/output (I/O) interface 411 . Moreover, the electronic device 40 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN) and/or a public network such as the Internet) through the network adapter 412 . As shown, network adapter 412 communicates with other modules of electronic device 40 via bus 403 . It should be appreciated that although not shown in FIG. 6, other hardware and/or software modules may be used in conjunction with electronic device 40, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape Drives and data backup storage systems, etc.

The processing unit 401 executes a variety of functional applications and data processing by running the programs stored in the system memory 402, for example, implementing the pedestrian trajectory prediction method provided by the embodiment of the present application.

Embodiment 5 of the present application provides a storage medium containing computer-executable instructions, the computer-executable instructions are configured to execute a pedestrian trajectory prediction method when executed by a computer processor, the method comprising:

Acquiring the current position coordinates of the target pedestrian in the vehicle coordinate system of each target vehicle, and the current relative speed between the target pedestrian and each target vehicle;

The computer storage medium in the embodiments of the present application may use any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more leads, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), Erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. The computer readable storage medium may be a non-transitory computer readable storage medium.

A computer readable signal medium may include a data signal carrying computer readable program code in baseband or as part of a carrier wave. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. .

Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including - but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program codes for performing the operations of the embodiments of the present application may be written in one or more programming languages or combinations thereof, the programming languages including object-oriented programming languages—such as Java, Smalltalk, C++, including A conventional procedural programming language - such as "C" or a similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as through an Internet service provider). Internet connection).

Claims

A pedestrian trajectory prediction method, comprising:

Obtaining the current position coordinates of the target pedestrian in the vehicle coordinate system of each of the multiple target vehicles, and the current relative speed between the target pedestrian and each of the target vehicles;

According to the current position coordinates, the current relative speed and the pre-built risk characteristic function, determine the collision risk value corresponding to the target pedestrian and each target vehicle at the next moment; wherein, the risk characteristic function is based on the sample The minimum distance between the pedestrian and the sample vehicle corresponding to the sample pedestrian, and the remaining time to reach the minimum distance are constructed;

determining the avoidance probability distribution corresponding to the target pedestrian according to the collision risk value;

According to the current relative speed, the current position coordinates and the pre-established human-vehicle interaction model, determine the avoidance movement speed of the target pedestrian; The sample position coordinates in the vehicle coordinate system of the sample vehicle corresponding to the sample pedestrian, and the establishment of the sample relative speed between the sample pedestrian and the sample vehicle;

A target predicted position of the target pedestrian is determined according to the current relative speed, the current position coordinates, the avoidance motion speed, and the avoidance probability distribution.
The method according to claim 1, further comprising:

Establishing the human-vehicle interaction model;

The establishment of the human-vehicle interaction model includes:

According to the predefined pedestrian speed influence function, the ordinate value of the sample pedestrian in the vehicle coordinate system of each sample vehicle corresponding to the sample pedestrian, and the sample relative speed between the sample pedestrian and the sample vehicle, establish The human-vehicle interaction model; wherein, the pedestrian speed influence function is a piecewise linear function.
The method according to claim 1, further comprising:

Constructing the risk characteristic function;

The construction of the risk characteristic function includes:

A risk characteristic function is constructed according to the minimum distance between the sample pedestrian and the sample vehicle corresponding to the sample pedestrian, and the remaining time to reach the minimum distance.
The method according to claim 3, further comprising:

For each sample vehicle, determine the remaining time according to the relative distance between the sample vehicle and the sample pedestrian, and the sample relative speed between the sample vehicle and the sample pedestrian;

The minimum distance is determined based on the relative distance, the sample relative velocity and the remaining time.
The method of claim 2, further comprising:

The first parameter in the pedestrian speed influence function and the second parameter in the risk characteristic function are determined according to the joint distribution of pedestrian trajectory prediction.
The method according to claim 5, further comprising:

Acquiring motion trajectory data of a sample pedestrian, wherein the motion trajectory data includes position coordinates of the sample pedestrian at multiple moments, and position coordinates of a sample vehicle corresponding to the sample pedestrian at the multiple moments;

According to the distribution probability of the first parameter, the distribution probability of the second parameter, the probability distribution of the position coordinates of the sample pedestrian at the next moment, the probability distribution of the sample pedestrian's choice to avoid, and the next The maximum probability of the relative velocity at a moment is determined in the joint distribution of the pedestrian trajectory predictions.
The method according to claim 1, further comprising:

For each target vehicle, take the vehicle center of mass of each target vehicle as the origin of the coordinate system, take the forward direction of the target vehicle as the positive direction of the horizontal axis, and rotate the horizontal axis 90 degrees counterclockwise as the vertical axis to construct the Describe the vehicle coordinate system of each target vehicle.
The method according to claim 1, further comprising:

Determining that a vehicle within a target range centered on the target pedestrian is a vehicle to be screened, and there is at least one vehicle to be screened;

For each vehicle to be screened, in response to determining that the angle between the direction of the horizontal axis of the vehicle coordinate system of the vehicle to be screened and the direction of travel of the target pedestrian is greater than a preset angle, determining each vehicle to be screened as a target vehicle.
A pedestrian trajectory prediction device, comprising:

An information acquisition module, configured to acquire the current position coordinates of the target pedestrian in the vehicle coordinate system of each of the multiple target vehicles, and the current relative speed between the target pedestrian and each of the target vehicles;

The collision risk value determination module is configured to determine the collision risk value corresponding to the target pedestrian and each target vehicle at the next moment according to the current position coordinates, the current relative speed and the pre-built risk characteristic function; wherein , the risk characteristic function is constructed according to the minimum distance between the sample pedestrian and the sample vehicle corresponding to the sample pedestrian, and the remaining time to reach the minimum distance;

The avoidance probability determination module is configured to determine the avoidance probability distribution corresponding to the target pedestrian according to the collision risk value;

The movement speed prediction module is configured to determine the avoidance movement speed of the target pedestrian according to the current relative speed, the current position coordinates and the pre-established human-vehicle interaction model; wherein, the pre-established human-vehicle interaction model is based on Establishing the sample position coordinates of the sample pedestrian in the vehicle coordinate system of the sample vehicle corresponding to the sample pedestrian, and the sample relative speed between the sample pedestrian and the sample vehicle;

The position prediction module is configured to determine the target predicted position of the target pedestrian according to the current relative speed, the current position coordinates, the avoidance motion speed and the avoidance probability distribution.
A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the pedestrian trajectory prediction method according to any one of claims 1-8 is implemented.