CN114792148A - Method and device for predicting motion trail - Google Patents

Abstract

The embodiment of the invention discloses a method and a device for predicting a motion trail, wherein the method comprises the following steps: obtaining historical track and motion attribute information of each traffic participating object corresponding to the target object and corresponding current map information; determining the track prediction characteristics corresponding to each traffic participant by using the characteristic extraction layer of the target track prediction model and the initial characteristics corresponding to each traffic participant; determining hidden random variable multi-modal probability distribution corresponding to each traffic participant by utilizing a feature extraction layer of a target track prediction model and track prediction features corresponding to each traffic participant; and determining the multi-modal predicted track corresponding to each traffic participating object by utilizing the characteristic regression layer of the target track prediction model, the track prediction characteristics corresponding to each traffic participating object and the hidden random variable multi-modal probability distribution corresponding to each traffic participating object so as to reduce the limitation of motion track prediction and better adapt to a more complex automatic driving scene.

Description

A kind of motion trajectory prediction method and device

technical field

The present invention relates to the technical field of trajectory prediction, and in particular, to a method and device for predicting a motion trajectory.

Background technique

In the field of autonomous driving, when an autonomous vehicle is driving, it needs to refer to the future motion trajectories of the surrounding traffic participants to plan the driving trajectory of the self-driving vehicle to ensure the safety of the self-driving vehicle and the traffic participants. Correspondingly, it is very important for autonomous vehicles to accurately and timely predict the future motion trajectories of traffic participants.

Considering that the future behavior of traffic participants has obvious uncertainty, that is, the future motion trajectories of traffic participants have obvious uncertainties. In related technologies, the probability of future motion trajectories of traffic participants is generally modeled by a mixed Gaussian distribution. distribution; and artificially design the modal categories and classification rules of future motion trajectories, and use the method of classification and regression to train the neural network model, so as to pass the neural network model and the historical motion trajectories of each traffic participant object and other corresponding dynamic information and information. The static information is used to predict the future running trajectories corresponding to each modal category of each traffic participant object and the corresponding probability of the future running trajectories. The other dynamic information corresponding to the traffic participant object may include other traffic participant objects and the historical track of the autonomous vehicle except the traffic participant object, and the static information may include current map information corresponding to the autonomous vehicle.

In the above process, it is necessary to manually design the number of modal categories and classification rules of the future motion trajectory predicted by the neural network model, which makes the predicted future motion trajectory limited to a certain extent, and it is difficult to adapt to more complex autonomous driving scenarios. .

SUMMARY OF THE INVENTION

The present invention provides a motion trajectory prediction method and device, so as to reduce the limitation of motion trajectory prediction and better adapt to more complex automatic driving scenarios. The specific technical solutions are as follows:

In a first aspect, an embodiment of the present invention provides a method for predicting a motion trajectory, the method comprising:

Obtain the historical trajectory and motion attribute information of each traffic participating object corresponding to the target object and the corresponding current map information;

Using the feature extraction layer of the target trajectory prediction model and the initial features corresponding to each traffic participant object, determine the trajectory prediction feature corresponding to each traffic participant object, wherein the initial feature corresponding to the traffic participant object includes: the historical trajectory and motion attributes of the traffic participant object information, and the historical trajectory and motion attribute information of other corresponding traffic participating objects and target objects, as well as the current map information;

Using the feature extraction layer of the target trajectory prediction model and the trajectory prediction features corresponding to each traffic participant, the multimodal probability distribution of the hidden random variables corresponding to each traffic participant is determined. The hidden random variable represents the behavior randomness of each traffic participant. ;

Using the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant object, and the multimodal probability distribution of hidden random variables corresponding to each traffic participant object, the multimodal prediction trajectory corresponding to each traffic participant object is determined.

Optionally, the initial features corresponding to the traffic participating objects are features arranged in chronological order, which include features of multiple historical moments corresponding to the traffic participating objects;

The step of determining the trajectory prediction feature corresponding to each traffic participant object by utilizing the feature extraction layer of the target trajectory prediction model and the corresponding initial features of each traffic participant object includes:

For each traffic participant object, using the feature extraction layer of the target trajectory prediction model, the following steps A-C are performed repeatedly for the initial feature corresponding to the traffic participant object, and the intermediate prediction feature corresponding to the traffic participant object is determined;

For each traffic participant object, the intermediate prediction features corresponding to each static object in the intermediate prediction features corresponding to the traffic participant object are fused based on the graph neural network, and the trajectory prediction feature corresponding to the traffic participant object is determined, wherein the static The object includes each static object in the current map information;

Step A: Perform nonlinear mapping on the to-be-processed feature corresponding to the traffic participant object from the feature dimension to obtain the map feature corresponding to the traffic participant object, wherein the to-be-processed feature is the initial feature or the previous time corresponding to the traffic participant object The iteratively generated intermediate prediction feature corresponding to the traffic participant object;

Step B: performing a feature aggregation operation on the mapping feature from the time dimension to obtain the aggregation feature corresponding to the traffic participant object;

Step C: Fusion of the aggregated feature and the feature of each historical moment in the feature to be processed.

Optionally, the step of using the feature extraction layer of the target trajectory prediction model and the trajectory prediction features corresponding to each traffic participant object to determine the multimodal probability distribution of the hidden random variables corresponding to each traffic participant object includes:

For each traffic participant object, use the feature extraction layer of the target trajectory prediction model and the trajectory prediction feature corresponding to the traffic participant object to determine the hidden random variable unimodal probability distribution corresponding to the traffic participant object;

For each traffic participant, using the normalized flow mapping algorithm and the unimodal probability distribution of the hidden random variable corresponding to the traffic participant, the multimodal probability distribution of the hidden random variable corresponding to the traffic participant is obtained.

Optionally, before the step of determining the trajectory prediction feature corresponding to each traffic participant object using the feature extraction layer of the target trajectory prediction model and the initial feature corresponding to each traffic participant object, the method further includes:

The process of obtaining a target trajectory prediction model by training, wherein the process includes:

Obtain the initial trajectory prediction model;

Obtain the sample training information corresponding to each sample traffic object and the sample future trajectory corresponding to each sample traffic object, wherein the sample training information corresponding to the sample traffic object includes: the sample historical trajectory and sample motion attribute information of the sample traffic object, its corresponding Sample historical trajectory, sample motion attribute information and sample static object information of sample dynamic objects;

For each sample traffic object, use the feature extraction layer of the initial trajectory prediction model and the initial sample feature corresponding to the sample traffic object to determine the sample prediction feature corresponding to the sample traffic object, wherein the initial sample corresponding to the sample traffic object The features include: sample historical trajectories and sample motion attribute information of sample traffic objects, sample historical trajectories, sample motion attribute information and sample static object information of corresponding sample dynamic objects;

For each sample traffic object, use the feature extraction layer of the initial trajectory prediction model and the sample prediction feature corresponding to the sample traffic object to determine the multimodal probability distribution of the latent random variable corresponding to the sample traffic object;

For each sample traffic object, use the feature regression layer of the initial trajectory prediction model, the sample prediction feature corresponding to the sample traffic object, and the multimodal probability distribution of the latent random variables corresponding to the sample traffic object to determine the corresponding sample traffic object. The multimodal prediction trajectory of ;

For each sample traffic object, use a preset variational algorithm to process the sample future trajectory corresponding to the sample traffic object, and obtain the variational probability distribution of the latent random variable corresponding to the sample traffic object;

For each sample traffic object, use the multimodal probability distribution of the hidden random variable corresponding to the sample traffic object and the variational probability distribution of the hidden random variable corresponding to the sample traffic object to determine the KL divergence of the hidden random variable corresponding to the sample traffic object value;

For each sample traffic object, use the multimodal predicted trajectory corresponding to the sample traffic object, the variational probability distribution of latent random variables and the sample future trajectory corresponding to the sample traffic object to determine the trajectory reconstruction loss value corresponding to the sample traffic object ;

For each sample traffic object, the KL divergence value of the hidden random variable corresponding to the sample traffic object and the trajectory reconstruction loss value corresponding to the sample traffic object are used to construct the variational lower bound of the maximum likelihood function; Whether the variational lower bound of the likelihood function is maximized;

If the variation lower bound of the constructed maximum likelihood function does not reach the maximum, adjust the model parameters of the feature extraction layer and feature regression layer of the initial trajectory prediction model, and return the traffic object for each sample, using The feature extraction layer of the initial trajectory prediction model and the initial sample feature corresponding to the sample traffic object, and the step of determining the sample prediction feature corresponding to the sample traffic object;

If the constructed variational lower bound of the maximized likelihood function is maximized, it is determined that the initial trajectory prediction model converges, and the target trajectory prediction model including the feature extraction layer and the feature regression layer is obtained.

Optionally, the feature regression layer of the target trajectory prediction model, the trajectory prediction feature corresponding to each traffic participant object, and the hidden random variable multimodal probability distribution corresponding to each traffic participant object are used to determine the corresponding traffic participant object. Steps for multimodal prediction of trajectories, including:

For each traffic participant object, sampling the multi-modal probability distribution of the hidden random variable corresponding to the traffic participant object, and obtain a plurality of hidden random variable samples corresponding to the traffic participant object;

Using the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant object, and a plurality of latent random variable samples corresponding to each traffic participant object, the multimodal prediction trajectory corresponding to each traffic participant object is determined.

In a second aspect, an embodiment of the present invention provides an apparatus for predicting a motion trajectory, the apparatus comprising:

an obtaining module, configured to obtain the historical trajectory and motion attribute information of each traffic participating object corresponding to the target object and the corresponding current map information;

The first determination module is configured to use the feature extraction layer of the target trajectory prediction model and the initial features corresponding to each traffic participant object to determine the trajectory prediction feature corresponding to each traffic participant object, wherein the initial feature corresponding to the traffic participant object includes: traffic The historical trajectory and motion attribute information of the participating objects, and the historical trajectory and motion attribute information of other corresponding traffic participating objects and target objects, and the current map information;

The second determination module is configured to use the feature extraction layer of the target trajectory prediction model and the trajectory prediction features corresponding to each traffic participant object to determine the multimodal probability distribution of hidden random variables corresponding to each traffic participant object, wherein the hidden random variables represent The randomness of behavior of each traffic participant;

The third determination module is configured to use the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant object, and the multimodal probability distribution of hidden random variables corresponding to each traffic participant object to determine each traffic participant object. Corresponding multimodal prediction trajectory.

Optionally, the initial features corresponding to the traffic participating objects are features arranged in chronological order, which include features of multiple historical moments corresponding to the traffic participating objects;

The first determination module is specifically configured to, for each traffic participant object, use the feature extraction layer of the target trajectory prediction model to perform the following steps A-C repeatedly for the initial feature corresponding to the traffic participant object, and determine the traffic participant. The intermediate prediction feature corresponding to the object;

For each traffic participant object, the intermediate prediction features corresponding to each static object in the intermediate prediction features corresponding to the traffic participant object are fused based on the graph neural network, and the trajectory prediction feature corresponding to the traffic participant object is determined, wherein the static The object includes each static object in the current map information;

Step A: Perform nonlinear mapping on the to-be-processed feature corresponding to the traffic participant object from the feature dimension to obtain the map feature corresponding to the traffic participant object, wherein the to-be-processed feature is the initial feature or the previous time corresponding to the traffic participant object The iteratively generated intermediate prediction feature corresponding to the traffic participant object;

Step B: performing a feature aggregation operation on the mapping feature from the time dimension to obtain the aggregation feature corresponding to the traffic participant object;

Step C: Fusion of the aggregated feature and the feature of each historical moment in the feature to be processed.

Optionally, the second determining module is specifically configured to, for each traffic participant object, use the feature extraction layer of the target trajectory prediction model and the trajectory prediction feature corresponding to the traffic participant object to determine the hidden traffic participant object corresponding to the traffic participant object. Unimodal probability distribution of random variables;

For each traffic participant, using the normalized flow mapping algorithm and the unimodal probability distribution of the hidden random variable corresponding to the traffic participant, the multimodal probability distribution of the hidden random variable corresponding to the traffic participant is obtained.

Optionally, the device further includes:

The training module is configured to obtain the target trajectory prediction model by training before determining the trajectory prediction feature corresponding to each traffic participant object using the feature extraction layer of the target trajectory prediction model and the initial features corresponding to each traffic participant object, wherein the The training module described above is specifically configured as

Obtain the initial trajectory prediction model;

Obtain the sample training information corresponding to each sample traffic object and the sample future trajectory corresponding to each sample traffic object, wherein the sample training information corresponding to the sample traffic object includes: the sample historical trajectory and sample motion attribute information of the sample traffic object, its corresponding Sample historical trajectory, sample motion attribute information and sample static object information of sample dynamic objects;

For each sample traffic object, use the feature extraction layer of the initial trajectory prediction model and the initial sample feature corresponding to the sample traffic object to determine the sample prediction feature corresponding to the sample traffic object, wherein the initial sample corresponding to the sample traffic object The features include: sample historical trajectories and sample motion attribute information of sample traffic objects, sample historical trajectories, sample motion attribute information and sample static object information of corresponding sample dynamic objects;

For each sample traffic object, use the feature extraction layer of the initial trajectory prediction model and the sample prediction feature corresponding to the sample traffic object to determine the multimodal probability distribution of the latent random variable corresponding to the sample traffic object;

For each sample traffic object, use the feature regression layer of the initial trajectory prediction model, the sample prediction feature corresponding to the sample traffic object, and the multimodal probability distribution of the latent random variables corresponding to the sample traffic object to determine the corresponding sample traffic object. The multimodal prediction trajectory of ;

For each sample traffic object, use a preset variational algorithm to process the sample future trajectory corresponding to the sample traffic object, and obtain the variational probability distribution of the latent random variable corresponding to the sample traffic object;

For each sample traffic object, use the multimodal probability distribution of the hidden random variable corresponding to the sample traffic object and the variational probability distribution of the hidden random variable corresponding to the sample traffic object to determine the KL divergence of the hidden random variable corresponding to the sample traffic object value;

For each sample traffic object, use the multimodal predicted trajectory corresponding to the sample traffic object, the variational probability distribution of latent random variables and the sample future trajectory corresponding to the sample traffic object to determine the trajectory reconstruction loss value corresponding to the sample traffic object ;

For each sample traffic object, the KL divergence value of the hidden random variable corresponding to the sample traffic object and the trajectory reconstruction loss value corresponding to the sample traffic object are used to construct the variational lower bound of the maximum likelihood function; Whether the variational lower bound of the likelihood function is maximized;

If the variation lower bound of the constructed maximum likelihood function does not reach the maximum, adjust the model parameters of the feature extraction layer and feature regression layer of the initial trajectory prediction model, and return the traffic object for each sample, using The feature extraction layer of the initial trajectory prediction model and the initial sample feature corresponding to the sample traffic object, and the step of determining the sample prediction feature corresponding to the sample traffic object;

If the constructed variational lower bound of the maximized likelihood function is maximized, it is determined that the initial trajectory prediction model converges, and the target trajectory prediction model including the feature extraction layer and the feature regression layer is obtained.

Optionally, the second determination module is specifically configured to, for each traffic participant object, sample the multimodal probability distribution of the hidden random variable corresponding to the traffic participant object, and obtain a plurality of hidden random variables corresponding to the traffic participant object. variable sample;

Using the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant object, and a plurality of latent random variable samples corresponding to each traffic participant object, the multimodal prediction trajectory corresponding to each traffic participant object is determined.

It can be seen from the above content that the method and device for predicting a motion trajectory provided by the embodiment of the present invention obtain the historical trajectory and motion attribute information of each traffic participating object corresponding to the target object and the corresponding current map information; The feature extraction layer and the initial features corresponding to each traffic participant object determine the trajectory prediction feature corresponding to each traffic participant object, wherein the initial feature corresponding to the traffic participant object includes: the historical trajectory and motion attribute information of the traffic participant object, and their corresponding The historical trajectory and motion attribute information of other traffic participants and target objects, as well as the current map information; using the feature extraction layer of the target trajectory prediction model and the trajectory prediction features corresponding to each traffic participant object, determine the hidden random variables corresponding to each traffic participant object. The modal probability distribution, in which the hidden random variables represent the behavior randomness of each traffic participant; the feature regression layer of the target trajectory prediction model, the trajectory prediction characteristics corresponding to each traffic participant, and the hidden random variables corresponding to each traffic participant are many The modal probability distribution is used to determine the multi-modal prediction trajectory corresponding to each traffic participant object.

By applying the embodiments of the present invention, the hidden random variables in the target trajectory prediction model that have learned the randomness of the behavior of each traffic participant, as well as the historical trajectory and motion attribute information of each traffic participant, and the corresponding dynamic object information, namely the Corresponding historical trajectory and motion attribute information of other traffic participants and target objects, and static object information, i.e. current map information, fitting the hidden random variable multimodal probability distribution of each traffic participant object, that is, the hidden random variable multimodal prior distribution, which represents the multiple possibilities of the future trajectories of each traffic participant and the target object, and then determines the multimodal prediction trajectory corresponding to each traffic participant, so as to achieve accurate determination of the multimodal prediction trajectory of each traffic participant , and the target trajectory prediction model including the hidden random variables that learn the behavior randomness of each traffic participant is universal to the scene, and there is no bottleneck restriction in algorithm design, and the training of the target trajectory prediction model is obtained with training. As the scale of data expands, the algorithm's ability to model the future trajectory distribution can be continuously enhanced, and the trajectory prediction ability can also be continuously improved. Of course, it is not necessary for any product or method of the present invention to achieve all of the advantages described above at the same time.

The innovative points of the embodiments of the present invention include:

1. Combined with the hidden random variables of the randomness of the behavior of each traffic participant, the historical trajectory and motion attribute information of each traffic participant, and the corresponding current map information, which have been learned in the target trajectory prediction model, construct the corresponding traffic participant object. Multi-modal probability distribution of hidden random variables, and then determine the multi-modal prediction trajectory corresponding to each traffic participant, so as to realize the accurate determination of the multi-modal prediction trajectory of each traffic participant, and this includes learning the trajectories of each traffic participant. The target trajectory prediction model of hidden random variables with random behavior is universal to the scene, and there is no bottleneck restriction in algorithm design. The modeling ability can be continuously strengthened, and then the trajectory prediction ability can also be continuously improved.

2. Perform feature processing on the features to be processed corresponding to the traffic participants from the feature dimension and the time dimension in turn, realize the aggregation extraction of the features of different feature dimensions and the features of different time dimensions of the features to be processed, obtain the aggregated features, and then combine the aggregated features with The features of each historical moment in the initial features are fused, and the above operations are repeated many times to obtain the deep abstract intermediate prediction features corresponding to the traffic participating objects. Fusion to determine the trajectory prediction feature corresponding to the traffic participant object, so as to provide a basis for ensuring the accuracy of subsequent future trajectory prediction results.

3. Through the normalized flow mapping algorithm and the unimodal probability distribution of the hidden random variables corresponding to the traffic participating objects, the multimodal probability distribution of the hidden random variables corresponding to the traffic participating objects is constructed, which is the multimodal probability distribution of the subsequent traffic participating objects. The prediction of the trajectory provides the basis.

4. Through the sample historical trajectory and sample motion attribute information corresponding to each sample traffic object, the sample historical trajectory, sample motion attribute information and static object information of the corresponding sample dynamic object, and the sample future trajectory corresponding to each sample traffic object, the training initial Trajectory prediction model, so that the hidden random variables in the initial trajectory prediction model can learn the randomness of the behavior of each sample traffic object, and provide a basis for the accurate prediction of the future trajectory of the subsequent traffic participants.

Description of drawings

In order to illustrate the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that are required in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the invention. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort.

1 is a schematic flowchart of a method for predicting a motion trajectory according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a hidden random variable unimodal probability distribution provided by an embodiment of the present invention being mapped to a hidden random variable multimodal probability distribution;

3 is a schematic flowchart of a training process of a target trajectory prediction model provided by an embodiment of the present invention;

FIG. 4 is a schematic structural diagram of an apparatus for predicting a motion trajectory according to an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, but not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

It should be noted that the terms "comprising" and "having" and any modifications thereof in the embodiments of the present invention and the accompanying drawings are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the steps or units listed, but optionally also includes steps or units not listed, or optionally also includes For other steps or units inherent to these processes, methods, products or devices.

The present invention provides a motion trajectory prediction method and device, so as to reduce the limitation of motion trajectory prediction and better adapt to more complex automatic driving scenarios. The embodiments of the present invention will be described in detail below.

FIG. 1 is a schematic flowchart of a method for predicting a motion trajectory according to an embodiment of the present invention. The method may include the following steps:

S101: Obtain historical trajectory and motion attribute information of each traffic participating object corresponding to the target object and corresponding current map information.

The motion trajectory prediction method provided in the embodiment of the present invention can be applied to any electronic device with computing capability, and the electronic device can be a terminal or a server. In one implementation, the functional software for implementing the motion trajectory prediction method may exist in the form of a separate client software, or may exist in the form of a plug-in of the currently related client software, which is all possible.

The target object may be an autonomous vehicle or an intelligent robot. The target object can obtain the historical trajectory and motion attribute information of each traffic participating object corresponding to it through the sensor set by it, wherein the motion attribute information of the traffic participating object can include the motion information and attribute information of the traffic participating object, wherein the traffic participating object The motion information includes but is not limited to: the speed and acceleration of the traffic participating objects. The attribute information of the traffic participating object may include, but is not limited to, the type, shape and size of the traffic participating object. The historical track of the traffic participant object includes: position information and posture information of the traffic participant object at each historical moment within a preset time period before the current moment.

The current moment may refer to the moment at which the electronic device currently wants to predict the trajectory. The "history" in the above-mentioned historical trajectory and historical moment is relative to the moment at which the electronic device currently predicts the trajectory, that is, the current moment, and refers to the trajectory generated in the time period before the current moment.

The local or connected storage device of the electronic device can pre-store the complete map information of the area where the target object is located, and the current map information can be the complete map information, or can be obtained from the complete map based on the current pose information of the target object at the current moment. In the information, it is possible to determine the map information within the area corresponding to the current pose information.

The sensors set by the target object may include but are not limited to: image acquisition equipment, wheel speed sensors, radar, IMU (Inertial measurement unit, inertial measurement unit), GPS (Global Positioning System, global positioning system) and GNSS (Global Navigation Satellite System) , GNSS/GNSS), etc.

In one case, if the target object is an autonomous vehicle, the corresponding traffic participation objects may include, but are not limited to, objects such as motor vehicles, bicycles, tricycles, pedestrians, and animals. In the case where the traffic participant object is a motor vehicle, the electronic device can also obtain the vehicle lamp information of the motor vehicle, for example, the on-off status of the turn signal.

In one case, the target object is an autonomous vehicle, and the current map information may include, but is not limited to, traffic sign information such as lane lines, parking spaces, sidewalks, traffic signs, traffic sign arrows, street light poles, etc., wherein, Each traffic sign information included in the current map information may be called a static object, and may also include static objects such as buildings, plants, and other objects with fixed positions in the scene.

S102: Determine the trajectory prediction feature corresponding to each traffic participant object by using the feature extraction layer of the target trajectory prediction model and the initial feature corresponding to each traffic participant object.

The initial features corresponding to the traffic participating object include: historical trajectory and motion attribute information of the traffic participating object, and the corresponding historical trajectory and motion attribute information of other traffic participating objects and target objects, as well as current map information.

The target trajectory prediction model is based on the sample historical trajectory and sample motion attribute information corresponding to the sample traffic objects, the sample historical trajectory and sample motion attribute information of the corresponding sample dynamic objects, the static object information, and the sample future corresponding to each sample traffic object. Trajectory training resulting model. The target trajectory prediction model is a neural network latent variable model. In order to make the layout clear, the training process of the target trajectory prediction model will be described later.

The sample dynamic objects corresponding to the sample traffic objects may include: other dynamic traffic objects in the scene where the sample traffic objects are located. The static object information corresponding to the sample traffic object may include: each static object in the map information corresponding to the scene where the sample traffic object is located.

For each traffic participant object, the electronic device may input the initial feature corresponding to the traffic participant object into the feature extraction layer of the target trajectory prediction model, so as to characterize the initial feature corresponding to the traffic participant object through the feature extraction layer of the target trajectory prediction model. Extracting and determining the trajectory prediction feature corresponding to the traffic participant object, and determining the trajectory prediction feature corresponding to each traffic participant object.

S103: Using the feature extraction layer of the target trajectory prediction model and the trajectory prediction feature corresponding to each traffic participant object, determine the multimodal probability distribution of the latent random variable corresponding to each traffic participant object.

Among them, the hidden random variables represent the behavior randomness of each traffic participant.

In this step, the target trajectory prediction model is a model with hidden random variables. After the electronic device obtains the trajectory prediction features corresponding to each traffic participant object, for each traffic participant object, the trajectory prediction feature corresponding to the traffic participant object and the target are used. The hidden random variables of the feature extraction layer of the trajectory prediction model are used to determine the multimodal probability distribution of the hidden random variables corresponding to the traffic participants. The randomness and uncertainty of the trajectory of the traffic participants are represented by hidden random variables.

In an implementation manner of the present invention, the S103 may include the following steps 011-012:

011: For each traffic participant object, use the feature extraction layer of the target trajectory prediction model and the trajectory prediction feature corresponding to the traffic participant object to determine the hidden random variable unimodal probability distribution corresponding to the traffic participant object.

012: For each traffic participant object, use the normalized flow mapping algorithm and the unimodal probability distribution of the hidden random variable corresponding to the traffic participant object to obtain the multimodal trajectory distribution corresponding to the hidden random variable corresponding to the traffic participant object.

In this implementation manner, the electronic device first uses the feature extraction layer of the target trajectory prediction model and the trajectory prediction feature corresponding to the traffic participant object to determine the unimodal probability distribution of the hidden random variable corresponding to the traffic participant object; Flow) mapping algorithm, the hidden random variable unimodal probability distribution corresponding to the traffic participant object is mapped into a multimodal probability distribution, and the hidden random variable multimodal probability distribution corresponding to the traffic participant object is obtained.

In one case, assuming that the unimodal probability distribution of the hidden random variable corresponding to the traffic participant object is a multivariate Gaussian distribution, the feature extraction layer of the target trajectory prediction model can output its corresponding mean and variance. , the unimodal probability distribution of the hidden random variable corresponding to the traffic participant object can be constructed. Subsequently, through the normalized flow mapping algorithm, the unimodal probability distribution of the hidden random variable corresponding to the traffic participant object is mapped to the multimodal trajectory distribution, and the multimodal probability distribution of the hidden random variable corresponding to the traffic participant object is obtained to simplify the process. The subsequent target trajectory prediction model maps the randomness of the trajectory, that is, the randomness of the hidden random variables, into the difficulty of the multimodal probability distribution in the trajectory space, so as to achieve a better multimodal future trajectory modeling effect. The effect diagram is shown in Figure 2, in which "single-modal distribution" represents the single-modal probability distribution of the hidden random variables corresponding to the traffic participants, and "multi-modal distribution" represents the multi-modal trajectories of the hidden random variables corresponding to the traffic participants distributed.

S104: Determine the multimodal predicted trajectory corresponding to each traffic participant by using the feature regression layer of the target trajectory prediction model, the trajectory prediction feature corresponding to each traffic participant object, and the multimodal probability distribution of the hidden random variable corresponding to each traffic participant object.

The electronic device determines the multimodal probability distribution of the hidden random variables corresponding to each traffic participant, and for each traffic participant, the feature regression layer of the target trajectory prediction model is used to predict the trajectory corresponding to the traffic participant object and the traffic participant object. The corresponding hidden random variable multimodal probability distributions are fused to determine the multimodal predicted trajectory corresponding to the traffic participants.

In an implementation manner of the present invention, the S104 may include the following steps 021-022:

021: For each traffic participant object, sample the multimodal probability distribution of the latent random variable corresponding to the traffic participant object to obtain a plurality of latent random variable samples corresponding to the traffic participant object.

022: Using the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant object, and a plurality of latent random variable samples corresponding to each traffic participant object, determine the multimodal prediction trajectory corresponding to each traffic participant object.

In this implementation manner, for each traffic participant object, the electronic device samples the multimodal probability distribution of the hidden random variable corresponding to the traffic participant object, and obtains a plurality of hidden random variable samples corresponding to the traffic participant object, and then uses the target trajectory The feature regression layer of the prediction model maps the trajectory prediction feature corresponding to the traffic participant object and a plurality of latent random variable samples corresponding to the traffic participant object to the trajectory space, that is, the trajectory prediction feature corresponding to the traffic participant object and the traffic participant object. Each corresponding hidden random variable sample is fused to obtain the multi-modal prediction trajectory corresponding to each traffic participant object.

By applying the embodiments of the present invention, the hidden random variables that have learned the randomness of the behavior of each traffic participant in the target trajectory prediction model, as well as the historical trajectory and motion attribute information of each traffic participant, and the corresponding dynamic object information, namely the The corresponding historical trajectory and motion attribute information of other traffic participants and target objects, and static object information, that is, the current map information, fit the conditional probability distribution of the future trajectory of the participants, that is, the hidden random variables corresponding to each traffic participant. modal probability distribution, and then determine the multi-modal prediction trajectory corresponding to each traffic participant, so as to realize the accurate determination of the multi-modal prediction trajectory of each traffic participant, and this includes learning the hidden hidden behavior of each traffic participant’s behavior randomness. The target trajectory prediction model of random variables is universal to this scenario, and there is no bottleneck restriction in algorithm design. With the expansion of the training data scale of the target trajectory prediction model obtained through training, the algorithm's ability to model the future trajectory distribution can be continuously strengthened. , and then the trajectory prediction ability can also be continuously improved.

In another embodiment of the present invention, the initial features corresponding to the traffic participating objects are features arranged in chronological order, which include features of multiple historical moments corresponding to the traffic participating objects;

The S102 may include the following steps 031-032:

031: For each traffic participant object, use the feature extraction layer of the target trajectory prediction model to perform the following steps A-C repeatedly for the initial feature corresponding to the traffic participant object to determine the intermediate prediction feature corresponding to the traffic participant object.

032: For each traffic participant object, fuse the intermediate prediction features corresponding to each static object in the intermediate prediction features corresponding to the traffic participant object based on the graph neural network, and determine the trajectory prediction feature corresponding to the traffic participant object.

The static objects include all static objects in the current map information.

Step A: Perform nonlinear mapping on the to-be-processed feature corresponding to the traffic participant object from the feature dimension to obtain the mapping feature corresponding to the traffic participant object.

The feature to be processed is the initial feature corresponding to the traffic participant object or the intermediate prediction feature corresponding to the traffic participant object generated in the previous iteration.

Step B: Perform a feature aggregation operation on the mapping feature from the time dimension to obtain the aggregated feature corresponding to the traffic participant object.

Step C: Integrate the aggregated features with the features of each historical moment in the features to be processed.

In this implementation manner, the initial features corresponding to the traffic participating objects are features arranged in chronological order, which include features of multiple historical moments corresponding to the traffic participating objects. It can be understood that each historical moment can correspond to multiple features corresponding to the traffic participating objects, for example: the location information of the traffic participating objects, attitude information such as heading angle, speed, shape, type and size, other traffic corresponding to the traffic participating objects. The position information and attitude information of participating objects and target objects, such as heading angle, speed, shape, type and size, as well as the relative position information and type of each static object in the current map information corresponding to the traffic participating object and its corresponding information. The features corresponding to each historical moment are arranged in the order of the information of each historical moment.

For each traffic participant object, the electronic device can first use the feature extraction layer of the target trajectory prediction model to non-linearly map the initial feature corresponding to the traffic participant object from the feature dimension, that is, for each historical moment, the feature, perform nonlinear mapping from the feature dimension to obtain the mapping feature corresponding to the traffic participating object; and then perform feature aggregation operation on the mapping feature from the time dimension, that is, perform feature aggregation operation on the mapping features corresponding to each historical moment, and obtain the traffic participation object. The aggregated feature corresponding to the object; the aggregated feature corresponding to the traffic participant object is fused with the features of each historical moment in the initial feature, and the resulting feature is taken as the new feature to be processed corresponding to the traffic participant object, and re-execution is performed for each traffic participant object. For a traffic participant object, first use the feature extraction layer of the target trajectory prediction model to non-linearly map the initial feature corresponding to the traffic participant object from the feature dimension to obtain the mapping feature corresponding to the traffic participant object, until the above steps are repeated. multiple times to obtain the intermediate prediction feature corresponding to the traffic participant object containing the deep abstract feature.

Further, for each traffic participant object, the electronic device fuses the intermediate prediction features corresponding to each static object in the intermediate prediction features corresponding to the traffic participant object based on the graph neural network, and determines the trajectory prediction feature corresponding to the traffic participant object.

In another embodiment of the present invention, before the S102, the method may further include:

The process of training to obtain a target trajectory prediction model, wherein, as shown in Figure 3, the process includes:

S301: Obtain an initial trajectory prediction model.

S302: Obtain sample training information corresponding to each sample traffic object and sample future trajectories corresponding to each sample traffic object.

The sample training information corresponding to the sample traffic object includes: the sample historical trajectory and sample motion attribute information of the sample traffic object, the sample historical trajectory, sample motion attribute information and sample static object information of the corresponding sample dynamic object. The sample motion attribute information may include motion information and attribute information of the sample traffic object, wherein the motion information of the sample traffic object includes, but is not limited to, information such as speed and acceleration of the sample traffic object. The attribute information of the sample traffic object may include, but is not limited to, the type, shape, and size of the sample traffic object. The sample historical trajectory of the sample traffic object includes: position information and attitude information of the sample traffic object at each historical moment in a preset time period before the sample trajectory collection moment. The sample future trajectories of the sample traffic objects are: the real running trajectories of the sample traffic objects at the sample track collection time and the first time period after that, including the sample traffic objects at the sample track collection time and the first time period after that. The real position information and attitude information.

The sample dynamic objects corresponding to the sample traffic objects are dynamic objects around the environment where the sample traffic objects are located, which may include vehicles, pedestrians, animals, etc.; the sample static object information corresponding to the sample traffic objects includes information about the environment where the sample traffic objects are located. Each static object in the map information.

In one case, the target object is an autonomous vehicle, and accordingly, the sample vehicle can collect corresponding information for each object in its environment during the driving process, the sample training information corresponding to a sample traffic object, and the sample future trajectory It may be determined based on sensor data collected by the sample vehicle through the sensors it is provided with.

In one case, in the case where the sample traffic object is a motor vehicle, the sample training information corresponding to the sample traffic object may also include vehicle lamp information of the motor vehicle, for example, the turning lights on and off.

S303: For each sample traffic object, use the feature extraction layer of the initial trajectory prediction model and the initial sample feature corresponding to the sample traffic object to determine the sample prediction feature corresponding to the sample traffic object.

The initial sample features corresponding to the sample traffic object include: sample historical trajectory and sample motion attribute information of the sample traffic object, and sample historical trajectory, sample motion attribute information and sample static object information of the corresponding sample dynamic object.

S304: For each sample traffic object, use the feature extraction layer of the initial trajectory prediction model and the sample prediction feature corresponding to the sample traffic object to determine the multimodal probability distribution of the latent random variable corresponding to the sample traffic object.

S305: For each sample traffic object, use the feature regression layer of the initial trajectory prediction model, the sample prediction feature corresponding to the sample traffic object, and the multimodal probability distribution of the latent random variables corresponding to the sample traffic object to determine the corresponding sample traffic object. The multimodal prediction trajectory of .

S306 : For each sample traffic object, use a preset variational algorithm to process the sample future trajectory corresponding to the sample traffic object to obtain a variational probability distribution of a latent random variable corresponding to the sample traffic object.

S307: For each sample traffic object, use the multimodal probability distribution of the hidden random variable corresponding to the sample traffic object and the variational probability distribution of the hidden random variable corresponding to the sample traffic object to determine the hidden random variable KL corresponding to the sample traffic object Divergence value.

S308: For each sample traffic object, use the multimodal predicted trajectory corresponding to the sample traffic object and the sample future trajectory corresponding to the sample traffic object to determine a trajectory reconstruction loss value corresponding to the sample traffic object.

S309: For each sample traffic object, use the KL divergence value of the latent random variable corresponding to the sample traffic object and the trajectory reconstruction loss value corresponding to the sample traffic object to construct a variational lower bound of the maximum likelihood function; Whether the variational lower bound of the constructed maximal likelihood function is maximized.

S310: If the variation lower bound of the constructed maximum likelihood function does not reach the maximum, adjust the model parameters of the feature extraction layer and the feature regression layer of the initial trajectory prediction model, and return to S303.

S311 : If the variation lower bound of the constructed maximization likelihood function is maximized, determine that the initial trajectory prediction model converges, and obtain a target trajectory prediction model including a feature extraction layer and a feature regression layer.

In order to ensure the accuracy of the future trajectory prediction of each traffic participant object, the embodiment of the present invention further includes a training process of the target trajectory prediction model. Correspondingly, the electronic device can first obtain an initial trajectory prediction model, where the initial trajectory prediction model can be a neural network latent variable model; obtain sample training information corresponding to each sample traffic object and sample future trajectories corresponding to each sample traffic object.

Further, for each sample traffic object, the electronic device inputs the initial sample feature corresponding to the sample traffic object into the feature extraction layer of the initial trajectory prediction model, and uses the feature extraction layer of the initial trajectory prediction model to the initial sample feature corresponding to the sample traffic object Perform feature extraction and fusion to determine the sample prediction features corresponding to the sample traffic objects. The process of feature extraction and fusion of the initial sample features corresponding to the sample traffic objects using the feature extraction layer of the initial trajectory prediction model can be found in the target trajectory prediction model. The feature extraction layer of the feature extraction and fusion process of the initial features of the traffic participating objects will not be repeated here.

For each sample traffic object, the feature extraction layer of the initial trajectory prediction model and the sample prediction feature corresponding to the sample traffic object are used to obtain the unimodal probability distribution of the latent random variable corresponding to the sample traffic object, and then through the normalized flow mapping algorithm, The hidden random variable unimodal probability distribution corresponding to the sample traffic object is mapped to the hidden random variable multimodal probability distribution corresponding to the sample traffic object.

For each sample traffic object, the sample prediction feature corresponding to the sample traffic object and the multimodal probability distribution of the latent random variable corresponding to the sample traffic object are input into the feature regression layer of the initial trajectory prediction model to pass the initial trajectory prediction model. The feature regression layer fuses the sample prediction feature corresponding to the sample traffic object and the multimodal probability distribution of the latent random variable corresponding to the sample traffic object to obtain the multimodal prediction trajectory corresponding to the sample traffic object.

Subsequently, in order to ensure the accuracy of the prediction results of the constructed target trajectory prediction model, the multimodal prediction trajectory corresponding to the sample traffic object and the sample future trajectory corresponding to the sample traffic object can be used to construct a maximum likelihood function. The variational lower bound is used to adjust the model parameters of the initial trajectory prediction model through the constructed variational lower bound of the maximized likelihood function, and then, the final target trajectory prediction model is obtained. In order to construct the variational lower bound of the maximum likelihood function, the electronic device uses a preset variational algorithm to process the sample future trajectory corresponding to the sample traffic object for each sample traffic object, and obtains the hidden random variable corresponding to the sample traffic object Variational probability distribution, wherein the preset variational algorithm may be a variational algorithm constructed based on the principle of variational Bayes.

For each sample traffic object, based on the KL divergence algorithm, the multimodal probability distribution of the hidden random variable corresponding to the sample traffic object and the variational probability distribution of the hidden random variable corresponding to the sample traffic object are used to determine the corresponding sample traffic object. Hidden random variable KL divergence value. And using the multi-modal prediction trajectory corresponding to the sample traffic object and the sample future trajectory corresponding to the sample traffic object, the trajectory reconstruction loss value corresponding to the sample traffic object is determined. Using the trajectory reconstruction loss value corresponding to the sample traffic object and the KL divergence value of the hidden random variable corresponding to the sample traffic object, the variational lower bound of the maximal likelihood function is constructed; and the variational lower bound of the maximal likelihood function is calculated. Corresponding function value, determine whether the variational lower bound of the constructed maximal likelihood function is maximized, that is, whether the function value corresponding to the variational lower bound of the maximized likelihood function is maximized, if the constructed maximal likelihood If the variational lower bound of the function is not maximized, use the preset optimization algorithm to adjust the model parameters of the feature extraction layer and the feature regression layer of the initial trajectory prediction model, and return to execute S203; When the lower bound is maximized, the initial trajectory prediction model is determined to converge, and the target trajectory prediction model including the feature extraction layer and the feature regression layer is obtained.

In one case, the probability distribution corresponding to the multimodal predicted trajectory corresponding to the obtained sample traffic object is constructed, and the multimodal predicted trajectory corresponding to the sample traffic object can be constructed by using the probability distribution, and the following formula (1) express:

p(x _f |x _p ,Φ)=∫p(x _f |z,x _p ,Φ)p(z|x _p ,Φ)dz; (1)

Among them, x _p represents the sample historical trajectory corresponding to the sample traffic object, x _f represents the multimodal prediction trajectory corresponding to the sample traffic object; Φ represents the sample traffic object corresponding to the sample traffic object except the sample corresponding to the sample traffic object. Other information other than the historical trajectory; p(x _f |x _p ,Φ) represents the probability distribution corresponding to the multimodal predicted trajectory of the sample traffic object; z represents the hidden random variable; p(z|x _p ,Φ) represents the sample The multimodal probability distribution of the hidden random variable corresponding to the traffic object is the prior distribution of the hidden random variable z given the sample historical trajectory and other information in the initial sample characteristics except the sample historical trajectory corresponding to the sample traffic object, It represents the randomness of the future trajectory of the sample traffic object according to the historical trajectory of the sample traffic object and the surrounding map, that is, the sample static object information and the sample dynamic object as a whole; p(x _f |z,x _p ,Φ) represents the sample traffic object The probability distribution corresponding to the multimodal predicted trajectory corresponding to the object is the probability distribution of the future trajectory given additional information such as hidden random variables, sample historical trajectories and maps, that is, by comprehensively considering all deterministic and random information, Output the predicted results of future trajectories. This modeling method can represent the randomness of the behavior of the sample traffic objects or traffic participants through the hidden random variable z, and map this randomness to the original trajectory data space using the neural network model, namely the initial trajectory prediction model or the target trajectory prediction model. , which can theoretically fit any future trajectory distribution, with high versatility and effect.

Correspondingly, the variational lower bound of the constructed maximal likelihood function can be expressed by the following formula;

logp(x _f |x _p ,Φ)≥E _q [(x _f |z,x _p ,Φ)]-KL(q(z|x _f ,x _p ,Φ)||p(z|x _p , Φ));

Among them, logp(x _f |x _p ,Φ) represents the constructed maximum likelihood function, E _q [(x _f |z,x _p ,Φ)] represents the trajectory reconstruction loss value corresponding to the sample traffic object, KL(q(z|x _f ,x _p ,Φ)||p(z|x _p ,Φ) represents the KL divergence value of the hidden random variable corresponding to the sample traffic object, E _q [(x _f |z,x _p ,Φ)]-KL(q(z|x _f ,x _p ,Φ)||p(z|x _p ,Φ)) represents the variational lower bound of the maximum likelihood function.

In this implementation, in the process of constructing the target trajectory prediction model, the historical trajectory of each traffic participant object, the operation attribute information and the information of the surrounding static objects are fully considered. The feature extraction layer of the initial trajectory prediction model extracts and fuses the features in each direction between the features, that is, the feature dimension and the time dimension, and realizes the sufficient extraction and fusion of the features corresponding to the traffic participants to support the follow-up. The model's prediction of future trajectories.

Corresponding to the foregoing method embodiments, an embodiment of the present invention provides an apparatus for predicting a motion trajectory. As shown in FIG. 4 , the apparatus may include:

The obtaining module 410 is configured to obtain the historical trajectory and motion attribute information of each traffic participating object corresponding to the target object and the corresponding current map information;

The first determination module 420 is configured to utilize the feature extraction layer of the target trajectory prediction model and the initial features corresponding to each traffic participant object to determine the trajectory prediction feature corresponding to each traffic participant object, wherein the corresponding initial features of the traffic participant object include: The historical trajectory and motion attribute information of the traffic participating object, and the historical trajectory and motion attribute information of other corresponding traffic participating objects and target objects, and the current map information;

The second determination module 430 is configured to use the feature extraction layer of the target trajectory prediction model and the trajectory prediction features corresponding to each traffic participant object to determine the multimodal probability distribution of the hidden random variables corresponding to each traffic participant object, wherein the hidden random variable Characterize the randomness of behavior of each traffic participant;

The third determination module 440 is configured to use the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant object, and the multimodal probability distribution of hidden random variables corresponding to each traffic participant object to determine each traffic participant The multimodal predicted trajectory corresponding to the object.

By applying the embodiments of the present invention, the hidden random variables in the target trajectory prediction model that have learned the randomness of the behavior of each traffic participant, as well as the historical trajectory and motion attribute information of each traffic participant, and the corresponding dynamic object information, namely the The corresponding historical trajectory and motion attribute information of other traffic participating objects and target objects, and static object information, that is, current map information, fit the conditional probability distribution of the future trajectory of the participating objects, that is, the hidden random variables corresponding to each traffic participating object. modal probability distribution, and then determine the multi-modal prediction trajectory corresponding to each traffic participant, so as to realize the accurate determination of the multi-modal prediction trajectory of each traffic participant, and this includes learning the hidden hidden behavior of each traffic participant’s behavior randomness. The target trajectory prediction model of random variables is universal to this scenario, and there is no bottleneck restriction in algorithm design. With the expansion of the training data scale of the target trajectory prediction model obtained through training, the algorithm's ability to model the future trajectory distribution can be continuously strengthened. , and then the trajectory prediction ability can also be continuously improved.

In another embodiment of the present invention, the initial features corresponding to the traffic participating objects are features arranged in chronological order, which include features of multiple historical moments corresponding to the traffic participating objects;

The first determining module 420 is specifically configured to, for each traffic participant object, use the feature extraction layer of the target trajectory prediction model to perform the following steps A-C repeatedly for the initial feature corresponding to the traffic participant object to determine the traffic participant object. The intermediate prediction features corresponding to the participating objects;

For each traffic participant object, the intermediate prediction features corresponding to each static object in the intermediate prediction features corresponding to the traffic participant object are fused based on the graph neural network, and the trajectory prediction feature corresponding to the traffic participant object is determined, wherein the static The object includes each static object in the current map information;

Step A: Perform nonlinear mapping on the to-be-processed feature corresponding to the traffic participant object from the feature dimension to obtain the map feature corresponding to the traffic participant object, wherein the to-be-processed feature is the initial feature or the previous time corresponding to the traffic participant object The iteratively generated intermediate prediction feature corresponding to the traffic participant object;

Step B: performing a feature aggregation operation on the mapping feature from the time dimension to obtain the aggregation feature corresponding to the traffic participant object;

Step C: Fusion of the aggregated feature and the feature of each historical moment in the feature to be processed.

In another embodiment of the present invention, the second determining module 430 is specifically configured to, for each traffic participant object, use the feature extraction layer of the target trajectory prediction model and the trajectory prediction feature corresponding to the traffic participant object to determine The unimodal probability distribution of the hidden random variable corresponding to the traffic participant object;

For each traffic participant, using the normalized flow mapping algorithm and the unimodal probability distribution of the hidden random variable corresponding to the traffic participant, the multimodal probability distribution of the hidden random variable corresponding to the traffic participant is obtained.

In another embodiment of the present invention, the device further includes:

The training module (not shown in the figure) is configured to obtain the target by training before determining the trajectory prediction feature corresponding to each traffic participant object using the feature extraction layer of the target trajectory prediction model and the initial features corresponding to each traffic participant object. A trajectory prediction model, wherein the training module is specifically configured as

Obtain the initial trajectory prediction model;

Obtain the sample training information corresponding to each sample traffic object and the sample future trajectory corresponding to each sample traffic object, wherein the sample training information corresponding to the sample traffic object includes: the sample historical trajectory and sample motion attribute information of the sample traffic object, its corresponding Sample historical trajectory, sample motion attribute information and sample static object information of sample dynamic objects;

For each sample traffic object, use the feature extraction layer of the initial trajectory prediction model and the initial sample feature corresponding to the sample traffic object to determine the sample prediction feature corresponding to the sample traffic object, wherein the initial sample corresponding to the sample traffic object The features include: sample historical trajectories and sample motion attribute information of sample traffic objects, sample historical trajectories, sample motion attribute information and sample static object information of corresponding sample dynamic objects;

For each sample traffic object, use the feature extraction layer of the initial trajectory prediction model and the sample prediction feature corresponding to the sample traffic object to determine the multimodal probability distribution of the latent random variable corresponding to the sample traffic object;

For each sample traffic object, use the feature regression layer of the initial trajectory prediction model, the sample prediction feature corresponding to the sample traffic object, and the multimodal probability distribution of the latent random variables corresponding to the sample traffic object to determine the corresponding sample traffic object. The multimodal prediction trajectory of ;

For each sample traffic object, use a preset variational algorithm to perform variational processing on the sample future trajectory corresponding to the sample traffic object, and obtain the variational distribution probability corresponding to the sample traffic object;

For each sample traffic object, use the multimodal probability distribution of the hidden random variable corresponding to the sample traffic object and the variational probability distribution of the hidden random variable corresponding to the sample traffic object to determine the KL divergence of the hidden random variable corresponding to the sample traffic object value;

For each sample traffic object, use the multimodal predicted trajectory corresponding to the sample traffic object, the variational probability distribution of latent random variables and the sample future trajectory corresponding to the sample traffic object to determine the trajectory reconstruction loss value corresponding to the sample traffic object ;

For each sample traffic object, the KL divergence value of the hidden random variable corresponding to the sample traffic object and the trajectory reconstruction loss value corresponding to the sample traffic object are used to construct the variational lower bound of the maximum likelihood function; Whether the variational lower bound of the likelihood function is maximized;

If the variation lower bound of the constructed maximum likelihood function does not reach the maximum, adjust the model parameters of the feature extraction layer and feature regression layer of the initial trajectory prediction model, and return the traffic object for each sample, using The feature extraction layer of the initial trajectory prediction model and the initial sample feature corresponding to the sample traffic object, and the step of determining the sample prediction feature corresponding to the sample traffic object;

If the constructed variational lower bound of the maximized likelihood function is maximized, it is determined that the initial trajectory prediction model converges, and the target trajectory prediction model including the feature extraction layer and the feature regression layer is obtained.

In another embodiment of the present invention, the second determining module 430 is specifically configured to, for each traffic participant object, sample the multimodal probability distribution of the hidden random variable corresponding to the traffic participant object to obtain the traffic participant object. Multiple samples of hidden random variables corresponding to the object;

Using the feature regression layer of the target trajectory prediction model, the trajectory prediction features corresponding to each traffic participant object, and a plurality of latent random variable samples corresponding to each traffic participant object, the multimodal prediction trajectory corresponding to each traffic participant object is determined.

The foregoing system and device embodiments correspond to the system embodiments, and have the same technical effects as the method embodiments. For specific descriptions, refer to the method embodiments. The apparatus embodiment is obtained based on the method embodiment, and the specific description can refer to the method embodiment section, which will not be repeated here. Those of ordinary skill in the art can understand that the accompanying drawing is only a schematic diagram of an embodiment, and the modules or processes in the accompanying drawing are not necessarily necessary to implement the present invention.

Those skilled in the art may understand that: the modules in the apparatus in the embodiment may be distributed in the apparatus in the embodiment according to the description of the embodiment, and may also be located in one or more apparatuses different from this embodiment with corresponding changes. The modules in the foregoing embodiments may be combined into one module, or may be further split into multiple sub-modules.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be The technical solutions described in the foregoing embodiments are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions in the embodiments of the present invention.

Claims (10)

1. A method for predicting a motion trajectory, the method comprising:

obtaining historical track and motion attribute information of each traffic participating object corresponding to the target object and corresponding current map information;

determining the track prediction characteristics corresponding to the traffic participation objects by utilizing the characteristic extraction layer of the target track prediction model and the initial characteristics corresponding to the traffic participation objects, wherein the initial characteristics corresponding to the traffic participation objects comprise: historical track and motion attribute information of the traffic participating objects, historical track and motion attribute information of other corresponding traffic participating objects and target objects, and the current map information;

determining hidden random variable multi-mode probability distribution corresponding to each traffic participant by using a feature extraction layer of a target track prediction model and track prediction features corresponding to each traffic participant, wherein the hidden random variables represent behavior randomness of each traffic participant;

and determining the multi-modal predicted track corresponding to each traffic participating object by utilizing the characteristic regression layer of the target track prediction model, the track prediction characteristics corresponding to each traffic participating object and the hidden random variable multi-modal probability distribution corresponding to each traffic participating object.

2. The method of claim 1, wherein the initial features corresponding to the traffic participant are features arranged based on time sequence, and the initial features comprise features of a plurality of historical moments corresponding to the traffic participant;

the step of determining the track prediction characteristics corresponding to the traffic participation objects by using the characteristic extraction layer of the target track prediction model and the initial characteristics corresponding to the traffic participation objects comprises the following steps:

aiming at each traffic participant, circularly executing the following steps A-C for a plurality of times on the initial characteristics corresponding to the traffic participant by using the characteristic extraction layer of the target track prediction model, and determining the intermediate prediction characteristics corresponding to the traffic participant;

for each traffic participating object, fusing intermediate prediction features corresponding to each static object in the intermediate prediction features corresponding to the traffic participating object based on a graph neural network, and determining track prediction features corresponding to the traffic participating object, wherein the static objects comprise each static object in the current map information;

step A: carrying out nonlinear mapping on the to-be-processed features corresponding to the traffic participating objects from feature dimensions to obtain mapping features corresponding to the traffic participating objects, wherein the to-be-processed features are initial features corresponding to the traffic participating objects or intermediate prediction features corresponding to the traffic participating objects generated in previous iteration;

and B: performing feature aggregation operation on the mapping features from a time dimension to obtain aggregation features corresponding to the traffic participation objects;

and C: and fusing the aggregation characteristics with the characteristics of the to-be-processed characteristics at each historical moment.

3. The method according to claim 1, wherein the step of determining the hidden random variable multi-modal probability distribution corresponding to each traffic participant by using the feature extraction layer of the target trajectory prediction model and the trajectory prediction features corresponding to each traffic participant comprises:

aiming at each traffic participant, determining hidden random variable monomodal probability distribution corresponding to the traffic participant by using a feature extraction layer of a target track prediction model and track prediction features corresponding to the traffic participant;

aiming at each traffic participant, a normalized stream mapping algorithm and hidden random variable monomodal probability distribution corresponding to the traffic participant are utilized to obtain hidden random variable multimodal probability distribution corresponding to the traffic participant.

4. The method according to any one of claims 1 to 3, wherein before the step of determining the trajectory prediction feature corresponding to each traffic participant by using the feature extraction layer of the target trajectory prediction model and the initial feature corresponding to each traffic participant, the method further comprises:

training a process of obtaining a target trajectory prediction model, wherein the process comprises:

obtaining an initial trajectory prediction model;

obtaining sample training information corresponding to each sample traffic object and a sample future track corresponding to each sample traffic object, wherein the sample training information corresponding to the sample traffic object comprises: sample historical track and sample motion attribute information of the sample traffic object, and sample historical track, sample motion attribute information and sample static object information of a corresponding sample dynamic object;

for each sample traffic object, determining a sample prediction feature corresponding to the sample traffic object by using a feature extraction layer of the initial trajectory prediction model and an initial sample feature corresponding to the sample traffic object, wherein the initial sample feature corresponding to the sample traffic object comprises: sample traffic object sample historical track and sample motion attribute information, and corresponding sample historical track, sample motion attribute information and sample static object information of a sample dynamic object;

aiming at each sample traffic object, determining hidden random variable multi-modal probability distribution corresponding to the sample traffic object by utilizing a feature extraction layer of the initial track prediction model and sample prediction features corresponding to the sample traffic object;

aiming at each sample traffic object, determining a multi-modal predicted track corresponding to the sample traffic object by utilizing a characteristic regression layer of the initial track prediction model, sample predicted characteristics corresponding to the sample traffic object and hidden random variable multi-modal probability distribution corresponding to the sample traffic object;

processing a sample future track corresponding to each sample traffic object by using a preset variation algorithm to obtain hidden random variable variation probability distribution corresponding to the sample traffic object;

aiming at each sample traffic object, determining a KL divergence value of a hidden random variable corresponding to the sample traffic object by utilizing the multi-modal probability distribution of the hidden random variable corresponding to the sample traffic object and the variation probability distribution of the hidden random variable corresponding to the sample traffic object;

for each sample traffic object, determining a track reconstruction loss value corresponding to the sample traffic object by utilizing a multi-modal predicted track corresponding to the sample traffic object and a sample future track corresponding to the sample traffic object;

aiming at each sample traffic object, constructing a variation lower bound of a maximum likelihood function by utilizing a KL divergence value of an implicit random variable corresponding to the sample traffic object and a track reconstruction loss value corresponding to the sample traffic object; judging whether the variation lower bound of the constructed maximum likelihood function reaches the maximum;

if the variation lower bound of the constructed maximum likelihood function does not reach the maximum, adjusting model parameters of a feature extraction layer and a feature regression layer of the initial track prediction model, returning to the step of determining sample prediction features corresponding to the sample traffic objects by using the feature extraction layer of the initial track prediction model and the initial sample features corresponding to the sample traffic objects;

and if the variation lower bound of the constructed maximum likelihood function reaches the maximum, determining the convergence of the initial trajectory prediction model to obtain the target trajectory prediction model comprising a feature extraction layer and a feature regression layer.

5. The method according to any one of claims 1 to 4, wherein the step of determining the multi-modal predicted trajectory corresponding to each traffic participant by using the feature regression layer of the target trajectory prediction model, the trajectory prediction feature corresponding to each traffic participant and the hidden random variable multi-modal probability distribution corresponding to each traffic participant comprises:

sampling hidden random variable multi-mode probability distribution corresponding to the traffic participation object aiming at each traffic participation object to obtain a plurality of hidden random variable samples corresponding to the traffic participation object;

and determining a multi-modal predicted track corresponding to each traffic participant by using the characteristic regression layer of the target track prediction model, the track prediction characteristics corresponding to each traffic participant and a plurality of hidden random variable samples corresponding to each traffic participant.

6. An apparatus for predicting a motion trajectory, the apparatus comprising:

the obtaining module is configured to obtain historical track and motion attribute information of each traffic participation object corresponding to the target object and corresponding current map information;

the first determining module is configured to determine a track prediction feature corresponding to each traffic participant by using the feature extraction layer of the target track prediction model and the initial feature corresponding to each traffic participant, wherein the initial feature corresponding to the traffic participant comprises: historical track and motion attribute information of the traffic participating objects, historical track and motion attribute information of other corresponding traffic participating objects and target objects, and the current map information;

the second determining module is configured to determine multi-modal probability distribution of hidden random variables corresponding to the traffic participating objects by using the feature extraction layer of the target track prediction model and the track prediction features corresponding to the traffic participating objects, wherein the hidden random variables represent behavior randomness of the traffic participating objects;

and the third determining module is configured to determine a multi-modal predicted track corresponding to each traffic participating object by using the characteristic regression layer of the target track prediction model, the track prediction characteristics corresponding to each traffic participating object and the hidden random variable multi-modal probability distribution corresponding to each traffic participating object.

7. The apparatus of claim 6, wherein the initial feature corresponding to the traffic participant object is a feature arranged based on a time sequence, and comprises a plurality of historical time features corresponding to the traffic participant object;

the first determining module is specifically configured to, for each traffic participant, circularly perform the following steps a-C on the initial feature corresponding to the traffic participant for multiple times by using the feature extraction layer of the target trajectory prediction model, and determine an intermediate prediction feature corresponding to the traffic participant;

for each traffic participating object, fusing the intermediate prediction features corresponding to each static object in the intermediate prediction features corresponding to the traffic participating object based on a graph neural network, and determining the track prediction features corresponding to the traffic participating object, wherein the static objects comprise each static object in the current map information;

step A: carrying out nonlinear mapping on the to-be-processed features corresponding to the traffic participation objects from feature dimensions to obtain mapping features corresponding to the traffic participation objects, wherein the to-be-processed features are initial features corresponding to the traffic participation objects or intermediate prediction features corresponding to the traffic participation objects generated in the previous iteration;

and B, step B: performing feature aggregation operation on the mapping features from the time dimension to obtain aggregation features corresponding to the traffic participation objects;

and C: and fusing the aggregation characteristics with the characteristics of the to-be-processed characteristics at each historical moment.

8. The apparatus of claim 6, wherein the second determining module is specifically configured to determine, for each traffic participant, a hidden random variable monomodal probability distribution corresponding to the traffic participant by using a feature extraction layer of a target trajectory prediction model and a trajectory prediction feature corresponding to the traffic participant;

aiming at each traffic participant, a normalized stream mapping algorithm and hidden random variable monomodal probability distribution corresponding to the traffic participant are utilized to obtain hidden random variable multimodal probability distribution corresponding to the traffic participant.

9. The apparatus of claim 6, wherein the apparatus further comprises:

a training module configured to train to obtain a target trajectory prediction model before determining a trajectory prediction feature corresponding to each traffic participant by using the feature extraction layer of the target trajectory prediction model and the initial feature corresponding to each traffic participant, wherein the training module is specifically configured to train to obtain the target trajectory prediction model

Obtaining an initial trajectory prediction model;

obtaining sample training information corresponding to each sample traffic object and a sample future track corresponding to each sample traffic object, wherein the sample training information corresponding to the sample traffic object comprises: the sample historical track and the sample motion attribute information of the sample traffic object, and the sample historical track, the sample motion attribute information and the sample static object information of the corresponding sample dynamic object;

for each sample traffic object, determining a sample prediction feature corresponding to the sample traffic object by using a feature extraction layer of the initial trajectory prediction model and an initial sample feature corresponding to the sample traffic object, wherein the initial sample feature corresponding to the sample traffic object comprises: sample traffic object sample historical track and sample motion attribute information, and corresponding sample historical track, sample motion attribute information and sample static object information of a sample dynamic object;

aiming at each sample traffic object, determining hidden random variable multi-modal probability distribution corresponding to the sample traffic object by utilizing a feature extraction layer of the initial track prediction model and sample prediction features corresponding to the sample traffic object;

aiming at each sample traffic object, determining a multi-modal predicted track corresponding to the sample traffic object by utilizing a characteristic regression layer of the initial track prediction model, sample predicted characteristics corresponding to the sample traffic object and hidden random variable multi-modal probability distribution corresponding to the sample traffic object;

processing a sample future track corresponding to each sample traffic object by using a preset variation algorithm to obtain hidden random variable variation probability distribution corresponding to the sample traffic object;

aiming at each sample traffic object, determining a KL divergence value of a hidden random variable corresponding to the sample traffic object by utilizing the multi-modal probability distribution of the hidden random variable corresponding to the sample traffic object and the variation probability distribution of the hidden random variable corresponding to the sample traffic object;

for each sample traffic object, determining a track reconstruction loss value corresponding to the sample traffic object by using a multi-modal predicted track, hidden random variable variation probability distribution and a sample future track corresponding to the sample traffic object;

aiming at each sample traffic object, constructing a variation lower bound of a maximized likelihood function by utilizing a KL divergence value of a hidden random variable corresponding to the sample traffic object and a track reconstruction loss value corresponding to the sample traffic object; judging whether the variation lower bound of the constructed maximum likelihood function reaches the maximum;

if the variation lower bound of the constructed maximum likelihood function does not reach the maximum, adjusting model parameters of a feature extraction layer and a feature regression layer of the initial trajectory prediction model, returning to the step of determining sample prediction features corresponding to the sample traffic objects by using the feature extraction layer of the initial trajectory prediction model and the initial sample features corresponding to the sample traffic objects;

and if the variation lower bound of the constructed maximum likelihood function reaches the maximum, determining the convergence of the initial trajectory prediction model to obtain the target trajectory prediction model comprising a feature extraction layer and a feature regression layer.

10. The apparatus according to any one of claims 6 to 9, wherein the second determining module is specifically configured to, for each traffic participant, sample a hidden random variable multi-modal probability distribution corresponding to the traffic participant to obtain a plurality of hidden random variable samples corresponding to the traffic participant;

and determining a multi-modal predicted track corresponding to each traffic participant by using the characteristic regression layer of the target track prediction model, the track prediction characteristics corresponding to each traffic participant and a plurality of hidden random variable samples corresponding to each traffic participant.

Images