CN112651557A - Trajectory prediction system and method, electronic device and readable storage medium - Google Patents
Trajectory prediction system and method, electronic device and readable storage medium Download PDFInfo
- Publication number
- CN112651557A CN112651557A CN202011566859.0A CN202011566859A CN112651557A CN 112651557 A CN112651557 A CN 112651557A CN 202011566859 A CN202011566859 A CN 202011566859A CN 112651557 A CN112651557 A CN 112651557A
- Authority
- CN
- China
- Prior art keywords
- predicted
- historical
- track
- semantic
- relative
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 230000007613 environmental effect Effects 0.000 claims abstract description 21
- 238000013528 artificial neural network Methods 0.000 claims description 35
- 125000004122 cyclic group Chemical group 0.000 claims description 30
- 230000000306 recurrent effect Effects 0.000 claims description 27
- 238000003062 neural network model Methods 0.000 claims description 24
- 230000015654 memory Effects 0.000 claims description 20
- 238000012549 training Methods 0.000 claims description 19
- 238000000605 extraction Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 8
- 230000007787 long-term memory Effects 0.000 claims description 4
- 230000006403 short-term memory Effects 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 230000004888 barrier function Effects 0.000 description 3
- 230000004927 fusion Effects 0.000 description 3
- 230000002093 peripheral effect Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Economics (AREA)
- Strategic Management (AREA)
- Marketing (AREA)
- Game Theory and Decision Science (AREA)
- Entrepreneurship & Innovation (AREA)
- Development Economics (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Traffic Control Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention provides a track prediction system and method, an electronic device and a readable storage medium, comprising: acquiring and obtaining historical positions of an object to be predicted and surrounding obstacles in a world coordinate system based on the historical positions of the self-vehicle in the world coordinate system, the historical relative positions of the object to be predicted relative to the self-vehicle and the environmental information of the object to be predicted; according to the historical positions of the object to be predicted and surrounding obstacles in a world coordinate system, obtaining and semantically extracting a relative historical track of the object to be predicted based on the historical relative position of the object to be predicted relative to the surrounding obstacles, and combining the relative historical track with the semantically extracted lane information to obtain a semantic historical track; and inputting the semantic historical track into a track prediction model to predict the track of the object to be predicted. The invention has great promotion in the aspect of unmanned computing power, minimizes the computing consumption in the unmanned operation, can ensure the prediction precision and better meets the engineering requirements.
Description
Technical Field
The present invention relates to the field of automatic driving, and in particular, to a trajectory prediction system and method, an electronic device, and a readable storage medium.
Background
With the development of the intelligent transportation field, the prediction algorithm of the motion trail of the motion object has great significance in the path planning field. By predicting the motion trail of the moving object, the path planning can be performed under the condition that the possible future motion trail of the moving object is known, and the method is favorable for preventing accidents such as collision.
In the prior art, the position of a surrounding vehicle can be obtained through sensing, the track prediction of an object to be predicted, Kalman Filtering (KF), a Gaussian Mixture Model (GMM), a Hidden Markov Model (HMM) and the like can be performed according to a vehicle dynamic model, and the method can establish an accurate track measurement model or a vehicle state transition model, but can only predict the state of the next motion of an object, and cannot meet the requirement of automatic driving on long-time prediction of a future obstacle. In addition, the track of the obstacle can be observed for a period of time, and the track of the object to be predicted can be predicted by using a long-short term memory network (LSTM), the accumulated error of the method is large, although the long-time prediction can be achieved, the predicted future track is often influenced by the loss of environmental change due to the fact that the prediction only depends on the position of the obstacle. Or, the global semantic data (CNN) and the global track data (LSTM) of the region where the object to be predicted is located are obtained, the global semantic data and the global track data are fused to obtain global fusion data, so that the features in the global fusion data are extracted, the global features are processed by using a trained track prediction model, and the target track of the object to be predicted is obtained.
Disclosure of Invention
The invention provides a trajectory prediction system and method, an electronic device and a readable storage medium, which can realize rapid and accurate obstacle trajectory prediction on the premise of limited computing resources of an unmanned vehicle.
The invention provides a track prediction method, which comprises the following steps:
acquiring and obtaining historical positions of an object to be predicted and surrounding obstacles in a world coordinate system based on the historical position of the self-vehicle in the world coordinate system, the historical relative position of the object to be predicted relative to the self-vehicle and environmental information of the object to be predicted, wherein the environmental information comprises historical relative position information and lane information of the surrounding obstacles around the object to be predicted relative to the self-vehicle;
according to historical positions of an object to be predicted and surrounding obstacles in a world coordinate system, obtaining and semantically extracting a relative historical track of the object to be predicted based on the historical relative position of the object to be predicted relative to the surrounding obstacles, and combining the relative historical track of the object to be predicted with lane information subjected to semantic extraction to obtain a semantic historical track;
and inputting the semantic historical track into a track prediction model to predict the track of the object to be predicted.
Preferably, in the trajectory prediction method, a historical trajectory of the object to be predicted, which is composed of historical positions of the object to be predicted in a world coordinate system, is a coordinate set of an x coordinate and a y coordinate of the object to be predicted in an utm coordinate system; the historical relative position of the object to be predicted relative to the surrounding obstacles comprises the distance between the object to be predicted and the front obstacle with the minimum distance to the object to be predicted, the distance between the object to be predicted and the rear obstacle with the minimum distance to the object to be predicted and the total number of the surrounding obstacles, and the lane information comprises the distance between the object to be predicted and the center line of the lane where the object to be predicted is located, wherein the distance is obtained through map information.
Preferably, the trajectory prediction method, wherein the method for generating the trajectory prediction model includes:
constructing a recurrent neural network model;
coding by using a long-short term memory network based on the semantic historical track of the object to be predicted to obtain an updated state and a hidden state of the recurrent neural network; inputting the encoded semantic historical track into a cyclic neural network model, decoding the current position of the object to be predicted step by step to obtain a predicted future track, calculating deviation with the real future track of the object to be predicted, performing error back propagation, and training the cyclic neural network to obtain a track prediction model.
Preferably, the trajectory prediction method further includes:
coding by using a long-short term memory network based on the semantic historical track of the object to be predicted to obtain an updated state and a hidden state of the recurrent neural network; and inputting the coded semantic historical track into a cyclic neural network model, and decoding the semantic historical track in steps by combining the updating state and the hidden state from the current position of the object to be predicted to obtain a predicted future track.
The present invention provides a trajectory prediction system comprising:
the information acquisition module is used for acquiring and obtaining historical positions of the object to be predicted and surrounding obstacles in the world coordinate system based on the historical position of the self-vehicle in the world coordinate system, the historical relative position of the object to be predicted relative to the self-vehicle and environmental information of the object to be predicted, wherein the environmental information comprises historical relative position information and lane information of the surrounding obstacles around the object to be predicted relative to the self-vehicle;
the semantic historical track generation module is used for obtaining and semantically extracting a relative historical track of the object to be predicted based on the historical relative position of the object to be predicted relative to surrounding obstacles according to the historical positions of the object to be predicted and the surrounding obstacles in a world coordinate system, and combining the relative historical track of the object to be predicted and the lane information after semantic extraction to obtain the semantic historical track;
and the track prediction module is used for inputting the semantic historical track into a track prediction model to predict the track of the object to be predicted.
Preferably, the trajectory prediction system is configured, where a historical trajectory of the object to be predicted, which is composed of historical positions of the object to be predicted in the world coordinate system, is a coordinate set of an x coordinate and a y coordinate of the object to be predicted in an utm coordinate system; the historical relative position of the object to be predicted relative to the surrounding obstacles comprises the distance between the object to be predicted and the front obstacle with the minimum distance to the object to be predicted, the distance between the object to be predicted and the rear obstacle with the minimum distance to the object to be predicted and the total number of the surrounding obstacles, and the lane information comprises the distance between the object to be predicted and the center line of the lane where the object to be predicted is located, wherein the distance is obtained through map information.
Preferably, in the trajectory prediction system, the trajectory prediction model is obtained by a recurrent neural network model building module and a recurrent neural network training module, where:
the recurrent neural network model building module is used for building a recurrent neural network model;
the cyclic neural network training module is used for coding by using a long-short term memory network based on the semantic historical track of the object to be predicted to obtain an updated state and a hidden state of the cyclic neural network; inputting the encoded semantic historical track into a cyclic neural network model, decoding the current position of the object to be predicted step by step to obtain a predicted future track, calculating deviation with the real future track of the object to be predicted, performing error back propagation, and training the cyclic neural network to obtain a track prediction model.
Preferably, the trajectory prediction system, wherein the trajectory prediction model is further obtained by a recurrent neural network prediction module, wherein:
the cyclic neural network prediction module is used for coding by using a long-term and short-term memory network based on the semantic historical track of the object to be predicted to obtain an updated state and a hidden state of the cyclic neural network; and inputting the coded semantic historical track into a cyclic neural network model, and decoding the semantic historical track in steps by combining the updating state and the hidden state from the current position of the object to be predicted to obtain a predicted future track.
The invention provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the steps of the trajectory prediction method when executing the program.
The invention provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the trajectory prediction method described.
According to the invention, the environment information of the self vehicle is subjected to semantic extraction, the semantic information which has the greatest influence on the prediction precision is obtained by summarizing, the semantic information which has the greatest influence on the prediction precision is obtained by utilizing the position relation of surrounding obstacles relative to the obstacles of the object to be predicted, the prediction is more accurate, the semantic history track is obtained by combining the relative history track of the object to be predicted and the environment information, the semantic history track is coded by a recurrent neural network, the state quantity of the network is transmitted, the future track of the vehicle is obtained by decoding the neural network, and the rapid and accurate prediction of the track of the obstacles can be realized on the premise of limited calculation resources of unmanned vehicles. The invention has great promotion in the aspect of unmanned driving computing power, saves tens of times or even dozens of times of computing power compared with the prior art which adopts a mode that the image belongs to a convolution model, minimizes the computing consumption in the unmanned driving operation, can ensure the prediction precision and better meets the engineering requirements.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic diagram of a trajectory prediction method provided by the present invention;
FIG. 2 is a schematic diagram of a trajectory prediction system provided by the present invention;
fig. 3 is a schematic diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic diagram of a trajectory prediction method provided by the present invention, as shown in fig. 1, the method includes:
s1: acquiring and obtaining historical positions of an object to be predicted and surrounding obstacles in a world coordinate system based on the historical position of the self-vehicle in the world coordinate system, the historical relative position of the object to be predicted relative to the self-vehicle and environmental information of the object to be predicted, wherein the environmental information comprises historical relative position information and lane information of the surrounding obstacles around the object to be predicted relative to the self-vehicle;
the object to be predicted is an object to be predicted (a vehicle around the vehicle) which needs to be predicted, the vehicle is an automatic driving vehicle, and a historical track of the object to be predicted, which is formed by historical positions of the object to be predicted in a world coordinate system, is a coordinate set of an x coordinate and a y coordinate of the object to be predicted in an utm coordinate system; utm (Universal Transverse Mercator Grid System) coordinate is a plane rectangular coordinate, and this coordinate Grid System and its projection are widely used in topographic map, as reference Grid for satellite image and natural resource database, and other applications requiring precise positioning.
The historical relative position of the object to be predicted relative to the surrounding obstacles comprises the distance between the object to be predicted and the front obstacle with the minimum distance to the object to be predicted, the distance between the object to be predicted and the rear obstacle with the minimum distance to the object to be predicted and the total number of the surrounding obstacles, and the lane information comprises the distance between the object to be predicted and the center line of the lane where the object to be predicted is located, wherein the distance is obtained through map information.
S2: according to historical positions of an object to be predicted and surrounding obstacles in a world coordinate system, obtaining and semantically extracting a relative historical track of the object to be predicted based on the historical relative position of the object to be predicted relative to the surrounding obstacles, and combining the relative historical track of the object to be predicted with lane information subjected to semantic extraction to obtain a semantic historical track;
the object to be predicted is composed of historical positions of the object to be predicted under a world coordinate systemThe historical track of (A) is a coordinate set of X coordinates and y coordinates of the object to be predicted under utm coordinate system { (X)-n+1,Y-n+1),(X-n+2,Y-n+2),...,(X-1,Y-1)，(X0,Y0) Where, X: the x coordinate of the object to be predicted under utm coordinate system; y: y coordinates of the object to be predicted under utm coordinate system; n: observing the historical frame number of an object to be predicted, wherein 0 is the current moment; the right subscript indicates time, and-n indicates the current n history frames.
At each moment, the environmental semantic features which have the greatest influence on the prediction accuracy are obtained through experimental summary, namely, the distance Df of the front nearest barrier, the distance Dr of the rear nearest barrier and the number N of the peripheral barriers of each object to be predicted are obtained through calculation according to the peripheral vehicles of the object to be predicted, and the distance L of the object to be predicted (the vehicle to be predicted) from the center line of the lane where the object to be predicted is located is obtained through a high-precision map. The environment semantic features can be extracted quickly without complex CNN network extraction, the method can minimize the calculation force requirement on the unmanned vehicle, and more accurate future track of the object to be predicted can be obtained through prediction.
Synthesizing the information to obtain semantic historical track { (X) of the object to be predicted-n+1,Y-n+1，Df-n+1，Dr-n+1，N-n+1,L-n+1),(X-n+2,Y-n+2,Df-n+2,Dr-n+2,N-n+2,L-n+2),...,(X-1,Y-1,Df-1,Dr-1,N-1,L-1),(X0,Y0,Df0,Dr0,N0,L0)}。
Wherein, X: the x coordinate of the object to be predicted under utm coordinate system;
y: y coordinates of the object to be predicted under utm coordinate system;
n: observing the historical frame number of an object to be predicted, wherein 0 is the current moment;
the right subscript indicates time, and-n indicates the current n history frames.
(X0,Y0,Df0,Dr0,N0,L0) Representing the semantic history track of the current time.
Specifically, semantic historical tracks and future tracks of objects to be predicted are acquired { (X)1,Y1),(X2,Y2),...,(XT-1,YT-1),(XT,YT) Generating a training data set, training a track prediction model, dividing one-time back propagation training into two steps, and coding by using a long-term and short-term memory network based on the historical track of an object to be predicted to obtain an updated state and a hidden state of the network; the second step is that: and (3) decoding step by step from the current position of the object to be predicted to obtain a future track, calculating deviation from the real future track of the object to be predicted, and performing error back propagation. In the running process of the automatic driving vehicle, the algorithm obtains an updated state and a hidden state by utilizing a historical track of an object to be predicted and a trained encoder, then three variables of the current position of the object to be predicted, the updated state and the hidden state are transmitted to a trained decoder, and the future track of the vehicle is obtained by step decoding.
Before the first training, the updating state and the hidden state can be defaulted without history, no value can be assigned during initialization, and a true value can be output only by using an input semantic history track for fitting. After the first training, the updated state and the hidden state can be formed and transmitted to the next cycle, and during the next training or prediction, the updated state and the hidden state obtained by learning can be utilized to be sequentially transmitted in a cycle. After the final training is completed, only the update state and the hidden state can be reserved.
S3: and inputting the semantic historical track into a track prediction model to predict the track of the object to be predicted.
The output is the track point of the future T frame of the object to be predicted, and the track point of the object to be predicted (namely one obstacle around the automatic driving vehicle) in the future 0-20s can be predicted.
The generation method of the track prediction model comprises the following steps:
constructing a recurrent neural network model;
coding by using a long-short term memory network based on the semantic historical track of the object to be predicted to obtain an updated state and a hidden state of the recurrent neural network; inputting the encoded semantic historical track into a cyclic neural network model, decoding the current position of the object to be predicted step by step to obtain a predicted future track, calculating deviation with the real future track of the object to be predicted, performing error back propagation, and training the cyclic neural network to obtain a track prediction model.
The generation method of the trajectory prediction model further comprises the following steps:
coding by using a long-short term memory network based on the semantic historical track of the object to be predicted to obtain an updated state and a hidden state of the recurrent neural network; and inputting the coded semantic historical track into a cyclic neural network model, and decoding the semantic historical track in steps by combining the updating state and the hidden state from the current position of the object to be predicted to obtain a predicted future track.
The method comprises the steps of acquiring historical relative positions of objects to be predicted and environment information of the objects to be predicted, wherein a positioning device and an obstacle sensing device are adopted, and the obstacle sensing device comprises at least one of a camera, a laser radar, a millimeter wave radar and a sensor.
According to the invention, the environment information of the object to be predicted is subjected to semantic extraction, the semantic information with the largest influence on the prediction precision is obtained by summarizing, the semantic history track is obtained by combining the relative history track of the object to be predicted and the environment information by utilizing the position relation of the peripheral obstacles relative to the object to be predicted, the prediction is more accurate and quickly obtained by utilizing the fusion result and the map information, the semantic history track is obtained by encoding the semantic history track through a recurrent neural network, the state quantity of the network is transmitted, the future track of the vehicle is obtained by decoding the neural network, and the rapid and accurate prediction of the track of the obstacle can be realized on the premise of limited calculation resources of the unmanned vehicle. The invention has great promotion in the aspect of unmanned driving computing power, saves tens of times or even dozens of times of computing power compared with the prior art which adopts a mode that the image belongs to a convolution model, minimizes the computing consumption in the unmanned driving operation, can ensure the prediction precision and better meets the engineering requirements.
The trajectory prediction system provided by the present invention is described below, and the trajectory prediction system described below and the trajectory prediction method described above may be referred to in correspondence.
Fig. 2 is a schematic diagram of a trajectory prediction system provided in the present invention, as shown in fig. 2, the system includes:
the information obtaining module 10 is configured to obtain and obtain historical positions of the object to be predicted and the surrounding obstacle in the world coordinate system based on a historical position of the own vehicle in the world coordinate system, a historical relative position of the object to be predicted with respect to the own vehicle, and environmental information of the object to be predicted, where the environmental information includes historical relative position information and lane information of the surrounding obstacle around the object to be predicted with respect to the own vehicle.
The semantic historical track generation module 20 is used for obtaining and semantically extracting a relative historical track of the object to be predicted based on the historical relative position of the object to be predicted relative to the surrounding obstacles according to the historical positions of the object to be predicted and the surrounding obstacles in a world coordinate system, and combining the relative historical track of the object to be predicted with the semantically extracted lane information to obtain a semantic historical track;
and the track prediction module 30 is configured to input the semantic historical track into a track prediction model to perform track prediction on the object to be predicted.
The track prediction model is obtained by a recurrent neural network model construction module and a recurrent neural network training module, wherein:
the recurrent neural network model building module is used for building a recurrent neural network model;
the cyclic neural network training module is used for coding by using a long-short term memory network based on the semantic historical track of the object to be predicted to obtain an updated state and a hidden state of the cyclic neural network; inputting the encoded semantic historical track into a cyclic neural network model, decoding the current position of the object to be predicted step by step to obtain a predicted future track, calculating deviation with the real future track of the object to be predicted, performing error back propagation, and training the cyclic neural network to obtain a track prediction model.
The trajectory prediction model is further obtained by a recurrent neural network prediction module, wherein:
the cyclic neural network prediction module is used for coding by using a long-term and short-term memory network based on the semantic historical track of the object to be predicted to obtain an updated state and a hidden state of the cyclic neural network; and inputting the coded semantic historical track into a cyclic neural network model, and decoding the semantic historical track in steps by combining the updating state and the hidden state from the current position of the object to be predicted to obtain a predicted future track. Multiple simulation tests can be performed by the recurrent neural network prediction module before formal use.
The information acquisition module 10 includes a positioning device and an obstacle sensing device, and the obstacle sensing device includes at least one of a camera, a laser radar, a millimeter wave radar, and a sensor.
Fig. 3 illustrates a physical structure diagram of an electronic device, which may include: a processor (processor)310, a communication Interface (communication Interface)320, a memory (memory)330 and a communication bus 340, wherein the processor 310, the communication Interface 320 and the memory 330 communicate with each other via the communication bus 340. The processor 310 may invoke logic instructions in the memory 330 to perform a trajectory prediction method comprising:
acquiring and obtaining historical positions of an object to be predicted and surrounding obstacles in a world coordinate system based on the historical position of the self-vehicle in the world coordinate system, the historical relative position of the object to be predicted relative to the self-vehicle and environmental information of the object to be predicted, wherein the environmental information comprises historical relative position information and lane information of the surrounding obstacles around the object to be predicted relative to the self-vehicle;
according to historical positions of an object to be predicted and surrounding obstacles in a world coordinate system, obtaining and semantically extracting a relative historical track of the object to be predicted based on the historical relative position of the object to be predicted relative to the surrounding obstacles, and combining the relative historical track of the object to be predicted with lane information subjected to semantic extraction to obtain a semantic historical track;
and inputting the semantic historical track into a track prediction model to predict the track of the object to be predicted. In addition, the logic instructions in the memory 330 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, the computer is capable of performing a trajectory prediction method comprising:
acquiring and obtaining historical positions of an object to be predicted and surrounding obstacles in a world coordinate system based on the historical position of the self-vehicle in the world coordinate system, the historical relative position of the object to be predicted relative to the self-vehicle and environmental information of the object to be predicted, wherein the environmental information comprises historical relative position information and lane information of the surrounding obstacles around the object to be predicted relative to the self-vehicle;
according to historical positions of an object to be predicted and surrounding obstacles in a world coordinate system, obtaining and semantically extracting a relative historical track of the object to be predicted based on the historical relative position of the object to be predicted relative to the surrounding obstacles, and combining the relative historical track of the object to be predicted with lane information subjected to semantic extraction to obtain a semantic historical track;
and inputting the semantic historical track into a track prediction model to predict the track of the object to be predicted.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program that, when executed by a processor, is implemented to perform a trajectory prediction method, the method comprising:
acquiring and obtaining historical positions of an object to be predicted and surrounding obstacles in a world coordinate system based on the historical position of the self-vehicle in the world coordinate system, the historical relative position of the object to be predicted relative to the self-vehicle and environmental information of the object to be predicted, wherein the environmental information comprises historical relative position information and lane information of the surrounding obstacles around the object to be predicted relative to the self-vehicle;
according to historical positions of an object to be predicted and surrounding obstacles in a world coordinate system, obtaining and semantically extracting a relative historical track of the object to be predicted based on the historical relative position of the object to be predicted relative to the surrounding obstacles, and combining the relative historical track of the object to be predicted with lane information subjected to semantic extraction to obtain a semantic historical track;
and inputting the semantic historical track into a track prediction model to predict the track of the object to be predicted.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A trajectory prediction method, comprising:
acquiring and obtaining historical positions of an object to be predicted and surrounding obstacles in a world coordinate system based on the historical position of the self-vehicle in the world coordinate system, the historical relative position of the object to be predicted relative to the self-vehicle and environmental information of the object to be predicted, wherein the environmental information comprises historical relative position information and lane information of the surrounding obstacles around the object to be predicted relative to the self-vehicle;
according to historical positions of an object to be predicted and surrounding obstacles in a world coordinate system, obtaining and semantically extracting a relative historical track of the object to be predicted based on the historical relative position of the object to be predicted relative to the surrounding obstacles, and combining the relative historical track of the object to be predicted with lane information subjected to semantic extraction to obtain a semantic historical track;
and inputting the semantic historical track into a track prediction model to predict the track of the object to be predicted.
2. The trajectory prediction method according to claim 1,
the historical track of the object to be predicted, which is formed by the historical positions of the object to be predicted in the world coordinate system, is a coordinate set of x coordinates and y coordinates of the object to be predicted in an utm coordinate system; the historical relative position of the object to be predicted relative to the surrounding obstacles comprises the distance between the object to be predicted and the front obstacle with the minimum distance to the object to be predicted, the distance between the object to be predicted and the rear obstacle with the minimum distance to the object to be predicted and the total number of the surrounding obstacles, and the lane information comprises the distance between the object to be predicted and the center line of the lane where the object to be predicted is located, wherein the distance is obtained through map information.
3. The trajectory prediction method according to claim 1, wherein the trajectory prediction model generation method includes:
constructing a recurrent neural network model;
coding by using a long-short term memory network based on the semantic historical track of the object to be predicted to obtain an updated state and a hidden state of the recurrent neural network; inputting the encoded semantic historical track into a cyclic neural network model, decoding the current position of the object to be predicted step by step to obtain a predicted future track, calculating deviation with the real future track of the object to be predicted, performing error back propagation, and training the cyclic neural network to obtain a track prediction model.
4. The trajectory prediction method according to claim 1, wherein the method for generating the trajectory prediction model further comprises:
coding by using a long-short term memory network based on the semantic historical track of the object to be predicted to obtain an updated state and a hidden state of the recurrent neural network; and inputting the coded semantic historical track into a cyclic neural network model, and decoding the semantic historical track in steps by combining the updating state and the hidden state from the current position of the object to be predicted to obtain a predicted future track.
5. A trajectory prediction system, comprising:
the information acquisition module is used for acquiring and obtaining historical positions of the object to be predicted and surrounding obstacles in the world coordinate system based on the historical position of the self-vehicle in the world coordinate system, the historical relative position of the object to be predicted relative to the self-vehicle and environmental information of the object to be predicted, wherein the environmental information comprises historical relative position information and lane information of the surrounding obstacles around the object to be predicted relative to the self-vehicle;
the semantic historical track generation module is used for obtaining and semantically extracting a relative historical track of the object to be predicted based on the historical relative position of the object to be predicted relative to surrounding obstacles according to the historical positions of the object to be predicted and the surrounding obstacles in a world coordinate system, and combining the relative historical track of the object to be predicted with the lane information after semantic extraction to obtain the semantic historical track;
and the track prediction module is used for inputting the semantic historical track into a track prediction model to predict the track of the object to be predicted.
6. The trajectory prediction system of claim 1, wherein the historical trajectory of the object to be predicted, which is composed of the historical positions of the object to be predicted in the world coordinate system, is a coordinate set of x and y coordinates of the object to be predicted in utm coordinate system; the historical relative position of the object to be predicted relative to the surrounding obstacles comprises the distance between the object to be predicted and the front obstacle with the minimum distance to the object to be predicted, the distance between the object to be predicted and the rear obstacle with the minimum distance to the object to be predicted and the total number of the surrounding obstacles, and the lane information comprises the distance between the object to be predicted and the center line of the lane where the object to be predicted is located, wherein the distance is obtained through map information.
7. The trajectory prediction system of claim 1, wherein the trajectory prediction model is derived by a recurrent neural network model building module and a recurrent neural network training module, wherein:
the recurrent neural network model building module is used for building a recurrent neural network model;
the cyclic neural network training module is used for coding by using a long-short term memory network based on the semantic historical track of the object to be predicted to obtain an updated state and a hidden state of the cyclic neural network; inputting the encoded semantic historical track into a cyclic neural network model, decoding the current position of the object to be predicted step by step to obtain a predicted future track, calculating deviation with the real future track of the object to be predicted, performing error back propagation, and training the cyclic neural network to obtain a track prediction model.
8. The trajectory prediction system of claim 3, wherein the trajectory prediction model is further derived by a recurrent neural network prediction module, wherein:
the cyclic neural network prediction module is used for coding by using a long-term and short-term memory network based on the semantic historical track of the object to be predicted to obtain an updated state and a hidden state of the cyclic neural network; and inputting the coded semantic historical track into a cyclic neural network model, and decoding the semantic historical track in steps by combining the updating state and the hidden state from the current position of the object to be predicted to obtain a predicted future track.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the trajectory prediction method according to any of claims 1-4 are implemented when the program is executed by the processor.
10. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the trajectory prediction method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011566859.0A CN112651557A (en) | 2020-12-25 | 2020-12-25 | Trajectory prediction system and method, electronic device and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011566859.0A CN112651557A (en) | 2020-12-25 | 2020-12-25 | Trajectory prediction system and method, electronic device and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112651557A true CN112651557A (en) | 2021-04-13 |
Family
ID=75363094
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011566859.0A Pending CN112651557A (en) | 2020-12-25 | 2020-12-25 | Trajectory prediction system and method, electronic device and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112651557A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113467462A (en) * | 2021-07-14 | 2021-10-01 | 中国人民解放军国防科技大学 | Pedestrian accompanying control method and device for robot, mobile robot and medium |
| CN113670310A (en) * | 2021-07-27 | 2021-11-19 | 际络科技(上海)有限公司 | Visual voice navigation method, device, equipment and storage medium |
| CN113715814A (en) * | 2021-09-02 | 2021-11-30 | 北京百度网讯科技有限公司 | Collision detection method, collision detection device, electronic apparatus, medium, and autonomous vehicle |
| CN113740837A (en) * | 2021-09-01 | 2021-12-03 | 广州文远知行科技有限公司 | Obstacle tracking method, device, equipment and storage medium |
| CN113753078A (en) * | 2021-08-17 | 2021-12-07 | 北京百度网讯科技有限公司 | Method and device for predicting obstacle trajectory, electronic equipment and automatic driving vehicle |
| CN114001976A (en) * | 2021-10-19 | 2022-02-01 | 杭州飞步科技有限公司 | Method, device and equipment for determining control error and storage medium |
| CN114155720A (en) * | 2021-11-29 | 2022-03-08 | 上海交通大学 | Vehicle detection and track prediction method for roadside laser radar |
| CN114239974A (en) * | 2021-12-21 | 2022-03-25 | 清华大学 | Multi-agent position prediction method and device, electronic equipment and storage medium |
| CN114802303A (en) * | 2022-04-26 | 2022-07-29 | 中国第一汽车股份有限公司 | Obstacle trajectory prediction method, obstacle trajectory prediction device, electronic device, and storage medium |
| CN115148025A (en) * | 2022-06-28 | 2022-10-04 | 重庆长安汽车股份有限公司 | Traffic target track prediction method and readable storage medium |
| CN115540893A (en) * | 2022-11-30 | 2022-12-30 | 广汽埃安新能源汽车股份有限公司 | Vehicle path planning method and device, electronic equipment and computer readable medium |
| CN115790606A (en) * | 2023-01-09 | 2023-03-14 | 深圳鹏行智能研究有限公司 | Trajectory prediction method, trajectory prediction device, robot, and storage medium |
| CN115943400A (en) * | 2021-04-28 | 2023-04-07 | 深圳元戎启行科技有限公司 | Trajectory prediction method and device based on time and space learning and computer equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108981726A (en) * | 2018-06-09 | 2018-12-11 | 安徽宇锋智能科技有限公司 | Unmanned vehicle semanteme Map building and building application method based on perceptual positioning monitoring |
| CN110223318A (en) * | 2019-04-28 | 2019-09-10 | 驭势科技（北京）有限公司 | A kind of prediction technique of multi-target track, device, mobile unit and storage medium |
| CN110371112A (en) * | 2019-07-06 | 2019-10-25 | 深圳数翔科技有限公司 | A kind of intelligent barrier avoiding system and method for automatic driving vehicle |
| CN111079721A (en) * | 2020-03-23 | 2020-04-28 | 北京三快在线科技有限公司 | Method and device for predicting track of obstacle |
| CN111707258A (en) * | 2020-06-15 | 2020-09-25 | 中国第一汽车股份有限公司 | External vehicle monitoring method, device, equipment and storage medium |
| CN111930110A (en) * | 2020-06-01 | 2020-11-13 | 西安理工大学 | Intent track prediction method for generating confrontation network by combining society |
| CN112015847A (en) * | 2020-10-19 | 2020-12-01 | 北京三快在线科技有限公司 | Obstacle trajectory prediction method and device, storage medium and electronic equipment |
-
2020
- 2020-12-25 CN CN202011566859.0A patent/CN112651557A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108981726A (en) * | 2018-06-09 | 2018-12-11 | 安徽宇锋智能科技有限公司 | Unmanned vehicle semanteme Map building and building application method based on perceptual positioning monitoring |
| CN110223318A (en) * | 2019-04-28 | 2019-09-10 | 驭势科技（北京）有限公司 | A kind of prediction technique of multi-target track, device, mobile unit and storage medium |
| CN110371112A (en) * | 2019-07-06 | 2019-10-25 | 深圳数翔科技有限公司 | A kind of intelligent barrier avoiding system and method for automatic driving vehicle |
| CN111079721A (en) * | 2020-03-23 | 2020-04-28 | 北京三快在线科技有限公司 | Method and device for predicting track of obstacle |
| CN111930110A (en) * | 2020-06-01 | 2020-11-13 | 西安理工大学 | Intent track prediction method for generating confrontation network by combining society |
| CN111707258A (en) * | 2020-06-15 | 2020-09-25 | 中国第一汽车股份有限公司 | External vehicle monitoring method, device, equipment and storage medium |
| CN112015847A (en) * | 2020-10-19 | 2020-12-01 | 北京三快在线科技有限公司 | Obstacle trajectory prediction method and device, storage medium and electronic equipment |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115943400A (en) * | 2021-04-28 | 2023-04-07 | 深圳元戎启行科技有限公司 | Trajectory prediction method and device based on time and space learning and computer equipment |
| CN113467462A (en) * | 2021-07-14 | 2021-10-01 | 中国人民解放军国防科技大学 | Pedestrian accompanying control method and device for robot, mobile robot and medium |
| CN113670310A (en) * | 2021-07-27 | 2021-11-19 | 际络科技(上海)有限公司 | Visual voice navigation method, device, equipment and storage medium |
| CN113670310B (en) * | 2021-07-27 | 2024-05-31 | 际络科技(上海)有限公司 | Visual voice navigation method, device, equipment and storage medium |
| CN113753078A (en) * | 2021-08-17 | 2021-12-07 | 北京百度网讯科技有限公司 | Method and device for predicting obstacle trajectory, electronic equipment and automatic driving vehicle |
| CN113753078B (en) * | 2021-08-17 | 2023-09-01 | 北京百度网讯科技有限公司 | Obstacle track prediction method and device, electronic equipment and automatic driving vehicle |
| CN113740837A (en) * | 2021-09-01 | 2021-12-03 | 广州文远知行科技有限公司 | Obstacle tracking method, device, equipment and storage medium |
| CN113740837B (en) * | 2021-09-01 | 2022-06-24 | 广州文远知行科技有限公司 | Obstacle tracking method, device, equipment and storage medium |
| CN113715814A (en) * | 2021-09-02 | 2021-11-30 | 北京百度网讯科技有限公司 | Collision detection method, collision detection device, electronic apparatus, medium, and autonomous vehicle |
| CN114001976A (en) * | 2021-10-19 | 2022-02-01 | 杭州飞步科技有限公司 | Method, device and equipment for determining control error and storage medium |
| CN114001976B (en) * | 2021-10-19 | 2024-03-12 | 杭州飞步科技有限公司 | Method, device, equipment and storage medium for determining control error |
| CN114155720A (en) * | 2021-11-29 | 2022-03-08 | 上海交通大学 | Vehicle detection and track prediction method for roadside laser radar |
| CN114239974B (en) * | 2021-12-21 | 2022-10-25 | 清华大学 | Multi-agent position prediction method and device, electronic equipment and storage medium |
| CN114239974A (en) * | 2021-12-21 | 2022-03-25 | 清华大学 | Multi-agent position prediction method and device, electronic equipment and storage medium |
| CN114802303A (en) * | 2022-04-26 | 2022-07-29 | 中国第一汽车股份有限公司 | Obstacle trajectory prediction method, obstacle trajectory prediction device, electronic device, and storage medium |
| CN115148025A (en) * | 2022-06-28 | 2022-10-04 | 重庆长安汽车股份有限公司 | Traffic target track prediction method and readable storage medium |
| CN115148025B (en) * | 2022-06-28 | 2023-10-20 | 重庆长安汽车股份有限公司 | Traffic target track prediction method and readable storage medium |
| CN115540893A (en) * | 2022-11-30 | 2022-12-30 | 广汽埃安新能源汽车股份有限公司 | Vehicle path planning method and device, electronic equipment and computer readable medium |
| CN115540893B (en) * | 2022-11-30 | 2023-03-14 | 广汽埃安新能源汽车股份有限公司 | Vehicle path planning method and device, electronic equipment and computer readable medium |
| CN115790606A (en) * | 2023-01-09 | 2023-03-14 | 深圳鹏行智能研究有限公司 | Trajectory prediction method, trajectory prediction device, robot, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112651557A (en) | Trajectory prediction system and method, electronic device and readable storage medium | |
| CN108921200B (en) | Method, apparatus, device and medium for classifying driving scene data | |
| CN112651990B (en) | Motion trajectory prediction method and system, electronic device and readable storage medium | |
| CN115690153A (en) | Intelligent agent track prediction method and system | |
| CN116309689B (en) | Obstacle track prediction method, device, equipment and medium | |
| CN116403174A (en) | End-to-end automatic driving method, system, simulation system and storage medium | |
| CN116595871A (en) | Vehicle track prediction modeling method and device based on dynamic space-time interaction diagram | |
| CN114715145B (en) | Trajectory prediction method, device and equipment and automatic driving vehicle | |
| CN116092055A (en) | Training method, acquisition method, device, equipment and automatic driving vehicle | |
| CN114584949B (en) | Method and equipment for determining attribute value of obstacle through vehicle-road cooperation and automatic driving vehicle | |
| CN114104005B (en) | Decision-making method, device and equipment of automatic driving equipment and readable storage medium | |
| CN116300928A (en) | Data processing method for vehicle and training method for data processing model | |
| CN114792320A (en) | Trajectory prediction method, trajectory prediction device and electronic equipment | |
| CN115719547A (en) | Traffic participant trajectory prediction method and system based on multiple interactive behaviors | |
| CN116071925B (en) | Track prediction method and device and electronic processing device | |
| CN114141009B (en) | Simulation traffic flow lane changing method and system based on multi-time sequence network | |
| CN116663329B (en) | Automatic driving simulation test scene generation method, device, equipment and storage medium | |
| CN114596552B (en) | Information processing method, training method, device, equipment, vehicle and medium | |
| CN117649665B (en) | Mining area automatic driving scene data labeling method and system | |
| CN115081186B (en) | Driving behavior simulation system supporting data driving and simulation method thereof | |
| CN115628736B (en) | Pedestrian track prediction method, equipment, mobile device and storage medium | |
| US20240300542A1 (en) | Trajectory prediction by sampling sequences of discrete motion tokens | |
| EP4099226A1 (en) | Training method for training a cvae to predict future properties of agents, prediction method, computer program(s), computer readable medium, future properties prediction system | |
| CN116946159A (en) | Motion trail prediction method, device, equipment and medium based on dynamic local map | |
| CN117746417A (en) | Target detection model construction method, target detection method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210413 |