WO2022088658A1

WO2022088658A1 - Pedestrian crossing intention estimation method and apparatus, device, and vehicle

Info

Publication number: WO2022088658A1
Application number: PCT/CN2021/095259
Authority: WO
Inventors: 范时伟; 李飞; 李向旭
Original assignee: 华为技术有限公司
Priority date: 2020-10-29
Filing date: 2021-05-21
Publication date: 2022-05-05
Also published as: CN114426030B; CN114426030A

Abstract

A pedestrian crossing intention estimation method and apparatus, a device, and a vehicle, relating to the field of smart driving. The method comprises: obtaining status information of at least one pedestrian around a vehicle (S301); according to pedestrian status information predicted at a previous time and an apparent intention of the pedestrian at the previous time, determining pedestrian status information predicted at a current time (S302); calculating a deep intention at the current time according to the pedestrian status information predicted at the current time, pedestrian status information observed at the current time and a deep intention of the pedestrian at the previous time (S303); according to status information of a first pedestrian at the current time and a deep intention of the first pedestrian at the previous time, calculating an apparent intention of the first pedestrian to cross a road (S304); and according to the status information of the pedestrian at the current time and the apparent intention at the current time, calculating a lane intention of each lane of the road where a vehicle is located (S305). By means of the method, a high-precision, real-time and accurate apparent intention estimation result is obtained, and then, whether the first pedestrian crosses the road at a first time is determined according to the apparent intention of the first pedestrian at the first time.

Description

A pedestrian crossing intention estimation method, device, device and car

This application claims the priority of the Chinese patent application filed on October 29, 2020 with the State Intellectual Property Office of the People's Republic of China, the application number is 202011176754.4, and the application title is "A method, device, equipment and automobile for estimating pedestrian crossing intention", all of which are The contents are incorporated herein by reference.

technical field

The present invention relates to the technical field of intelligent driving, and in particular, to a method, device, device and automobile for estimating pedestrian's passing intention.

Background technique

With the advent of smart driving, smart cars have become the focus of research by major manufacturers. For intelligent driving, its realization process mainly includes steps such as positioning, perception, prediction, planning and control. Among them, the main function of the prediction step is to estimate the future position and future behavior of the surrounding objects, so that the planning control module can make corresponding decisions, thereby avoiding the occurrence of traffic accidents.

However, due to the characteristics of pedestrians with high dynamics and less constraints from maps and traffic rules, it is quite challenging to predict pedestrian trajectories and intentions. If the prediction is inaccurate, the vehicle is prone to collision incidents, which brings great hidden dangers to the vehicle.

SUMMARY OF THE INVENTION

In order to solve the above-mentioned problems, the embodiments of the present application provide a method, apparatus, device, and vehicle for estimating pedestrian's passing intention.

In a first aspect, the present application provides a method for estimating a pedestrian's passing intention, including: acquiring state information of at least one pedestrian around a vehicle, where the at least one pedestrian includes a first pedestrian; according to the state information of the first pedestrian at a first moment and the deep intent of the first pedestrian at the second moment, calculate the surface intent of the first pedestrian at the first moment, the second moment is before the first moment, and the deep intent is the first pedestrian The probability of crossing the road in the entire life cycle, the surface intent is the probability that the first pedestrian is or is about to cross the road at the current moment; according to the surface intent of the first pedestrian at the first moment, determine the Whether the first pedestrian crosses the road at the first moment.

In this embodiment, by continuously acquiring information such as pedestrians on the road where the vehicle is traveling, lane lines, lane markings, and other surrounding vehicles, etc., the behavior characteristics, static interaction characteristics, dynamic interaction characteristics, etc. of pedestrians at the current moment are analyzed. After the state information, combined with the deep intent at the previous moment, the surface intent estimation result with high precision, real-time and accuracy can be obtained.

In an embodiment, before calculating the surface intention of the first pedestrian at the first moment according to the state information of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment, the method includes: : Calculate the likelihood probability of the pedestrian feature of the first pedestrian at the first moment according to the state information of the first pedestrian at the first moment; The deep intention of the first pedestrian at the moment, calculating the surface intention of the first pedestrian at the first moment, including: according to the likelihood probability of the pedestrian feature of the first pedestrian at the first moment and the second moment For the deep intention of the first pedestrian, the surface intention of the first pedestrian at the first moment is calculated.

In this embodiment, in the process of acquiring the state information of pedestrians, sometimes the monitored information may not be completely correct, and errors or errors may occur. Errors such as pedestrian detection and pedestrian speed being detected will have a greater impact on pedestrian characteristics. Therefore, after obtaining the pedestrian status information, various detected information are fused and filtered to smooth the error to a certain extent. , so that the pedestrian features corresponding to the detected pedestrian state information are more realistic.

In one embodiment, the method further includes: when the surface intention of the first pedestrian at the first moment is greater than a set threshold, controlling the speaker to play an early warning signal and/or displaying the warning information on the display screen.

In an implementation manner, before calculating the surface intention of the first pedestrian at the first moment according to the state information of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment, the The method includes: calculating the state information of the predicted first pedestrian at the first moment according to the state information of the first pedestrian at the second moment and the surface intention of the first pedestrian at the second moment; The state information of the first pedestrian, the state information of the first pedestrian at the first moment, and the deep intention of the first pedestrian at the second moment are predicted, and the deep intention of the first pedestrian at the first moment is calculated.

In this embodiment, the deep intent at the previous moment is re-estimated and updated according to the deep intent at the previous moment, the state information of the pedestrian at the current moment, and the state information of the pedestrian predicted at the current moment, so that the new deep intent is reliable. It increases with the accumulation of historical observational information.

In one embodiment, the method further includes: calculating at least one of the roads on which the vehicle is located according to the state information of the first pedestrian at the first moment and the surface intention of the first pedestrian at the first moment Lane intent of the lane, where the lane intent is the probability that the first pedestrian traverses the first lane, and the at least one lane includes the first lane.

In this embodiment, according to the state information of the first pedestrian at the first moment, the road structure of each lane and related vehicles are calculated to extract the lane-level static interaction features and dynamic interaction features, as well as the behavior features of the target pedestrian. , and then calculate the pedestrian's lane intention, so that the vehicle can output the pedestrian's passing intention relative to each lane, so that the autonomous vehicle can respond in advance.

In a second aspect, the present application further provides an apparatus for estimating a pedestrian's passing intention, including: an acquisition unit for acquiring status information of at least one pedestrian around the vehicle, the at least one pedestrian including the first pedestrian; and a processing unit for According to the state information of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment, the surface intention of the first pedestrian at the first moment is calculated, and the second moment is before the first moment , the deep intention is the probability of the first pedestrian crossing the road in the entire life cycle, and the surface intention is the probability that the first pedestrian is or is about to cross the road at the current moment; the processing unit, also for determining whether the first pedestrian crosses the road at the first moment according to the surface intention of the first pedestrian at the first moment.

In an embodiment, the processing unit is specifically configured to calculate the likelihood probability of the pedestrian feature of the first pedestrian at the first moment according to the state information of the first pedestrian at the first moment; The likelihood probability of the pedestrian feature of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment are used to calculate the surface intention of the first pedestrian at the first moment.

In one embodiment, the processing unit is further configured to control the speaker to play an early warning signal and/or display the warning information on the display screen when the surface intention of the first pedestrian at the first moment is greater than a set threshold .

In an embodiment, the processing unit is further configured to calculate the predicted first pedestrian at the first moment according to the state information of the first pedestrian at the second moment and the surface intention of the first pedestrian at the second moment According to the state information of the predicted first pedestrian at the first moment, the state information of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment, calculate the first moment The deep intent of the first pedestrian.

In an implementation manner, the processing unit is further configured to calculate, according to the state information of the first pedestrian at the first moment and the surface intention of the first pedestrian at the first moment, the speed of the road where the vehicle is located. Lane intent of at least one lane, the lane intent being the probability of the first pedestrian crossing the first lane, the at least one lane including the first lane.

In a third aspect, the present application further provides a device, including at least one processor, where the processor is configured to execute instructions stored in the memory, so that the terminal executes each possible implementation of the first aspect.

In a fourth aspect, the present application also provides an automobile for implementing the various possible embodiments of the first aspect.

In a fifth aspect, the present application further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed in a computer, the computer is made to execute the various possible embodiments of the first aspect.

In a sixth aspect, the present application further provides a computing device including a memory and a processor, wherein the memory stores executable codes, and when the processor executes the executable codes, the first Aspects of various possible implementations.

Description of drawings

The following briefly introduces the accompanying drawings required in the description of the embodiments or the prior art.

1 is a schematic structural diagram of a vehicle according to an embodiment of the present application;

Fig. 2 is a kind of scene of the vehicle driving on the road;

FIG. 3 is a process flow diagram of a method for estimating pedestrian passing intention provided by an embodiment of the present application;

4 is a schematic diagram of modeling surface-level intentions and deep-level intentions through a dynamic Bayesian network structure provided by an embodiment of the present application;

Figure 5 is a scene of vehicles and pedestrians on the road;

FIG. 6 is a schematic structural diagram of an apparatus for estimating a pedestrian's passing intention according to an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.

In the embodiments of the present application, two concepts will be mentioned later, namely "surface intent" and "deep intent". Among them, the superficial intention is the probability that the pedestrian is or is about to cross the road at the current moment, that is, the pedestrian’s intention to cross the road in an instant or a short time; Long-term consideration of travel intentions with low environmental impact.

Table 1 Comparison of surface intent and deep intent

Since the surface intent represents the pedestrian's current passing intent, the surface intent is very important for real-time performance. Therefore, in the embodiment of the present application, the surface intent is estimated by extracting the pedestrian's detailed information and interaction information. However, since the extracted detail information and interactive information are usually noisy, which easily affects the stability of the surface intent estimation, the embodiment of the present application also introduces a deep intent, and uses the surface intent data and pedestrian feature information at the previous moment to calculate Then, based on the data of the deep intention, the stability of the surface intention is improved, so that the calculated surface intention can more accurately predict the pedestrian's intention to cross the road.

FIG. 1 is a frame structure diagram of a vehicle according to an embodiment of the present application. As shown in FIG. 1 , the vehicle 100 includes an input device 101 , a memory 102 , a processor 103 and a bus 104 . Wherein, the input device 101 , the memory 102 and the processor 103 in the vehicle 100 can establish a communication connection through the bus 104 .

The input device 101 may include a vehicle camera, a vehicle radar, a vehicle navigation, a global positioning system (global positioning system, GPS) sensor and other devices. Among them, the on-board camera is used to photograph the driving direction of the vehicle 100 to obtain images including information such as lane lines, lane markings, pedestrians, etc. of the road; the on-board radar can be laser radar, millimeter-wave radar, etc. Transmit signals and receive returned signals to obtain distance information between the vehicle 100 and other vehicles, pedestrians and other obstacles around the vehicle 100; in-vehicle navigation is used to generate navigation based on the location of the vehicle 100 and the destination location information input by the driver Route; GPS sensors are used to obtain real-time location information of the vehicle 100 .

The memory 102 can be a random-access memory (RAM), a hard disk drive (HDD), a solid state drive (SSD), or other devices with storage functions, for storing surface-level intentions and deep-level intentions , high-precision maps, etc., in order to use historically stored data for subsequent calculations in the process of calculating surface intent and deep intent.

The processor 103 may be an electronic control unit (ECU) of the vehicle 100. After receiving each data sent by the input device 101, the processor 103 performs processing to obtain the status of the pedestrians on the road in front of the vehicle 100 and on both sides of the road. information, road lane lines, lane markings and other information, and then according to the pedestrian state information, (optionally, the pedestrian's surface intention stored in the memory 102 at the previous moment) and the deep intention stored in the memory 102 at the previous moment, The surface intention of the pedestrian crossing the road in the road in front of the vehicle 100 and on both sides of the road is calculated, and the probability of the pedestrian crossing the road is obtained. When the probability of the pedestrian crossing the road is greater than the set threshold, a warning message is generated and displayed on the display screen or played on the speaker to remind the driver to pay attention to the pedestrian crossing on the road ahead.

The processor 103 is further configured to, according to the deep intention of the previous moment stored in the memory 102, the state information of the pedestrian at the current moment, and the state information of the pedestrian predicted at the current moment, perform the processing on the deep intention of the previous moment stored in the memory 102. Re-estimate and update to obtain the new deep intention of the current moment, so that its reliability increases with the accumulation of historical observation information.

The processor 103 is further configured to calculate the probability of the pedestrian crossing each lane according to the received state information of the pedestrian, the lane lines of the road, lane markings and other information and the new deep intention at the current moment stored in the memory 102 (subsequent Known as "lane intent"), it determines which lanes on the current road are drivable, which lanes are dangerous for pedestrians to cross, and instructs the driver how to avoid the danger in the event of a hazard.

Wherein, the processor 103 of the present application may also be a server, the vehicle 100 communicates with the server through a communication unit, and then the function of the processor 103 is replaced by the server.

Figure 2 is a scene of a vehicle driving on a road. The vehicle 100 is driving in the inner lane of the road where the vehicle 100 is traveling, there are no other vehicles on the road where the vehicle 100 is traveling in the same direction, and a pedestrian is about to cross the road on the left front of the road where the vehicle 100 is traveling. The following embodiments of the present application describe the implementation process of the technical solution of the present application by taking the scene where the vehicle 100 shown in FIG. 2 is located as an example.

FIG. 3 is a process flow chart of a method for estimating a pedestrian's passing intention provided by an embodiment of the present application. As shown in FIG. 3 , the specific implementation process of the processor 103 is as follows:

Step S301 , the processor 103 acquires the status information of at least one pedestrian around the vehicle 100 . In the present application, the extracted state information of pedestrians mainly includes three types of features: behavioral features of pedestrians, static interactive features of pedestrians, and dynamic interactive features of pedestrians.

The behavior feature of the pedestrian represents the detailed action of the pedestrian observed by the vehicle 100 , including the pedestrian's motion state, body orientation, body language, and the like. Exemplarily, after the camera of the vehicle 100 obtains the video, the visual perception module (in the processor 100) uses algorithms such as regional multi-person pose estimation (RMPE) to first detect pedestrians in the video, The bounding box of each pedestrian is obtained, then human key points are detected in each bounding box, and each key point in each bounding box is connected into the pose of each pedestrian. At the same time, the processor 103 calculates the distance between each pedestrian and the vehicle 100 according to algorithms such as monocular ranging and binocular ranging. The processor 103 calculates the pedestrian's orientation relative to the vehicle 100 according to the pedestrian's key points and posture, and then calculates the pedestrian's relative direction to the road as the pedestrian's orientation feature; the processor 103 uses the pedestrian's key points and posture to infer the pedestrian's motion characteristics, The motion or static information of the pedestrian is used as the motion feature of the pedestrian; the processor 103 analyzes the key points of the pedestrian's hand, and obtains the pedestrian's body language, such as waving, pointing and other actions, as the body language feature of the pedestrian. The above three features reflect the detailed information of pedestrians, have strong real-time performance, and are used to judge the surface intention of pedestrians.

The static interaction features of pedestrians represent information about pedestrians and the surrounding road environment. Exemplarily, the processor 103 of the vehicle 100 can obtain the distance between the pedestrian and the vehicle 100 by sending and receiving millimeter-wave signals through radar (taking a millimeter-wave radar as an example), images captured by a camera, etc. The position of the pedestrian, determine the distance between the pedestrian and the road boundary, the distance between the pedestrian and the lane where the vehicle 100 is located, the distance between the pedestrian and each lane line in the road, and other information, and then use these information as the pedestrian's static interactive features.

The dynamic interaction feature of the pedestrian represents the interaction information between the pedestrian and surrounding dynamic objects, such as the vehicle 100 . Exemplarily, the processor 102 of the vehicle 100 obtains information such as the distance, speed, and relative position of surrounding pedestrians relative to the vehicle 100 , and information such as the distance, speed, and relative position of other vehicles relative to the vehicle 100 through devices such as cameras and radars. . It then predicts the trajectories of pedestrians and other vehicles over a period of time. Finally, the processor 103 calculates information such as the minimum distance between each pedestrian and each vehicle, the collision time and other information according to the motion trajectories of pedestrians and other vehicles, and then uses these information as dynamic interaction characteristics of pedestrians.

Step S304, the processor 103 calculates the surface intention of the first pedestrian crossing the road according to the state information of the first pedestrian at the current moment and the deep intention of the first pedestrian at the previous moment. The first pedestrian is the pedestrian on the left front of the road in the driving direction of the vehicle 100 in FIG. 2 , the current time corresponds to the above-mentioned second time, and the previous time corresponds to the above-mentioned first time.

In an automatic driving scenario, the vehicle 100 can obtain the motion information of other vehicles and obstacles around it through various input devices 101, and calculate the interaction information between it and the pedestrian, as a basis for judging the pedestrian's passing intention. However, sometimes the information monitored by the input device 101 may not be completely correct, and errors or errors may occur. For example, because the pedestrian target is small and moving at a slow speed, the input device 101 may not necessarily detect the pedestrian, and the detected speed of the pedestrian remains unchanged. This will have a greater impact on pedestrian characteristics. Therefore, after obtaining the pedestrian's state information, this application fuses and filters various detected information, and smoothes the error to a certain extent, so that the pedestrian's state information is detected. The corresponding pedestrian features are more realistic. Exemplarily, the present application combines three types of features: pedestrian behavioral features, pedestrian static interaction features, and pedestrian dynamic interaction features, and calculates the likelihood probability of pedestrian feature Z by formula (1), that is:

Among them, M _t represents the pedestrian's walking intention at time t, DTC _t represents the static interaction feature of the pedestrian at time t, D ^min _t represents the dynamic interaction feature of the pedestrian at time t, OR _t represents the orientation of the pedestrian at time t, MO _t represents the pedestrian's motion characteristics at time t, and BL _t represents the pedestrian's body language at time t. The likelihood probability of each feature is estimated by maximum likelihood through a large number of data samples, and the probability parameter is calculated.

In the process of calculating the surface intention of pedestrians crossing the road at the current moment, the present application uses the dynamic Bayesian network structure to model the calculation of the surface intention of pedestrians and the deep intention of pedestrians. Pedestrian state information Y and predicted pedestrian state information X are used as input information, and then in the order of time, the model shown in Figure 4 is constructed according to the interaction between deep intent and surface intent. As shown in Figure 4, the process of calculating the surface intention M _t at time t is related to the surface intention M _t-1 at the previous time, the pedestrian feature Z _t at time t, and the deep intention D t at time _t .

Exemplarily, taking the pedestrian's state information as the pedestrian's position and speed as an example, the processor 103 calculates the surface intention M _t at time t and obtains it by formula (2), specifically:

Among them, P(M _t |M _t-1 ) represents the transition probability of the surface intention, and the transition probability of the surface intention represents the influence of the historical surface intention on the current surface intention, which plays the role of smoothing the intention; P(M _t |Z _t-1 ) represents the likelihood probability of pedestrian features, and the likelihood probability of pedestrian features represents the observed pedestrian behavior characteristics and interaction characteristics to infer real-time pedestrian crossing intentions; P(D _t , D _t-1 , X _t , Y _t ) represent deep intentions, and deep intentions represent the influence of pedestrian deep intentions on surface intentions.

If the detected pedestrian is detected for the first time and the historical deep intention of the pedestrian is not stored in the memory 102 at this time, an initial value of the deep intention, such as 0.5, is stored in the memory 102 . When the processor 103 calculates the surface intention, the initial value is selected to calculate the surface intention, and the calculated surface intention at this time is mainly determined by the characteristics of the pedestrian at the current moment.

After obtaining the superficial intention, the processor 103 determines whether the probability of the calculated superficial intention exceeds the set threshold, and if it is greater than the set threshold, the processor 103 can play voice through the speaker, display prompt information through the display screen, etc. to remind the driver The operator can also directly control the vehicle 100 to slow down, avoid pedestrians passing by, and other behaviors, so that the vehicle 100 can drive safely on the current road.

In the embodiment of the present application, various types of input devices 101 on the vehicle 100 are used to continuously acquire information such as pedestrians on the road where the vehicle 100 is traveling, lane lines, lane markings, and other surrounding vehicles, and then analyze the current moment of pedestrians' information. After the behavior features, static interaction features, dynamic interaction features and other information, combined with the surface intent at the last moment and the deep intent at the last moment stored in the memory 102, a surface intent estimation with high precision, real-time and accuracy can be obtained result.

Among them, the accuracy refers to the use of various factors to predict pedestrians, and to mine the deep intentions of pedestrians, reducing the impact of noise in pedestrian perception. Timeliness refers to using real-time elements such as detailed information and interactive information as the basis for judging surface intentions, and using hysteresis elements such as tracking information to adjust deep intentions, which improves the accuracy of predictions. Timeliness.

In this embodiment of the present application, the processor 103 further implements the following process before calculating the surface intent at the current moment in step S304:

Step S302, the processor 103 determines the pedestrian state information predicted at the current time according to the pedestrian state information predicted at the previous time and the pedestrian's surface intention at the previous time.

Specifically, the processor 103 first obtains the speed and position of the pedestrian state information predicted at the last moment, and then determines whether the pedestrian is crossing the road according to the surface intention at the last moment, and then the processor 103 determines whether the pedestrian is crossing the road according to the predicted pedestrian's speed at the last moment. Speed and position, as well as the time difference between the previous moment and the current moment, calculate the speed and position of the predicted pedestrian at the current moment, and obtain the pedestrian status information at the current moment.

Step S303, the processor 103 calculates the deep intention at the current moment according to the predicted pedestrian state information at the current moment, the observed pedestrian status information at the current moment, and the deep intention of the pedestrian at the previous moment.

Wherein, the deep intention referred to in this application refers to whether the pedestrian actually has a passing intention within the life cycle observed by the vehicle 100 , and the intention is a long-term stable intention. In practice, the deep intention of pedestrians is deterministic and does not change with time, but in the process of intention estimation, due to the unobservability of deep intentions and the reliability of reference information, the node is gradual and convergent in the estimation process. The reliability of the process increases with the accumulation of historical observational information.

Specifically, since the deep intent has higher requirements on stability and accuracy, and lower requirements on real-time performance, this application mainly uses the predicted state information of the pedestrian at the previous moment, the actual status information of the pedestrian at the current moment, the surface layer intent at the current moment, and The deep intention of the pedestrian at the previous moment is estimated and updated to the deep intention of the current moment.

Exemplarily, taking the pedestrian's state information as the pedestrian's position and speed as an example, as shown in Fig. 4 , the processor 103 uses the uniform speed model to predict the pedestrian's motion trajectory, and by comparing the pedestrian's position and speed information predicted at the previous moment with The surface intention t of the pedestrian at the previous moment is input into the uniform speed model, and the predicted position and speed information of the pedestrian at the current moment are calculated, and then the correction of the deep intention is calculated by formula (3) according to the position and speed information of the pedestrian observed at the current moment. amount, that is:

The processor 103 corrects the deep intention of the pedestrian obtained at the previous moment according to the obtained correction amount of the deep intention, and obtains the deep intention of the current moment.

Formula (3) can take the lane direction of the road in the direction in which the vehicle 100 travels as a reference line, and take the actual position information of the pedestrian at the current moment and the predicted position information of the pedestrian at the current moment as the horizontal and vertical directions, respectively. At this time, the calculation The formula is:

Among them, x represents the predicted position of the pedestrian at the current moment, y represents the actual position of the pedestrian at the current moment, v _x represents the predicted speed of the pedestrian at the current moment, v _y represents the actual speed of the pedestrian at the current moment, and L represents the direction perpendicular to the lane , S represents the direction parallel to the lane. Equation (4) above expresses the effect of the difference between the pedestrian's motion along the lane and perpendicular to the lane and the predicted motion on the deep intent. For example, when the pedestrian's movement distance along the vertical lane is greater than the movement distance predicted by the historical intention, the calculated deep intention is that the probability of passing through is greater than the probability of non-travelling. The deep intent of the moment.

In addition, in the embodiment of the present application, after the processor 103 calculates the surface intention of the pedestrian at the current moment, if it is determined that there is a pedestrian crossing the road, it needs to determine how the vehicle 100 avoids the pedestrian. The specific implementation process is as follows:

Step S305 , the processor 103 calculates the lane intention of each lane of the road where the vehicle 100 is located according to the current state information of the pedestrian and the surface intention at the current time.

Specifically, when the processor 103 determines, according to the surface intention at the current moment, that the probability of the surface intention is greater than the set threshold, it indicates that a pedestrian is about to cross the road, and continues to use the pedestrian position, pedestrian speed, and pedestrian distance in the pedestrian status information. The distance between the lanes, the distance between the pedestrian and the vehicle 100 and other information, calculate the probability of the vehicle 100 colliding with the pedestrian in this lane, and the probability of the vehicle 100 turning to other lanes to collide with the pedestrian, so as to calculate the vehicle 100. Lane intent for each lane of the road where 100 is located.

Exemplarily, as shown in FIG. 5 , the processor 103 obtains that the speed of the pedestrian is v1, the speed of the vehicle 100 is v2, the vertical distance between the pedestrian and the vehicle 100 is L1, the width of the lane is L2, and the distance between the pedestrian and the vehicle 100 is L1. The boundary distance of the road is L3, and then it is judged whether the time t1=L3/v1 when the pedestrian reaches the road boundary where the vehicle 100 is located is greater than t2=L1/v2, if it is greater, and combined with other factors such as pedestrian movement, orientation, and deep intention to calculate When the lane intention of the lane where the vehicle 100 is currently located is less than the set threshold, it is safe for the vehicle 100 to drive on the current lane; if it is smaller than the vehicle 100 calculated in combination with other factors such as pedestrian movement, orientation, and deep intention When the lane intention of the currently located lane is greater than the set threshold, it is unsafe for the vehicle 100 to drive in the currently located lane. Then judge whether the time t3=(L3+L2)/v1 for the pedestrian to reach the boundary of the lane adjacent to the lane where the vehicle 100 is located is greater than t2 (for the convenience of calculation, ignore the time for the vehicle 100 to turn from the current lane to the adjacent lane), if it is greater than, When the lane intention of the lane where the vehicle 100 is currently located, calculated in combination with other factors such as pedestrian movement, orientation, and deep intention, is less than the set threshold, it is safe for the vehicle 100 to turn to the adjacent lane; When the lane intention of the current lane of the vehicle 100 calculated from other elements such as , orientation, and deep intention is greater than the set threshold, it is unsafe for the vehicle 100 to drive in the adjacent lane. At this time, the vehicle 100 can be controlled to slow down, or Brake to stop.

In the present application, the lane-level static interaction features and dynamic interaction features are extracted by using the road structure of each lane and related vehicles, combined with the behavioral features of the target pedestrians to infer the pedestrian's lane intention, so that the vehicle 100 can output the pedestrian's passing intention relative to each lane. Enabling autonomous vehicles to respond in advance.

FIG. 6 is a schematic structural diagram of an apparatus for estimating a pedestrian's passing intention according to an embodiment of the present application. As shown in FIG. 6 , the apparatus 600 includes an acquisition unit 601 and a processing unit 602 .

The obtaining unit 601 is used to obtain the status information of at least one pedestrian around the vehicle, and the at least one pedestrian includes the first pedestrian;

The processing unit 602 is configured to calculate the surface intention of the first pedestrian at the first moment according to the state information of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment, where the second moment is at Before the first moment, the deep intention is the probability of the first pedestrian crossing the road in the entire life cycle, and the surface intention is the probability that the first pedestrian is or will cross the road at the current moment;

The processing unit 602 is further configured to determine whether the first pedestrian crosses the road at the first moment according to the surface intention of the first pedestrian at the first moment.

In an embodiment, the processing unit 602 is specifically configured to calculate the likelihood probability of the pedestrian feature of the first pedestrian at the first moment according to the state information of the first pedestrian at the first moment; The likelihood probability of the pedestrian feature of the first pedestrian at the moment and the deep intention of the first pedestrian at the second moment are used to calculate the surface intention of the first pedestrian at the first moment.

In one embodiment, the processing unit 602 is further configured to control the speaker to play a warning signal and/or display warning information on the display screen when the surface intention of the first pedestrian at the first moment is greater than a set threshold.

In an embodiment, the processing unit 602 is further configured to calculate the predicted state of the first pedestrian at the first moment according to the state information of the first pedestrian at the second moment and the surface intention of the first pedestrian at the second moment information; according to the predicted state information of the first pedestrian at the first moment, the state information of the first pedestrian at the first moment, and the deep intention of the first pedestrian at the second moment, calculate the first pedestrian at the first moment. The deep intentions of the group.

In an embodiment, the processing unit 602 is further configured to calculate at least one road on which the vehicle is located according to the state information of the first pedestrian at the first moment and the surface intention of the first pedestrian at the first moment Lane intent of the lane, where the lane intent is the probability that the first pedestrian traverses the first lane, and the at least one lane includes the first lane.

The present invention provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed in a computer, the computer is made to execute any one of the above methods.

The present invention provides a computing device, including a memory and a processor, wherein executable codes are stored in the memory, and when the processor executes the executable codes, any one of the above methods is implemented.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Experts may use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of the embodiments of the present application.

Furthermore, various aspects or features of the embodiments of the present application may be implemented as a method, apparatus or article of manufacture using standard programming and/or engineering techniques. The term "article of manufacture" as used in this application encompasses a computer program accessible from any computer readable device, carrier or medium. For example, computer readable media may include, but are not limited to, magnetic storage devices (eg, hard disks, floppy disks, or magnetic tapes, etc.), optical disks (eg, compact discs (CDs), digital versatile discs (DVDs) etc.), smart cards and flash memory devices (eg, erasable programmable read-only memory (EPROM), card, stick or key drives, etc.). Additionally, various storage media described herein can represent one or more devices and/or other machine-readable media for storing information. The term "machine-readable medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and/or carrying instructions and/or data.

In the above embodiments, the pedestrian crossing intention estimation apparatus 600 in FIG. 6 may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented in software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer may be a general purpose computer, special purpose computer, computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server, or data center Transmission to another website site, computer, server, or data center is by wire (eg, coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes an integration of one or more available media. The usable media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVD), or semiconductor media (eg, Solid State Disk (SSD)), and the like.

It should be understood that, in various embodiments of the embodiments of the present application, the size of the sequence numbers of the above-mentioned processes does not mean the sequence of execution, and the execution sequence of each process should be determined by its functions and internal logic, and should not be The implementation process of the embodiments of the present application constitutes any limitation.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described systems, devices and units may refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

The functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application can be embodied in the form of software products in essence, or the parts that make contributions to the prior art or the parts of the technical solutions, and the computer software products are stored in a storage medium , including several instructions to cause a computer device (which may be a personal computer, a server, or an access network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the embodiments of this application. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

The above are only specific implementations of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Changes or substitutions that are thought of should be covered within the protection scope of the embodiments of the present application.

Claims

A method for estimating pedestrian crossing intention, comprising:

acquiring status information of at least one pedestrian around the vehicle, where the at least one pedestrian includes the first pedestrian;

According to the state information of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment, the surface intention of the first pedestrian at the first moment is calculated, and the second moment is before the first moment , the deep intention is the probability of the first pedestrian crossing the road in the entire life cycle, and the surface intention is the probability that the first pedestrian is or is about to cross the road at the current moment;

According to the surface intention of the first pedestrian at the first moment, it is determined whether the first pedestrian is crossing the road at the first moment.
The method according to claim 1, wherein, calculating the first pedestrian at the first moment according to the state information of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment Before the surface intent, include:

According to the state information of the first pedestrian at the first moment, the likelihood probability of the pedestrian feature of the first pedestrian at the first moment is calculated;

The calculation of the surface intention of the first pedestrian at the first moment according to the state information of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment includes:

According to the likelihood probability of the pedestrian feature of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment, the surface intention of the first pedestrian at the first moment is calculated.
The method according to any one of claims 1-2, wherein the method further comprises:

When the surface intention of the first pedestrian at the first moment is greater than the set threshold, the speaker is controlled to play an early warning signal and/or the warning information is displayed on the display screen.
The method according to any one of claims 1-3, characterized in that, in the calculation of the first moment according to the state information of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment Before the surface intent of the first pedestrian, the method includes:

According to the state information of the first pedestrian at the second time and the surface intention of the first pedestrian at the second time, calculate the state information of the predicted first pedestrian at the first time;

Calculate the first row at the first moment according to the state information of the predicted first pedestrian at the first moment, the state information of the first pedestrian at the first moment, and the deep intention of the first pedestrian at the second moment deep intentions of man.
The method according to any one of claims 1-4, wherein the method further comprises:

According to the state information of the first pedestrian at the first moment and the surface intention of the first pedestrian at the first moment, calculate the lane intention of at least one lane of the road where the vehicle is located, where the lane intention is the first lane intention The probability that a pedestrian traverses a first lane, the at least one lane including the first lane.
A device for estimating pedestrian's passing intention, comprising:

an acquisition unit, configured to acquire status information of at least one pedestrian around the vehicle, where the at least one pedestrian includes the first pedestrian;

The processing unit is configured to calculate the surface intention of the first pedestrian at the first moment according to the state information of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment, where the second moment is at Before the first moment, the deep intention is the probability of the first pedestrian crossing the road in the entire life cycle, and the surface intention is the probability that the first pedestrian is or will cross the road at the current moment;

The processing unit is further configured to determine whether the first pedestrian crosses the road at the first moment according to the surface intention of the first pedestrian at the first moment.
The device according to claim 6, wherein the processing unit is specifically configured to calculate the similarity of the pedestrian characteristics of the first pedestrian at the first moment according to the state information of the first pedestrian at the first moment probability;

According to the likelihood probability of the pedestrian feature of the first pedestrian at the first moment and the deep intention of the first pedestrian at the second moment, the surface intention of the first pedestrian at the first moment is calculated.
The device according to any one of claims 6-7, wherein the processing unit is further configured to control the speaker to play an early warning when the surface intention of the first pedestrian at the first moment is greater than a set threshold signal and/or show warning messages on the display.
The device according to any one of claims 6-8, wherein the processing unit is further configured to use the state information of the first pedestrian at the second moment and the surface intention of the first pedestrian at the second moment , calculate the state information of the predicted first pedestrian at the first moment;

Calculate the first row at the first moment according to the state information of the predicted first pedestrian at the first moment, the state information of the first pedestrian at the first moment, and the deep intention of the first pedestrian at the second moment deep intentions of man.
The device according to any one of claims 6-9, characterized in that the processing unit is further configured to: according to the state information of the first pedestrian at the first moment and the first row at the first moment The surface intention of the person is to calculate the lane intention of at least one lane of the road where the vehicle is located, where the lane intention is the probability of the first pedestrian crossing the first lane, and the at least one lane includes the first lane.
A device comprising at least one processor for executing instructions stored in a memory to cause a terminal to execute the method according to any one of claims 1-5.
A vehicle for carrying out the method of any one of claims 1-5.
A computer-readable storage medium on which a computer program is stored, when the computer program is executed in a computer, the computer is made to perform the method of any one of claims 1-5.
A computing device, comprising a memory and a processor, wherein executable code is stored in the memory, and when the processor executes the executable code, the processor of any one of claims 1-5 is implemented. method.