WO2024087712A1

WO2024087712A1 - Target behavior prediction method, intelligent device and vehicle

Info

Publication number: WO2024087712A1
Application number: PCT/CN2023/104261
Authority: WO
Inventors: 胡少伟; 李云龙
Original assignee: 华为技术有限公司
Priority date: 2022-10-27
Filing date: 2023-06-29
Publication date: 2024-05-02
Also published as: CN117944671A

Abstract

A target behavior prediction method, which is applied to an intelligent device. The method comprises: acquiring the motion state of a target object in a surrounding environment (S610); performing collision analysis according to the motion state of the target object, so as to determine a risk coefficient of the target object (S620), wherein the risk coefficient is used for indicating the possibility of a collision occurring between the target object and an intelligent device; determining the priority of the target object according to the risk coefficient (S630); according to preset correspondences between priority levels and prediction models, determining a target prediction model, which matches the priority (S640); and predicting the motion trajectory of the target object according to the target prediction model (S650). The method can divide objects in the surrounding environment of an intelligent device into different priorities according to degrees of collision risk, and use prediction models having different computational complexities to predict future motion trajectories for objects having different priorities, and therefore behavior prediction of the objects in the surrounding environment is implemented, and the consumption of computational resources is also reduced and the prediction delay is shortened. Further disclosed are an intelligent device, an on-board device, a vehicle, a robot, a chip system, a computer storage medium and a computer program product.

Description

A target behavior prediction method, intelligent device and vehicle

This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on October 27, 2022, with application number 202211329915.8 and application name “A Target Behavior Prediction Method, Intelligent Device and Vehicle”, all contents of which are incorporated by reference in this application.

Technical Field

The present application relates to the field of artificial intelligence, and in particular to a target behavior prediction method, intelligent device and vehicle.

Background technique

Artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

Autonomous driving is a mainstream application in the field of artificial intelligence. Autonomous driving technology relies on the collaborative efforts of computer vision, radar, monitoring devices, and global positioning systems to enable smart devices (such as autonomous vehicles, robots, or other autonomous driving devices) to achieve automatic driving without the need for active human operation.

At present, smart devices usually need to perceive the surrounding environmental information to predict the behavior of target objects (such as pedestrians, vehicles, etc.) in the surrounding environment, that is, to predict the future motion trajectory, so that smart devices can respond and perform corresponding operations in time, such as planning the future driving path of smart devices to avoid collisions, screening possible interactive objects of smart devices to improve interaction efficiency, etc. However, when there are a large number of target objects in the surrounding environment, due to the very limited processing power of smart devices, smart devices cannot predict the behavior of all target objects in time, resulting in a large prediction delay, which affects the response speed.

Summary of the invention

The present application provides a target behavior prediction method, an intelligent device, and a vehicle, which can predict the future motion trajectory of target objects with different collision risk levels in the surrounding environment of the intelligent device using prediction models of different computational complexity. While achieving the behavior prediction of objects in the surrounding environment, it also reduces the consumption of computing resources, reduces prediction latency, and improves response efficiency.

In order to achieve the above objectives, this application adopts the following technical solutions:

In a first aspect, the present application provides a target behavior prediction method that can be applied to smart devices, and the target behavior prediction method includes: performing a collision analysis based on the motion state of the target object to determine the risk coefficient of the target object, and the risk coefficient is used to indicate the possibility of a collision between the target object and the smart device; determining the priority of the target object based on the risk coefficient; determining a target prediction model that matches the priority based on the correspondence between the preset priority level and the prediction model; and predicting the motion trajectory of the target object based on the target prediction model.

According to the solution provided in the first aspect above, the intelligent device can perform collision analysis on different objects in the surrounding environment of the intelligent device according to the motion states of different objects in the surrounding environment to determine the collision risk level of each object. Then the intelligent device can divide the objects in the surrounding environment into different priorities according to the collision risk level, and for objects of different priorities, the intelligent device can use different prediction models to predict the future motion trajectory. For example, for high-priority objects, a refined prediction model with high computational complexity can be used to predict the future motion trajectory, while for low-priority objects, a simplified prediction model with low computational complexity can be used to predict the future motion trajectory. The hierarchical prediction of the behavior trajectories of different objects in the surrounding environment by the intelligent device is realized. In this way, even if there are a large number of target objects in the surrounding environment, since the behavior trajectories of some objects are predicted by simplified prediction models with low computational complexity, the consumption of computing resources of the intelligent device can be greatly reduced, which reduces the prediction delay and improves the response efficiency of the intelligent device.

In a possible implementation, the above-mentioned collision analysis based on the motion state of the target object and determining the risk coefficient of the target object may include: obtaining multiple driving positions on the driving route of the smart device; determining the risk coefficient corresponding to each of the multiple driving positions according to the motion state of the target object, wherein the risk coefficient corresponding to each driving position is used to indicate the possibility of collision between the target object and the smart device at each driving position; determining the risk coefficient of the target object according to the minimum value of the risk coefficient corresponding to each driving position. Risk factor of the target object. In this way, the smart device can select multiple driving positions from the vehicle's driving route as evaluation points for possible future collisions between the vehicle and the target objects in the surrounding environment. The smart device can then comprehensively consider the probability of collision at multiple evaluation points and accurately determine the collision risk level of the target object.

In a possible implementation, the above-mentioned determination of the risk coefficient corresponding to each of the multiple driving positions according to the motion state of the target object may include: determining the first distance between the target object and the target driving position after a specified time period according to the motion state of the target object, wherein the target driving position is any one of the multiple driving positions; determining the second distance between the smart device and the target driving position after a specified time period; obtaining the sum of the first distance and the second distance as the risk coefficient corresponding to the target driving position. In this way, for each evaluation point on the driving route of the self-vehicle, the smart device can accurately analyze the possibility of collision at the evaluation point according to the remaining collision distance of the target object and the self-vehicle from the evaluation point after driving for a period of time. It can be understood that the smaller the remaining collision distance is, the more likely it is that the target object and the self-vehicle will collide near the evaluation point after driving for a period of time.

In a possible implementation, the above-mentioned acquisition of multiple driving positions on the driving route of the smart device may include: acquiring the driving route of the smart device within a preset time period in the future; acquiring a driving position on the driving route at every specified interval to obtain multiple driving positions on the driving route. In this way, the smart device can obtain multiple evaluation position points where collisions may occur in the future by sampling the target driving route of the vehicle at a fixed distance.

In a possible implementation, the above-mentioned acquisition of multiple driving positions on the driving route of the smart device may include: acquiring the driving route of the smart device within a preset time period in the future; acquiring a driving position on the driving route at a specified interval to obtain multiple driving positions on the driving route. In this way, the smart device can obtain multiple evaluation position points that may cause collisions in the future by regularly sampling the target driving route of the vehicle.

In a possible implementation, the above-mentioned collision analysis based on the motion state of the target object and determining the risk factor of the target object may include: determining the collision time of the target object and the smart device according to the motion state of the target object; and determining the risk factor of the target object according to the collision time. In this way, the smart device can also accurately determine the collision risk degree of the target object according to the time when the vehicle may collide with the target object in the surrounding environment in the future.

In a possible implementation, there are multiple targets in the surrounding environment of the smart device. The above-mentioned determination of the priority of the target according to the risk coefficient may include: sorting the multiple targets according to the risk coefficient of each target among the multiple targets; and determining the priority of each target according to the sorted multiple targets. In this way, the smart device can prioritize the targets according to their importance or urgency based on the degree of collision risk of different objects in the surrounding environment. The targets that are more likely to collide are assigned a higher priority, and the targets that are less likely to collide are assigned a lower priority.

In a possible implementation, determining the priority of each target object according to the sorted multiple targets may include: dividing the sorted multiple targets into different priorities according to a preset number of priorities to obtain the priority of each target object. In this way, the smart device may not limit the number of priorities to be divided. When more or fewer priorities need to be divided, the smart device can adjust the parameter size of the number of priorities. Therefore, the expansion of priority levels can be achieved without changing the collision risk analysis method.

In a possible implementation, the target behavior prediction method may further include: displaying a first interface, the first interface being used to input a priority number; and in response to a user input operation, obtaining the priority number input by the user as a preset priority number. In this way, the user can customize the priority number as needed, thereby improving the user experience.

In a possible implementation, the target behavior prediction method may further include: determining the color corresponding to each target object according to the preset correspondence between the priority level and the color and the priority of each target object; and displaying multiple targets based on the color corresponding to each target object. In this way, the smart device can use different colors to display targets of different priorities on the display screen, so that the user can directly see the priority grading effect of the smart device on the targets in the surrounding environment.

In a possible implementation, the correspondence between the above-mentioned preset priority levels and prediction models may include: the first priority corresponds to the first prediction model, the second priority corresponds to the second prediction model, the first priority is higher than the second priority, and the computational complexity of the first prediction model is higher than the computational complexity of the second prediction model. In this way, the smart device can use prediction models with different computational complexities to predict the future motion trajectory of target objects of different priorities in the surrounding environment. While realizing the prediction of the behavior of objects in the surrounding environment, it also reduces the consumption of computing resources and reduces the prediction delay.

In a second aspect, the present application provides a smart device, including: an analysis unit, a rating unit, a matching unit, and a prediction unit. The analysis unit is used to perform collision analysis based on the motion state of the target object and determine the risk coefficient of the target object, and the risk coefficient is used to indicate the possibility of collision between the target object and the smart device; the rating unit is used to determine the priority of the target object based on the risk coefficient; the matching unit is used to determine the priority matching the target object based on the correspondence between the preset priority level and the prediction model. A target prediction model; a prediction unit, used to predict the motion trajectory of the target object according to the target prediction model.

In a possible implementation, the above analysis unit can be used to: obtain multiple driving positions on the driving route of the smart device; determine the risk coefficient corresponding to each of the multiple driving positions according to the motion state of the target object, and the risk coefficient corresponding to each driving position is used to indicate the possibility of collision between the target object and the smart device at each driving position; determine the risk coefficient of the target object according to the minimum value of the risk coefficient corresponding to each driving position. In this way, the smart device can select multiple driving positions from the driving route of the vehicle as evaluation points for possible future collisions between the vehicle and the target objects in the surrounding environment, so that the smart device can comprehensively consider the possibility of collision at multiple evaluation points and accurately determine the collision risk level of the target object.

In a possible implementation, the above-mentioned analysis unit can be used to: determine the first distance between the target object and the target driving position after a specified time period according to the motion state of the target object, wherein the target driving position is any one of a plurality of driving positions; determine the second distance between the smart device and the target driving position after a specified time period; obtain the sum of the first distance and the second distance as the risk coefficient corresponding to the target driving position. In this way, for each evaluation point on the driving route of the self-vehicle, the smart device can accurately analyze the possibility of collision at the evaluation point according to the remaining collision distance of the target object and the self-vehicle from the evaluation point after driving for a period of time. It can be understood that the smaller the remaining collision distance, the more likely it is that the target object and the self-vehicle will collide near the evaluation point after driving for a period of time.

In a possible implementation, the analysis unit can be used to: obtain the driving route of the smart device within a preset time period in the future; obtain a driving position on the driving route at each specified distance interval to obtain multiple driving positions on the driving route. In this way, the smart device can obtain multiple evaluation position points where collisions may occur in the future by sampling the target driving route of the vehicle at a fixed distance.

In a possible implementation, the analysis unit can be used to: obtain the driving route of the smart device within a preset time period in the future; obtain a driving position on the driving route at each specified time interval to obtain multiple driving positions on the driving route. In this way, the smart device can obtain multiple evaluation position points where collisions may occur in the future by regularly sampling the target driving route of the vehicle.

In a possible implementation, the above analysis unit can also be used to: determine the collision time of the target object and the smart device according to the motion state of the target object; and determine the risk factor of the target object according to the collision time. In this way, the smart device can also accurately determine the collision risk degree of the target object according to the time when the vehicle may collide with the target object in the surrounding environment in the future.

In a possible implementation, there are multiple targets in the surrounding environment of the smart device, and the rating unit can be used to: sort the multiple targets according to the risk factor of each target among the multiple targets; and determine the priority of each target according to the sorted multiple targets. In this way, the smart device can prioritize the targets according to their importance or urgency based on the degree of collision risk of different objects in the surrounding environment. The targets that are more likely to collide will be assigned a higher priority, and the targets that are less likely to collide will be assigned a lower priority.

In a possible implementation, the rating unit can be used to classify the sorted multiple targets into different priorities according to the preset number of priorities to obtain the priority of each target. In this way, the smart device does not need to limit the number of priorities to be divided. When more or fewer priorities need to be divided, the smart device can adjust the parameter size of the number of priorities. Therefore, the expansion of priority levels can be achieved without changing the collision risk analysis method.

In a possible implementation, the smart device may further include: a display unit and an acquisition unit. The display unit is used to display a first interface, and the first interface is used to input a priority number; the acquisition unit is used to respond to the user's input operation and obtain the priority number input by the user as the preset priority number. In this way, the user can customize the priority number according to the needs, thereby improving the user's experience.

In a possible implementation, the smart device may further include: a color picking unit and a display unit. The color picking unit is used to determine the color corresponding to each target object according to the correspondence between the preset priority level and the color and the priority of each target object; the display unit is used to display multiple targets based on the color corresponding to each target object. In this way, the smart device can use different colors to display targets of different priorities on the display screen, so that the user can directly see the priority grading effect of the smart device on the targets in the surrounding environment.

In a possible implementation, the correspondence between the preset priority levels and the prediction models may include: the first priority level corresponds to the first prediction model, the second priority level corresponds to the second prediction model, the first priority level is higher than the second priority level, and the computational complexity of the first prediction model is higher than the computational complexity of the second prediction model. In this way, the smart device can use prediction models with different computational complexities to predict the future motion trajectory of target objects with different priorities in the surrounding environment. While predicting the behavior of the subject, it also reduces the consumption of computing resources and reduces the prediction delay.

In a third aspect, the present application provides an intelligent device, comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the intelligent device executes the target behavior prediction method in any possible implementation of the first aspect.

In a fourth aspect, the present application provides a vehicle-mounted device, comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the vehicle-mounted device executes the target behavior prediction method in any possible implementation of the first aspect.

In a fifth aspect, the present application provides a vehicle, the vehicle comprising the vehicle-mounted device of the fourth aspect of the present application. The vehicle can be used to implement the target behavior prediction method in any possible implementation of the first aspect.

In a sixth aspect, the present application provides a robot, comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the robot executes the target behavior prediction method in any possible implementation of the first aspect.

In a seventh aspect, the present application provides a target behavior prediction device, which is included in an intelligent device, a robot, a vehicle, or an on-board device, and has the function of realizing the behavior of the intelligent device in any of the methods in the first aspect and the possible implementation of the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions.

In an eighth aspect, the present application provides a chip system, which is applied to a smart device. The chip system includes one or more interface circuits and one or more processors. The interface circuit and the processor are interconnected by a line. The interface circuit is used to receive a signal from a memory of the smart device and send the signal to the processor, where the signal includes a computer instruction stored in the memory. When the processor executes the computer instruction, the smart device executes the target behavior prediction method in any possible implementation of the first aspect above.

In a ninth aspect, the present application provides a computer storage medium comprising computer instructions, which, when executed on a smart device, enables the smart device to execute a target behavior prediction method in any possible implementation of the first aspect.

In a tenth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the target behavior prediction method in any possible implementation of the first aspect.

It can be understood that the beneficial effects that can be achieved by the intelligent device of the second aspect and any possible implementation thereof, the intelligent device of the third aspect, the vehicle-mounted device of the fourth aspect, the vehicle of the fifth aspect, the robot of the sixth aspect, the target behavior prediction device of the seventh aspect, the chip system of the eighth aspect, the computer storage medium of the ninth aspect, and the computer program product of the tenth aspect can be referred to the beneficial effects of the first aspect and any possible implementation thereof, and will not be repeated here.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a structural schematic diagram 1 of a vehicle provided in an embodiment of the present application;

FIG2 is a second structural schematic diagram of a vehicle provided in an embodiment of the present application;

FIG3 is a schematic diagram of the structure of a computer system provided in an embodiment of the present application;

FIG4 is a schematic diagram 1 of an application of a cloud-side command for an autonomous driving vehicle provided in an embodiment of the present application;

FIG5 is a second schematic diagram of an application of a cloud-side commanded autonomous driving vehicle provided in an embodiment of the present application;

FIG6 is a method flow chart of a target behavior prediction method provided in an embodiment of the present application;

FIG7 is a schematic diagram of a collision analysis provided in an embodiment of the present application;

FIG8 is a first schematic diagram of an interface of a vehicle-mounted display screen provided in an embodiment of the present application;

FIG. 9 is a second schematic diagram of an interface of a vehicle-mounted display screen provided in an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. It should be understood that in the present application, "at least one" means one or more, and "more than one" means two or more. "And/or" is used to refer to a plurality of items. For describing the association relationship of associated objects, it indicates that there may be three types of relationships. For example, "A and/or B" may indicate that only A exists, only B exists, and both A and B exist. A and B may be singular or plural.

The scheme of the embodiment of the present application can be applied to smart devices. Among them, the smart device can be an autonomous driving vehicle, a robot, or other electronic device with autonomous driving capability, and the embodiment of the present application does not limit this. In some embodiments, the scheme of the embodiment of the present application can also be applied to other devices (such as cloud servers, mobile phone terminals, etc.) with the function of controlling the above-mentioned smart devices. Smart devices or other devices can implement the target behavior prediction method provided by the embodiment of the present application through the components (including hardware and software) contained therein, and perform collision risk detection on the target objects in the surrounding environment of the smart device according to the data collected by the sensor, so as to determine the priority of the collision risk degree of the target object, so that the smart device or other device can use a refined and high-computational complexity prediction model to predict the future motion trajectory of high-priority targets, and use a simplified and low-computational complexity prediction model to predict the future motion trajectory of low-priority targets.

It can be understood that the target behavior prediction method provided in the embodiment of the present application does not use a refined, high-complexity prediction model to predict the future motion trajectory of all targets in the surrounding environment of the smart device, but divides the targets in the surrounding environment of the smart device into different priorities according to the degree of collision risk. Therefore, the future motion trajectory of low-priority targets can be predicted using a simplified, low-complexity prediction model, which greatly reduces the consumption of computing resources and reduces the prediction delay.

The following will take the smart device as an autonomous driving vehicle (hereinafter referred to as the vehicle) as an example to schematically illustrate the solution provided in the embodiment of the present application.

Please refer to FIG. 1, which shows a functional block diagram of a vehicle 100 provided in an embodiment of the present application. The vehicle 100 may include various devices, components, etc. disposed in the vehicle 100 and/or the body of the vehicle 100. In one embodiment, the devices and components disposed in the vehicle 100 may include, but are not limited to, an automatic driving system and an automatic driving function application. It is understood that an automatic driving system is usually disposed in a vehicle with a certain automatic driving capability.

The vehicle 100 may include various subsystems, such as a travel system 102, a sensor system 104, a control system 106, one or more peripheral devices 108, and a power source 110, a computer system 112, a hierarchical prediction system 114, and a user interface 116. Optionally, the vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple elements. In addition, each subsystem and element of the vehicle 100 may be interconnected by wire or wirelessly.

The travel system 102 may include components for providing powered motion for the vehicle 100. In one embodiment, the travel system 102 may include an engine, an energy source, a transmission, and wheels/tires. The engine may be an internal combustion engine, an electric motor, an air compression engine, or other types of engine combinations, such as a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air compression engine.

The engine can convert the energy source into mechanical energy. Examples of energy sources include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. The energy source can also provide energy for other systems of the vehicle 100.

The transmission can transmit mechanical power from the engine to the wheels. The transmission can include a gearbox, a differential, and a drive shaft. In one embodiment, the transmission can also include other devices, such as a clutch. Among them, the drive shaft can include one or more shafts that can be coupled to one or more wheels.

The sensor system 104 (also called "collection device") may include several sensors for sensing information about the surrounding environment of the vehicle 100. For example, the sensor system 104 may include a positioning system (the positioning system may be a global positioning system (GPS) system, or a Beidou system or other positioning system), an inertial measurement unit (IMU), a radar, a laser rangefinder, and a camera. The sensor system 104 may also include sensors of the internal systems of the monitored vehicle 100 (for example, an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (such as position, shape, direction, speed, etc.). Such detection and recognition are key functions for the safe operation of the autonomous driving of the vehicle 100.

Among other things, the positioning system may be used to estimate the geographic location of the vehicle 100. The IMU may be used to sense the position and orientation changes of the vehicle 100 based on inertial acceleration. In one embodiment, the IMU may be a combination of an accelerometer and a gyroscope.

The radar can use radio signals to sense objects in the surrounding environment of the vehicle 100, such as pedestrians, cyclists (i.e., bicyclists), motorcycles, other vehicles, and other types of obstacles. In some embodiments, in addition to sensing objects, the radar can also be used to sense one or more states of the object's speed, position, and direction of travel.

A laser rangefinder may utilize laser light to sense objects in the environment in which the vehicle 100 is located. In some embodiments, a laser rangefinder may include one or more laser sources, a laser scanner, and one or more detectors, among other system components.

The camera may be used to capture multiple images of the surroundings of the vehicle 100. The camera may be a still camera or a video camera.

The control system 106 is for controlling the operation of the vehicle 100 and its components. The control system 106 may include various components, such as a steering system, a throttle, a brake unit, a computer vision system, a route control system, and an obstacle avoidance system. Among them, the steering system can be operated to adjust the forward direction of the vehicle 100. Alternatively, the steering system can be a steering wheel system. The throttle can be used to control the operating speed of the engine and thus control the speed of the vehicle 100. The brake unit can be used to control the deceleration of the vehicle 100.

The computer vision system can process and analyze the images captured by the camera to identify various types of objects and/or features in the environment surrounding the vehicle 100. The computer vision system can use object recognition algorithms, structure from motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system can be used to map the environment, track objects, estimate the speed of objects, etc.

The route control system is used to determine the driving route of the vehicle 100. In some embodiments, the route control system can combine data from sensors, GPS, and one or more predetermined maps to determine the driving route for the vehicle 100.

The obstacle avoidance system is used to identify, evaluate, and avoid or otherwise negotiate obstacles in the environment of the vehicle 100 .

Of course, in one example, the control system 106 may include additional or alternative components other than the above components, or may reduce some of the above components.

The vehicle 100 interacts with external sensors, other vehicles, other computer systems, or users through the peripheral device 108. The peripheral device 108 may include a wireless communication system, an onboard computer, a microphone, and/or a speaker. In some embodiments, the peripheral device 108 provides a means for the user of the vehicle 100 to interact with the user interface 116. For example, the onboard computer may provide information to the user of the vehicle 100. The user interface 116 may also operate the onboard computer to receive the user's input. The onboard computer may be operated via a touch screen. In other cases, the peripheral device 108 may provide a means for the vehicle 100 to communicate with other devices located in the vehicle. For example, a microphone may receive audio (e.g., voice commands or other audio input) input by a user of the vehicle 100. Similarly, a speaker may output audio to the user of the vehicle 100.

Some or all functions of the vehicle 100 are controlled by a computer system 112. The computer system 112 may include at least one processor that executes instructions stored in a non-transitory computer-readable medium such as a data storage device. The computer system 112 may also be a plurality of computing devices that control individual components or subsystems of the vehicle 100 in a distributed manner.

The processor may include one or more processing units, for example, the processor may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a memory, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and/or a neural-network processing unit (neural-network processing unit, NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

In some embodiments, the memory may contain instructions (e.g., program logic) that can be executed by the processor to perform various functions of the vehicle 100, including those described above. The memory may also contain additional instructions, including instructions for one or more of the travel system 102, the sensor system 104, the control system 106, the peripheral device 108, and the hierarchical prediction system 114 to send data, receive data from it, interact with it, and/or control it. In addition to instructions, the memory may also store data, such as road maps, route information, vehicle data such as the vehicle's location, direction, speed, and other information, such as the location, orientation, speed, etc. of various objects in the vehicle's surroundings. This information can be used by the vehicle 100, the computer system 112, and the hierarchical prediction system 114 during operation of the vehicle 100 in autonomous, semi-autonomous, and/or manual modes.

Exemplarily, the memory can obtain information about objects in the surrounding environment obtained by the vehicle 100 based on the sensors in the sensor system 104, such as the location of obstacles such as other vehicles and pedestrians, the distance between the obstacles and the vehicle 100, and other information. The memory can also obtain environmental information from the sensor system 104 or other components of the vehicle 100. The environmental information can be, for example, whether there are green belts, lanes, pedestrians, etc. near the current environment of the vehicle, or whether there are green belts, lanes, pedestrians, etc. near the current environment calculated by the vehicle through a machine learning algorithm. In addition to the above content, the memory can also store the status information of the vehicle itself, as well as the status information of the target objects (pedestrians, other vehicles, etc.) that interact with the vehicle, wherein the status information of the vehicle includes but is not limited to the location, speed, acceleration, heading angle, etc. of the vehicle.

In some embodiments, the processor may also execute the target behavior prediction method of the embodiment of the present application to reduce computing resources and reduce prediction delay. For example, the processor may obtain the above information from the memory and predict the target behavior based on the environmental information of the vehicle environment. The priority level of the target object is determined based on the information of the vehicle's own state information, the state information of the target object, etc., so as to determine the prediction model for predicting the motion trajectory of the target object based on the priority level, thereby controlling the vehicle 100 to use a refined prediction model with high computational complexity to predict the future motion trajectory of high-priority targets, and to use a simplified prediction model with low computational complexity to predict the future motion trajectory of low-priority targets. The specific target behavior prediction method can be referred to the following introduction.

The computer system 112 may control functions of the vehicle 100 based on input received from various subsystems (e.g., the travel system 102, the sensor system 104, the control system 106, and the hierarchical prediction system 114) and from the user interface 116. For example, the computer system 112 may utilize input from the control system 106 in order to control the steering unit to avoid obstacles detected by the sensor system 104 and the obstacle avoidance system. In some embodiments, the computer system 112 may provide control over many aspects of the vehicle 100 and its subsystems.

The hierarchical prediction system 114 can determine the degree of collision risk of pedestrians, cyclists, motorcycles, other vehicles and other targets in the surrounding environment of the vehicle 100 based on the vehicle data input from various subsystems (for example, the travel system 102, the sensor system 104, and the control system 106) and the motion state of the targets, and classify them into different priorities, so as to use different prediction models to predict the future motion trajectories of targets of different priorities, thereby realizing the hierarchical prediction function of the vehicle 100.

It is understood that the structure shown in FIG. 1 is for illustration only and does not limit the structure of the vehicle in the embodiment of the present application. In other embodiments of the present application, the vehicle 100 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange components differently, or have different configurations with the same functions as shown in FIG. 1 or more functions than shown in FIG. 1. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

An autonomous vehicle traveling on a road, such as vehicle 100 above, can determine an adjustment instruction for the current speed based on objects in its surrounding environment. The objects in the surrounding environment of vehicle 100 can be traffic control devices, other types of static objects such as green belts, or various types of dynamic objects such as pedestrians, cyclists, motorcycles, other vehicles, etc. In some examples, vehicle 100 can consider each object in the surrounding environment independently and determine the speed adjustment instruction of vehicle 100 based on the respective characteristics of the object, such as its current speed, acceleration, distance from the vehicle, etc.

Optionally, the vehicle 100 as an autonomous vehicle or a computer device associated therewith (such as the computer system 112, a computer vision system, and a memory) can evaluate the risk factor of a collision between the identified object and the vehicle 100 based on the characteristics of the identified object and the driving route of the vehicle 100 in the future, and then divide the identified objects into different priorities based on the risk factor obtained from the evaluation, so that different prediction models can be used to predict the future motion trajectories of objects of different priorities, thereby achieving the purpose of reducing computing resources and reducing prediction delays.

Optionally, the vehicle 100 can adjust its driving strategy based on the predicted future motion trajectory of the object. In other words, the autonomous vehicle can determine what stable state the vehicle needs to adjust to (e.g., accelerate, decelerate, turn, or stop, etc.) based on the predicted future motion trajectory of the object. In this process, other factors can also be considered to determine the speed adjustment instructions of the vehicle 100, such as the lateral position of the vehicle 100 in the road, the curvature of the road, the proximity of static and dynamic objects, etc.

In addition to providing instructions to adjust the speed of the autonomous vehicle, the computer device may also provide instructions to modify the steering angle of vehicle 100 so that the autonomous vehicle follows a given trajectory and/or maintains a safe lateral and longitudinal distance between the autonomous vehicle and nearby objects (e.g., cars in adjacent lanes).

It is understood that the vehicle 100 may be a car, a truck, a motorcycle, a bus, a ship, an airplane, a helicopter, a lawn mower, an entertainment vehicle, an amusement park vehicle, construction equipment, a tram, a golf cart, a train, and a cart, etc., and the present application embodiment does not make any special restrictions. For example, the vehicle 100 may also be a smart car or a smart robot with autonomous driving capability in the field of smart home.

In other embodiments of the present application, the autonomous driving vehicle may further include a hardware structure and/or a software module to implement the above functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. Whether one of the above functions is implemented in the form of a hardware structure, a software module, or a hardware structure plus a software module depends on the specific application and design constraints of the technical solution.

Referring to FIG. 2 , illustratively, the vehicle 200 may include the following modules:

Environmental perception module 201: used to obtain information about other vehicles and pedestrians in the surrounding environment of vehicle 200 through vehicle-mounted sensors and/or roadside sensors. Optionally, vehicle 200 may also be vehicle 100 in FIG. 1. The sensor may be a laser radar, a millimeter wave radar, a visual sensor, etc. The perception module 201 obtains the video stream data originally collected by the sensor, the point cloud data of the radar, etc., and then processes these original video stream data and radar point cloud data to obtain identifiable structured data such as the position, size, speed, and direction of travel of people and other vehicles. Among them, due to the variety of sensors, the perception module 201 can determine the position, speed, direction of travel, and other information of other vehicles and pedestrians in the surrounding environment of the vehicle 200 based on the data collected by all or a certain type or a certain sensor. The perception module 201 is also used to send the position, speed, direction of travel, and other information of other vehicles and pedestrians in the surrounding environment determined based on the data obtained by the sensor to the classification module 203.

The route acquisition module 202 is used to obtain the driving route of the vehicle 200 for a period of time in the future according to the planned route of the vehicle 200. Optionally, the planned route of the vehicle 200 can be obtained through a road navigation (route) module. When the vehicle 200 needs to go to a certain place, the route module can guide the vehicle 200 to travel on what kind of road according to the existing map or road network information, the location information of the starting point and the location information of the destination, so as to successfully complete the travel from the starting point to the destination. That is, the route module can navigate a planned route from the starting point to the destination. Optionally, the route module can obtain the planned route of the vehicle 200 through a navigation request. The navigation request may include the location information of the starting point of the vehicle 200 and the location information of the destination. For example, the vehicle 200 can obtain the navigation request by the user clicking or touching the screen of the vehicle navigation, or by the user's voice command. Then the route module can use the vehicle GPS in conjunction with the electronic map to plan the route, and then obtain the planned route of the vehicle 200, and obtain the driving route of the vehicle 200 for a period of time in the future according to the planned route of the vehicle 200.

It should be noted that the solution provided in the present application can obtain the planned route of vehicle 200 in a variety of ways, and the methods for obtaining vehicle navigation information in the relevant technology can be adopted in the embodiments of the present application. For example, when there is no map, the planned route of vehicle 200 can also be obtained through the equivalent route of the lane where vehicle 200 is located provided by the road structure recognition module. Among them, the road structure recognition module is used to obtain road information through on-board sensors and/or roadside sensors, such as road boundary information, lane information where vehicle 200 is located, lane boundary information, etc., to determine the road structure of the lane where vehicle 200 is located based on the road information, so that vehicle 200 can generate the driving route of vehicle 200 based on the road structure.

The risk classification module 203 is used to obtain the position, speed, and forward direction of other vehicles and pedestrians in the surrounding environment of the vehicle 200 from the environment perception module 201, and to obtain the driving route of the vehicle 200 for a period of time in the future from the route acquisition module 202. Then, based on the obtained driving route of the vehicle 200 for a period of time in the future and the position, speed, and forward direction of other vehicles and pedestrians, collision analysis is performed on other vehicles and pedestrians to obtain the possibility of collision between other vehicles and pedestrians and the vehicle 200 in the future, that is, the risk coefficient, so as to classify other vehicles and pedestrians into different priorities according to the risk coefficient. The risk classification module 203 is also used to determine the priority level to be divided into level 1, level 2, level 3, ... level n according to the preset parameter n (n is a positive integer). Therefore, the number of priority levels to be divided can be changed by adjusting the numerical value of the preset parameter n. For example, the priority level can be expanded by increasing the numerical value of the preset parameter n. The risk classification module 203 is also used to send the priority of other vehicles and pedestrians in the surrounding environment that it finally obtains to the trajectory prediction module 204.

The trajectory prediction module 203 is used to receive the priorities of other vehicles and pedestrians in the surrounding environment of the vehicle 200 sent by the risk classification module 203, and use a matching prediction model to predict the future motion trajectory according to the priorities of other vehicles and pedestrians received. For example, for other vehicles 1 and pedestrian 1 with high priority in the surrounding environment, a refined prediction model with high computational complexity is used to predict the future motion trajectory, and for other vehicles 2 and pedestrian 2 with low priority in the surrounding environment, a simplified prediction model with low computational complexity is used to predict the future motion trajectory. Optionally, when the priority levels to be divided are level 1, level 2, level 3, ... level n, the trajectory prediction module 203 can also determine the one-to-one corresponding prediction model type as prediction model 1, prediction model 2, prediction model 3, ... prediction model n according to the priority level. Therefore, different prediction models can be used to predict other vehicles and pedestrians of different priorities, so as to reduce computing resources and reduce prediction delay.

In some embodiments, vehicle 200 may also include a display module (not shown in Figure 2) for receiving the priorities of other vehicles and pedestrians in the surrounding environment of vehicle 200 sent by the risk grading module 203, and using different colors to display other vehicles and pedestrians of different priorities, so as to more conveniently and intuitively view the grading effects of other vehicles and pedestrians in the surrounding environment of vehicle 200.

In some embodiments, the vehicle 200 may further include a storage component (not shown in FIG. 2 ) for storing executable codes of the above-mentioned modules. Running these executable codes may implement part or all of the method flow of the embodiments of the present application.

In a possible implementation, as shown in FIG3 , the computer system 112 shown in FIG1 may include a processor 301, the processor 301 is coupled to a system bus 302, the processor 301 may be one or more processors, each of which may include one or more processor cores. A display adapter (video adapter) 303 may drive a display 324, and the display 324 The system bus 302 is coupled to the input/output (I/O) bus 305 through the bus bridge 304. The I/O interface 306 is coupled to the I/O bus 305. The I/O interface 306 communicates with various I/O devices, such as an input device 307 (such as a keyboard, a mouse, a touch screen, etc.), a multimedia disk (media tray) 308 (such as a multimedia interface). The transceiver 309 (which can send and/or receive radio communication signals), the camera 310 (which can capture static and dynamic digital video images) and the external universal serial bus (USB) port 311. Optionally, the interface connected to the I/O interface 306 can be a USB interface.

The processor 301 may be any conventional processor, including a reduced instruction set computer (RISC) processor, a complex instruction set computer (CISC) processor, or a combination thereof. Alternatively, the processor 301 may also be a dedicated device such as an application specific integrated circuit (ASIC). Alternatively, the processor 301 may also be a neural network processor or a combination of a neural network processor and the conventional processors.

Alternatively, in various embodiments described herein, the computer system 112 may be located remotely from the autonomous vehicle and in wireless communication with the autonomous vehicle. In other aspects, some of the processes described herein may be executed on a processor within the autonomous vehicle, and other processes may be executed by a remote processor, including taking actions required to perform a single maneuver.

Computer system 112 can communicate with software deployment server 313 through network interface 312. Optionally, network interface 312 can be a hardware network interface, such as a network card. Network 314 can be an external network, such as the Internet, or an internal network, such as Ethernet or a virtual private network (VPN). Optionally, network 314 can also be a wireless network, such as a wireless fidelity (Wi-Fi) network, a cellular network, etc.

The hard disk drive interface 315 is coupled to the system bus 302. The hard disk drive interface 315 is connected to a hard disk drive 316. The system memory 317 is coupled to the system bus 302. The data running in the system memory 317 may include an operating system (OS) 318 and an application program 319 of the computer system 112.

The operating system (OS) 318 includes, but is not limited to, a shell 320 and a kernel 321. The shell 320 is an interface between the user and the kernel 321 of the operating system 318. The shell 320 is the outermost layer of the operating system 318. The shell manages the interaction between the user and the operating system 318: waiting for user input, interpreting user input to the operating system 318, and processing various output results of the operating system 318.

The kernel 321 is composed of the part of the operating system 318 used to manage memory, files, peripherals and system resources, and directly interacts with the hardware. The kernel 321 of the operating system 318 usually runs processes and provides communication between processes, and provides functions such as CPU time slice management, interrupts, memory management, and IO management.

Application 319 includes programs 323 related to autonomous driving, such as programs for managing the interaction between the autonomous driving car and obstacles on the road, programs for controlling the driving route or speed of the autonomous driving car, and programs for controlling the interaction between the autonomous driving car and other cars/autonomous driving cars on the road. Application 319 also exists on the system of deploying server 313. In one embodiment, when the application 319 needs to be executed, the computer system 112 can download the application 319 from the deploying server 313.

In the embodiment of the present application, the application 319 may include an application for controlling the vehicle to predict the future motion trajectory of the target object in the surrounding environment according to the hierarchical prediction system 114. The processor 301 of the computer system 112 calls the application 319 to perform the following steps: perform collision analysis according to the motion state of the target object, determine the risk coefficient of the target object, and the risk coefficient is used to indicate the possibility of collision between the target object and the vehicle; determine the priority of the target object according to the risk coefficient; determine the target prediction model that matches the priority according to the correspondence between the preset priority level and the prediction model; and predict the motion trajectory of the target object according to the target prediction model.

Sensor 322 is associated with computer system 112. Sensor 322 is used to detect the environment around computer system 112. For example, sensor 322 can detect surrounding animals, cars, people and other objects. Further, sensor 322 can also detect the environment around the above-mentioned animals, cars, people and other objects. For example: the environment around the car, for example, the lane where the car is located, etc. Optionally, if computer system 112 is located in an autonomous driving car, sensor 322 can be at least one of a camera, an infrared sensor, a chemical detector, a microphone and other devices.

In other embodiments of the present application, the computer system 112 may also receive information from other computer systems or transfer information to other computer systems. Alternatively, sensor data collected from the sensor system 120 of the vehicle 100 may be transferred to another computer, which processes the data. As shown in FIG. 4 , data from the computer system 112 may be transmitted to a cloud-side computer system 410 via a network for further processing. The network and intermediate nodes may include various configurations and coordination The present invention relates to a computer system that is capable of communicating with other computers via a variety of protocols, including the Internet, the World Wide Web, an intranet, a virtual private network, a wide area network, a local area network, a private network using one or more company's proprietary communication protocols, Ethernet, WiFi, and HTTP, and various combinations of the foregoing. Such communications may be performed by any device capable of transmitting data to and from other computers, such as modems and wireless interfaces.

In one example, computer system 112 may include a server having multiple computers, such as a load balancing server cluster. Server 420 exchanges information with different nodes of a network in order to receive, process, and transmit data from computer system 112. Computer system 410 may have a configuration similar to computer system 112, and have processor 430, memory 440, instructions 450, and data 460.

In one example, the data 460 of the server 420 may include providing weather-related information. For example, the server 420 may receive, monitor, store, update, and transmit various information related to targets in the surrounding environment. The information may include target categories, target shape information, and target tracking information, such as in the form of reports, radar information, forecasts, etc.

5, which is an example of interaction between an autonomous driving vehicle and a cloud service center (cloud server). The cloud service center can receive information (such as data or other information collected by vehicle sensors) from vehicles 513 and 512 in its operating environment 500 via a network 511 such as a wireless communication network. Vehicles 513 and 512 can be autonomous driving vehicles.

The cloud service center 520 runs the stored program related to controlling the automatic driving of the vehicle according to the received data to control the vehicles 513 and 512. The program related to controlling the automatic driving of the vehicle may be: a program for managing the interaction between the automatic driving vehicle and obstacles on the road, or a program for controlling the route or speed of the automatic driving vehicle, or a program for controlling the interaction between the automatic driving vehicle and other automatic driving vehicles on the road.

Exemplarily, cloud service center 520 may provide portions of a map to vehicle 513, vehicle 512 via network 511. In other examples, operations may be divided between different locations. For example, multiple cloud service centers may receive, verify, combine, and/or send information reports. In some examples, information reports and/or sensor data may also be sent between vehicles. Other configurations are also possible.

In some examples, the cloud service center 520 sends the autonomous vehicle a suggested solution for possible driving situations in the environment (e.g., informing the obstacle ahead and informing how to bypass it). For example, the cloud service center 520 can assist the vehicle in determining how to proceed when facing a specific obstacle in the environment. The cloud service center 520 sends a response to the autonomous vehicle indicating how the vehicle should proceed in a given scenario. For example, based on the collected sensor data, the cloud service center 520 can confirm the presence of a temporary parking sign ahead of the road, or, for example, based on the "lane closed" sign and the sensor data of the construction vehicle, determine that the lane is closed due to construction. Accordingly, the cloud service center 520 sends a suggested operating mode for the vehicle to pass through the obstacle (e.g., instructing the vehicle to change lanes to another road). When the cloud service center 520 observes the video stream in its operating environment 500 and has confirmed that the autonomous vehicle can safely and successfully pass through the obstacle, the operating steps used by the autonomous vehicle can be added to the driving information map. Accordingly, this information can be sent to other vehicles in the area that may encounter the same obstacle, so as to assist other vehicles not only to identify the closed lane but also to know how to pass through it.

The methods in the following embodiments can all be implemented in a vehicle having the above hardware structure or other devices having the function of controlling a vehicle, such as an autonomous driving vehicle, or a processor in a vehicle or other devices having the function of controlling a vehicle, such as the processor 301 and the processor 430 in the computer system 112 mentioned above.

At present, vehicles usually need to predict the future motion trajectory of target objects such as pedestrians, cyclists, motorcycles or other vehicles in the surrounding environment, so that the vehicle can respond in time and perform corresponding operations, such as planning the vehicle's current driving path to avoid collisions. However, in order to ensure the accuracy of trajectory prediction, the computational complexity of the prediction model used to predict the trajectory is usually relatively high. When there are a large number of target objects around that need to predict the trajectory, for vehicles with limited computing resources, not only will a large amount of computing resources be consumed, resulting in the overall performance of the vehicle being affected, but also the inability to predict the behavior of all target objects in a timely manner, resulting in a large prediction delay, which affects the response speed of the vehicle.

In order to solve the above problems, this application provides a solution that can obtain the importance score of the ground truth by simulating the real driving data of the vehicle, and train the neural network to use the trained neural network to predict the importance score of the target objects in the surrounding environment, so that the behavior of the target objects in the surrounding environment can be predicted according to the importance score ranking. However, this solution is difficult to obtain data, and the neural network itself consumes a lot of computing resources.

The present application also provides a solution that can divide the space around the vehicle into different areas according to certain rules, and combine the road topology to divide the objects in the surrounding environment into three levels: caution, normal, and ignore. However, this division method depends on the road topology, and the rules are complex and difficult to maintain. Different levels depend on different rules. Otherwise, there is no unified comparison index, and it is impossible to divide into more levels, and the level scalability is poor.

Based on this, the embodiments of the present application provide a target behavior prediction method and an intelligent device. The intelligent device can determine the priority of the target object based on the collision risk detection of the target object in the surrounding environment, and then for the high-priority target object, a refined prediction model with high computational complexity can be used to predict the future motion trajectory, while for the low-priority target object, a simplified prediction model with low computational complexity can be used to predict the future motion trajectory. In this way, not all targets in the surrounding environment use a refined prediction model with high computational complexity to predict the future motion trajectory, but the targets in the surrounding environment are divided into different priorities according to the degree of collision risk, so that the future motion trajectory of the low-priority target object can be predicted using a simplified prediction model with low computational complexity, which greatly reduces the consumption of computing resources while also reducing the prediction delay.

It can be understood that, since the solution provided in the embodiment of the present application prioritizes the objects in the surrounding environment by calculating the collision risk coefficient, the risk coefficients of different objects can be directly compared, with a unified comparison index. Moreover, the solution provided in the embodiment of the present application does not rely on maps, does not require data drive, can be used in scenarios with or without maps, and has low computational complexity.

The following will take the intelligent device as an autonomous driving vehicle (hereinafter referred to as vehicle) as an example, and introduce a target behavior prediction method provided by an embodiment of the present application in conjunction with the accompanying drawings. As shown in Figure 6, the target behavior prediction method may include:

S610: The vehicle obtains the motion status of the target object in the surrounding environment.

The target object may be an obstacle such as a pedestrian, a cyclist, a motorcycle or other vehicles in the surrounding environment of the vehicle. Optionally, the target object may be a movable object in the surrounding environment of the vehicle, or a stationary object in the surrounding environment of the vehicle, such as a roadblock, a roadside trash can, etc. In the embodiment of the present application, for the sake of distinction, the vehicle to which the target behavior prediction method is applied may be referred to as the self-vehicle, and other vehicles in the surrounding environment of the self-vehicle may be referred to as other vehicles.

In the embodiment of the present application, the motion state of the target object may include the position of the target object, the speed of the target object, the forward direction of the target object, that is, the speed direction, etc. Optionally, the motion state of the target object may also include the distance between the target object and the vehicle.

Optionally, the motion state of the target object can be detected by the environment perception module in Figure 2. As an implementation, the environment perception module can include a radar, a laser rangefinder or a camera, etc. As an implementation, the environment perception module can include the sensor 322 in Figure 3.

In some embodiments, step S610 may also include obtaining a target driving route of the vehicle.

The target driving route may refer to the driving route of the vehicle within a preset time period in the future starting from the current moment. The preset time period may be reasonably set in advance according to the application scenario, and the embodiment of the present application does not limit the length of the preset time period. For example, the preset time period may be 10 seconds (S).

Optionally, the vehicle may determine the target driving path of the vehicle based on parameters such as the planned route of the vehicle, the position of the vehicle, and the speed of the vehicle. For example, the vehicle may determine the approximate driving distance of the vehicle on the planned route within a preset time period in the future based on the position of the vehicle and the speed of the vehicle, thereby determining the target driving path of the vehicle from the planned route of the vehicle.

Optionally, the target driving route of the vehicle may be obtained by the route acquisition module 202 in Fig. 2. As an implementation, the route acquisition module 202 may include a route control system in the control system 106 in Fig. 1 .

Optionally, the route acquisition module 202 can determine the target driving path of the vehicle based on parameters such as the planned route of the vehicle, the position of the vehicle, and the speed of the vehicle. Exemplarily, the speed of the vehicle can be detected by a speed sensor. The position of the vehicle can be determined by a positioning system, for example, it can be determined by a positioning system in the vehicle 100 of FIG. 1. The planned route of the vehicle can be determined by a road navigation (route) module.

In some embodiments, when there is no planned route for the vehicle, the vehicle may also determine the target driving route of the vehicle based on the road structure of the lane where the vehicle is located. The road structure of the lane includes the lane driving direction, lane boundary information, etc. Optionally, the road structure of the lane where the vehicle is located may be determined by a road structure recognition module.

S620: The vehicle performs a collision analysis based on the motion state of the target object to determine the risk factor of the target object.

The risk factor is used to indicate the probability of a collision between the target object and the vehicle. In the embodiment of the present application, the vehicle can perform a collision analysis based on the motion state of the target object to assess the risk of a collision between the target object and the vehicle in the future.

In the embodiment of the present application, step S620 may also include sampling the target driving route of the vehicle to obtain multiple sampling positions on the target driving route. Optionally, the vehicle may use the sampling positions as the positions where the vehicle and the target object may collide in the future to perform collision analysis. In this way, by using the future driving route of the vehicle to assess the risk of collision, the vehicle can consider The future interaction relationship between the ego vehicle and the target object can be determined to avoid the problem of low prediction trajectory accuracy caused by subsequent misdivision of priorities.

Among them, sampling the target driving route of the vehicle can be understood as selecting multiple driving position points on the target driving route of the vehicle as multiple sampling positions. Optionally, the vehicle can select a position point as a sampling position at every specified distance or every specified time on the target driving route of the vehicle, so that the vehicle can obtain multiple sampling positions on the target driving route. Optionally, the vehicle can also randomly select multiple position points on the target driving route of the vehicle as multiple sampling positions.

Optionally, the number of sampling positions may be a fixed number, that is, when the vehicle performs a fixed number of samplings on the target driving route of the vehicle, a fixed number of sampling positions may be obtained. For example, when the number of sampling positions is fixed to 8, the vehicle may randomly select 8 driving position points on the target driving route of the vehicle as 8 sampling positions.

Optionally, the number of sampling positions can also be randomly generated, for example, a number is randomly selected from 5 to 10 as the number of vehicles to be sampled.

As an implementation mode, for each sampling position on the target driving route, the vehicle can determine the remaining collision distance between the target object and the vehicle relative to the sampling position after a specified period of time according to the motion state of the target object, and use the remaining collision distance as the collision risk coefficient of the sampling position. The vehicle can determine the risk coefficient of the target object according to the collision risk coefficient of each sampling position on the target driving route.

The specified time period can be reasonably set in advance according to the application scenario, and the embodiment of the present application does not limit the length of the specified time period. For example, the specified time period can be 3 seconds (S).

Optionally, the vehicle may determine a first distance between the target object and a target sampling position after a specified time period according to the motion state of the target object, wherein the target sampling position is any sampling position among multiple sampling positions. At the same time, the vehicle may also determine a second distance between the vehicle and the target sampling position after a specified time period. The vehicle may use the sum of the first distance and the second distance as the remaining collision distance between the target object and the vehicle relative to the target sampling position after a specified time period, and the remaining collision distance may be used as the collision risk coefficient of the target sampling position.

For example, referring to FIG. 7 , assuming that the driving speed of the vehicle 701 is v _e , the driving speed of the target object 702 is v _o , and the multiple black dots on the target driving route 703 of the vehicle are sampling positions, taking the i-th sampling position p _i on the target driving route 703 as an example, the vehicle can first calculate the distance _do that the target object 702 moves toward the sampling position p _i within the specified time period t _p :
v′ _o =max(v _o *cosθ,0);

d _o =v′ _o *t _p .

Wherein, θ is the angle between the velocity direction of the target object 702 and the line connecting the target object 702 to the sampling position _pi . The max() function is used to output the maximum value, and v′ _o is the velocity of the target object 702 in the direction of the line connecting the target object 702 to the sampling position _pi .

Next, the vehicle can calculate the remaining distance d ₁ between the target object 702 and the sampling position p _i after the specified time period t _p :

d ₁ =max(d _oi -d _o ,0).

Wherein, d _oi is the initial distance between the target object 702 and the sampling position _pi at the beginning of this collision analysis.

The vehicle can then calculate the distance d _e that the vehicle 701 moves toward the sampling position _pi within the specified time period t _p :

d _e = _ve *t _p .

Similarly, the vehicle can calculate the remaining distance d ₂ between the vehicle 701 and the sampling position p _i after the specified time period t _p :

d ₂ =max( _dei - _de ,0).

Wherein, d _ei is the initial distance between the ego vehicle 701 and the sampling position _pi at the beginning of this collision analysis.

The vehicle can determine the remaining collision distance _dpi between the target object and the vehicle relative to the sampling position p _i after the specified time period t _p based on the remaining distance _d1 between the target object 702 and the sampling position p _i and the remaining distance _d2 between the vehicle ₇₀₁ and the sampling position p _i :
d _pi =d ₁ +d ₂ .

The remaining collision distance _dpi can be used as the collision risk coefficient corresponding to the sampling position _p1 . It can be understood that when the value of the remaining collision distance _dpi is smaller, that is, the collision risk coefficient corresponding to the sampling position _p1 is smaller, the vehicle can consider that the target object and the vehicle are more likely to collide near the sampling position _p1 after the specified time period _tp .

In the embodiment of the present application, when the vehicle samples the target driving route of the vehicle and obtains n sampling positions on the target driving route, the vehicle can use the above method to calculate the collision risk coefficient corresponding to each of the n sampling positions, namely _dp0 , _dp1 , _dp2 , ..., _dpn .

Optionally, the vehicle can use the smallest collision risk coefficient among the n sampling positions as the risk coefficient of the target object. Risk factor of the object

Since the smaller the collision risk coefficient is, the more likely the target object and the ego vehicle are to collide after the specified time period _tp , it can be understood that the smaller the risk coefficient of the target object is, the more likely the target object and the ego vehicle are to collide after the specified time period _tp .

Optionally, the vehicle may also compare the minimum collision risk factor and the safety factor among the n sampling positions, and use the ratio of the minimum collision risk factor to the safety factor as the risk factor of the target object. It is understood that the safety factor may be determined in advance based on historical data analysis and combined with the actual situation of the vehicle.

In some embodiments, the vehicle may also calculate the collision time when the target object and the vehicle may collide based on the motion state of the target object and the object kinematic model, so as to determine the risk factor of the target object based on the collision time. As an implementation method, the vehicle may compare the collision time with a preset time, so as to use the ratio of the collision time to the preset time as the risk factor of the target object. It is understood that the preset time may be determined in advance based on historical data analysis and in combination with the actual situation of the vehicle.

In some embodiments, the vehicle can predict whether the driving trajectories of the vehicle and the target object intersect based on the motion state of the target. If they intersect, the time from the current position of the vehicle and the target object to the intersection of the two trajectories is calculated respectively, and the absolute value of the time difference between the two is calculated to determine the risk coefficient of the target object based on the absolute value. As an implementation method, the vehicle can compare the absolute value of the time difference with the preset time, and use the ratio of the absolute value of the time difference to the preset time as the risk coefficient of the target object. It can be understood that the preset time can be determined in advance based on historical data analysis and combined with the actual situation of the vehicle.

S630: The vehicle determines the priority of the target object according to the risk factor.

In the embodiment of the present application, after determining the risk factor of each target object in the surrounding environment, the vehicle can prioritize each target object in the surrounding environment according to the risk factor, so as to distinguish targets of different priorities in the surrounding environment.

Optionally, the vehicle can determine the priority level to be divided according to a preset parameter n, and the priority level includes level 1, level 2, level 3, ... level n. Among them, parameter n can be understood as the number of priorities (i.e., the number of levels) to be divided for the target objects in the surrounding environment of the vehicle. When the number of levels of the target object classification needs to be changed, the vehicle can adjust the size of parameter n. For example, increasing the value of the preset parameter n can achieve the expansion of the priority level. In this way, the expansion of the priority level can be achieved without changing the calculation method of the risk coefficient.

Optionally, the preset parameter n may be pre-stored in the vehicle or acquired in real time. As an implementation mode, the vehicle may display a first interface through an onboard display screen, and the first interface is used to provide a user with a quick input of the priority number.

Optionally, the first interface may provide multiple options for the number of priorities, such as 3 priorities, 5 priorities, etc. The user may select an option for confirmation. The vehicle may respond to the user's confirmation operation and obtain the number of priorities confirmed by the user as a preset parameter n.

Optionally, the first interface may also provide an input box for the number of priorities. The user may enter a specific value in the input box, and then the vehicle may respond to the user's input operation and obtain the number of priorities entered by the user as a preset parameter n.

In an embodiment of the present application, the vehicle may classify targets in the surrounding environment into different priorities according to the risk factor of each target in the surrounding environment and a preset parameter n.

As a method, the objects in the surrounding environment can be sorted in order from small to large according to the risk coefficient of each object in the surrounding environment. Among them, the higher the ranking of the object, the smaller the corresponding risk coefficient, that is, the more likely the object and the vehicle will collide after a specified time period. At this time, the sorted objects can be divided into priority levels according to the preset parameter n, from high level (level n) to low level (level 1). Among them, the higher the level of the object, the higher the priority, so that the objects that are more likely to collide are classified into higher priorities.

Optionally, a higher level may also mean a lower priority, that is, level 1 is the highest priority and level n is the lowest priority. At this time, the sorted targets can be divided into priority levels according to the preset parameter n, in the order from low level (level 1) to high level (level n). Among them, the target closer to the front is divided into a lower level, that is, a higher priority, so that the target that is more likely to collide is divided into a higher priority.

As another way, the objects in the surrounding environment can also be sorted in descending order according to the risk coefficient of each object in the surrounding environment. Among them, the later the object is ranked, the smaller its corresponding risk coefficient is, that is, the more likely it is for the object to collide with the vehicle after a specified time period. At this time, the sorted objects can be divided into priority levels according to the preset parameter n, in order from high level (level 1) to low level (level n). Among them, the later the object is, the higher the level is, that is, the higher the priority is, so that the objects that are more likely to collide are divided into higher levels. priority.

Optionally, the vehicle can also determine the risk coefficient range corresponding to different levels according to the preset parameter n, and the vehicle can match the risk coefficient of each target object in the surrounding environment with the risk coefficient range corresponding to different levels. When the risk coefficient of a target object falls within the risk coefficient range corresponding to a certain level, the target object can be classified into the level. In this way, the priority division of the targets in the surrounding environment is achieved.

In some embodiments, the vehicle may also use different colors to display objects of different priorities on the vehicle display screen, thereby intuitively displaying the priority grading effect of objects in the vehicle's surrounding environment.

Optionally, the vehicle can determine a target color that matches the priority of the target object according to a preset correspondence between colors and priority levels, and display the target object in the target color. The correspondence between colors and priority levels can be reasonably set in advance according to actual application conditions. The correspondence between colors and priority levels can be pre-stored in the vehicle or obtained from other devices such as a cloud server.

Optionally, the vehicle may also randomly generate corresponding colors according to the number of priority levels to be divided, wherein the randomly generated colors correspond to the priorities one by one to distinguish objects of different priorities.

As an exemplary scenario, please refer to Figure 8, which shows an intersection scene where vehicle 800 turns left. The boxes in the scene represent different types of targets, such as motor vehicles, motorcycles, pedestrians, etc. The dotted arrow on each target represents the speed direction of the target, and the solid arrow represents the direction of the target. Optionally, the vehicle can also display information about the target, such as displaying the target's identity, speed, and other information below each target. The target's identity can be the type of target, such as non-motor vehicles, motorcycles, pedestrians, cars, etc.; it can also be a unique identifier (identity document, ID) of the target.

As shown in Figure 8, a large number of targets appear in the scene and all of them need to predict their motion trajectories. Among them, target 801 on the zebra crossing in front of the left side of vehicle 800 in Figure 8 is pedestrian A crossing the road; target 802 behind the right side of vehicle 800 is cyclist A; target 803 at the corner of the intersection in front of vehicle 800 is pedestrian B; target 804 close to vehicle 800 after exiting the intersection is vehicle A (i.e., another vehicle); target 805 far from the current position of vehicle 800 after exiting the intersection is vehicle B; target 806 behind vehicle 800 and moving away from vehicle 800 in speed direction is vehicle C; target 807 in front of the right side of vehicle 800 and moving away from vehicle 800 is cyclist B.

Optionally, after collision analysis of the targets around vehicle 800 is performed by the target behavior prediction method of the embodiment of the present application, the targets around vehicle 800 can be divided into three priority levels. Among them, target 801 (i.e. pedestrian A crossing the road) and target 802 (i.e. cyclist A) have the highest priority, target 803 (i.e. pedestrian B) and target 804 (i.e. vehicle A) have the second highest priority, and target 805 (i.e. vehicle B), target 806 (i.e. vehicle C), and target 807 (i.e. cyclist B) have the lowest priority.

Optionally, the vehicle can also use three colors to display the above three priority targets on the vehicle display screen. For example, assuming that the highest priority corresponds to red, the second priority corresponds to yellow, and the lowest priority corresponds to blue, the vehicle can display target 801 (i.e. pedestrian A crossing the road) and target 802 (i.e. cyclist A) in red, target 803 (i.e. pedestrian B) and target 804 (i.e. vehicle A) in yellow, and target 805 (i.e. vehicle B), target 806 (i.e. vehicle C), and target 807 (i.e. cyclist B) in blue on the vehicle display screen.

As another exemplary scenario, please refer to FIG9 , which shows a scenario where a vehicle 900 is traveling straight. The boxes in the scenario represent different types of targets, such as motor vehicles, motorcycles, pedestrians, etc. The dotted arrow on each target represents the speed direction of the target, and the solid arrow represents the direction of the target.

As shown in Figure 9, a large number of targets also appear in the scene, all of which need to predict their movement trajectories. Among them, target 901 in front of the right side of vehicle 900 and with the same speed and direction as vehicle 900 in Figure 9 is cyclist C; target 902 behind vehicle 900 and traveling in the same direction as vehicle 900 is vehicle D; target 903 in front of the right side of vehicle 900 and outside the motor vehicle lane is cyclist D; target 904 in front of the left side of vehicle 900 and outside the motor vehicle lane is cyclist E; target 905 in the left rear side of vehicle 900 and outside the motor vehicle lane is pedestrian C; target 906 in the left rear side of vehicle 900 and traveling in the opposite direction of vehicle 900 is cyclist F.

Optionally, after collision analysis of the targets around vehicle 900 is performed by the target behavior prediction method of the embodiment of the present application, the targets around vehicle 900 can be divided into three priority levels. Among them, the priority of target 901 (i.e., cyclist C) and target 902 (i.e., vehicle D) is level 1, the priority of target 903 (i.e., cyclist D) and target 904 (i.e., cyclist E) is level 2, and the priority of target 905 (i.e., pedestrian C) and target 906 (i.e., cyclist F) is level 3. Similarly, the vehicle can also use three colors to display the targets of the above three priorities on the on-board display screen.

S640: The vehicle determines a target prediction model that matches the priority of the target object based on the correspondence between the preset priority levels and the prediction models.

In the embodiment of the present application, after the vehicle has divided the priority level of each target object in the surrounding environment, different prediction models can be used to predict the future movement trajectories of targets of different priorities.

Optionally, the vehicle may store a preset correspondence between priority levels and prediction models. When the vehicle divides the priority level of a target object in the surrounding environment, it may determine a target prediction model that matches the priority level of the target object based on the preset correspondence.

Optionally, the correspondence between the preset priority levels and the prediction models may be a one-to-one correspondence, that is, each priority level corresponds to a prediction model. Different prediction models have different computational complexities. As an implementation, the correspondence between the preset priority levels and the prediction models may include: a first priority level corresponds to a first prediction model, and a second priority level corresponds to a second prediction model. The first priority level is higher than the second priority level, and the computational complexity of the first prediction model is higher than the computational complexity of the second prediction model.

Optionally, the computational complexity of the prediction model can be determined based on the processing time of the prediction model, the number of operation instructions and the number of memory interaction instructions, the performance consumed, etc. The embodiments of the present application are not limited to this. As an implementation method, the prediction model provided in the present application can be a rule-based prediction model such as a constant velocity (CV) model and a Markov model. Such prediction models are usually simpler and have lower computational complexity. As another implementation method, the prediction model provided in the present application can also be a prediction model based on a neural network. Such prediction models usually have higher computational complexity.

Optionally, the correspondence between the preset priority levels and the prediction models can also be a many-to-one relationship, that is, there can be multiple priority levels corresponding to one prediction model. For example, assuming that the number of priority levels n is 6, level 6 is the lowest priority, and level 1 is the highest priority. Among them, prediction model 1 can correspond to priority levels 1, 2, and 3, prediction model 2 can correspond to priority levels 4 and 5, and prediction model 3 can only correspond to the highest priority level 6.

In the embodiment of the present application, the higher the priority, the higher the computational complexity of the corresponding prediction model, and the lower the priority, the lower the computational complexity of the corresponding prediction model. Thus, when the target objects that are more likely to collide are classified as higher priorities, a more precise prediction model with higher computational complexity can be used to quickly and accurately predict the future motion trajectory of the target object. When the target objects that are less likely to collide are classified as lower priorities, a simpler prediction model with lower computational complexity can be used to quickly and accurately predict the future motion trajectory of the target object. In this way, by rationally using prediction models of different complexities, it is possible to reduce the consumption of computing resources, reduce prediction delays, and also achieve accurate prediction of the motion trajectories of a large number of targets.

S650: The vehicle predicts the movement trajectory of the target object according to the target prediction model.

In the embodiment of the present application, after the vehicle determines the target prediction model that matches the priority of the target object, it can sample the prediction model to predict the future motion trajectory of the target object. In this way, it is possible to predict the future motion trajectory of target objects of different priorities in the surrounding environment of the smart device using prediction models of different computational complexities. While realizing the prediction of the behavior of objects in the surrounding environment, it also reduces the consumption of computing resources and reduces the prediction delay.

Optionally, after predicting the target's motion trajectory, the vehicle can perform corresponding operations. For example, the vehicle can plan its own driving path at the current moment or in a specified time period in the future to avoid collision. For another example, the vehicle can play a horn to alert the target.

In summary, the target behavior prediction method of the embodiment of the present application solves the computing power bottleneck problem by predicting the motion trajectories of a large number of targets in a hierarchical manner, that is, for high-priority targets, a refined prediction model with high computational complexity can be used for trajectory prediction, and for low-priority targets, a prediction model with low computational complexity can be used for trajectory prediction. This achieves the purpose of reducing computing resources and reducing prediction latency, and also indirectly improves the prediction accuracy of a large number of targets.

It can be understood that the target behavior prediction method of the embodiment of the present application can also be applied to robots or other electronic devices with autonomous driving capabilities, and the embodiment of the present application is not limited to this.

For example, when the target behavior prediction method of the embodiment of the present application is applied to a sweeping robot in the field of smart home, the sweeping robot can perform collision analysis on multiple characters in the surrounding environment to obtain the risk coefficient of each character, and determine the priority of each character according to the risk coefficient. When the future motion trajectory of the character is predicted using the corresponding prediction model according to the priority of each character, the sweeping robot can plan its own driving path at the current moment or in a specified time period in the future to avoid collision, or the sweeping robot can screen out characters that may need to interact based on the future motion trajectory of each character to improve interaction efficiency.

It can be understood that the target behavior prediction method of the embodiment of the present application can be applied to various scenarios. It is necessary to perceive surrounding targets and judge the risk of collision. For example, it can be applied to scenarios where it is necessary to judge the strength of the kinematic interaction between surrounding objects and itself.

It is understandable that in order to achieve the above functions, the vehicle includes hardware and/or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of this application.

In this embodiment, the smart watch can be divided into functional modules according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

An embodiment of the present application also provides a vehicle-mounted device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the vehicle-mounted device implements each function or step performed by the vehicle in each of the above-mentioned method embodiments.

The present application also provides a target behavior prediction device, which can be applied to the above-mentioned vehicle-mounted device. The device is used to execute each function or step executed by the vehicle in the above-mentioned method embodiment.

An embodiment of the present application also provides a vehicle, which includes the above-mentioned vehicle-mounted equipment or target positioning device.

The embodiment of the present application further provides an intelligent device, which includes the above-mentioned target positioning device. The intelligent device may be a robot or other electronic device with autonomous driving capability.

The embodiment of the present application also provides a chip system, which includes at least one processor and at least one interface circuit. The processor and the interface circuit can be interconnected through a line. The interface circuit can read the instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the vehicle-mounted device can perform the various functions or steps performed by the vehicle in the above method embodiment. Of course, the chip system can also include other discrete devices, which are not specifically limited in the embodiment of the present application.

An embodiment of the present application also provides a computer storage medium, which includes computer instructions. When the computer instructions are executed on the above-mentioned vehicle-mounted device, the vehicle-mounted device executes each function or step executed by the vehicle computer in the above-mentioned method embodiment.

The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute each function or step executed by the vehicle computer in the above method embodiment.

Among them, the vehicle-mounted equipment, vehicle, robot, computer storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

A target behavior prediction method, characterized in that it is applied to a smart device, and the method comprises:

Performing a collision analysis according to the motion state of the target object to determine a risk factor of the target object, wherein the risk factor is used to indicate the possibility of a collision between the target object and the smart device;

Determining the priority of the target object according to the risk factor;

According to the correspondence between the preset priority level and the prediction model, determining the target prediction model matching the priority;

The motion trajectory of the target object is predicted according to the target prediction model.
The method according to claim 1 is characterized in that the step of performing collision analysis according to the motion state of the target object to determine the risk factor of the target object comprises:

Acquire multiple driving positions on the driving route of the smart device;

Determine, according to the motion state of the target object, a risk coefficient corresponding to each of the multiple driving positions, wherein the risk coefficient corresponding to each driving position is used to indicate the possibility of a collision between the target object and the smart device at each driving position;

The risk coefficient of the target object is determined according to the minimum value of the risk coefficients corresponding to each driving position.
The method according to claim 2 is characterized in that determining the risk coefficient corresponding to each of the multiple driving positions according to the motion state of the target object comprises:

Determining, according to the motion state of the target object, a first distance between the target object and a target driving position after a specified time period, wherein the target driving position is any one of the multiple driving positions;

determining a second distance between the smart device and the target driving location after the specified time period;

The sum of the first distance and the second distance is obtained as a risk coefficient corresponding to the target driving position.
The method according to claim 2, characterized in that the step of obtaining a plurality of driving positions on the driving route of the smart device comprises:

Obtaining a driving route of the smart device within a preset time period in the future;

A driving position on the driving route is acquired at every designated distance, so as to obtain a plurality of driving positions on the driving route.
The method according to claim 2, characterized in that the step of obtaining a plurality of driving positions on the driving route of the smart device comprises:

Obtaining a driving route of the smart device within a preset time period in the future;

At each designated time interval, a driving position on the driving route is acquired to obtain a plurality of driving positions on the driving route.
The method according to claim 1 is characterized in that the step of performing collision analysis according to the motion state of the target object to determine the risk factor of the target object comprises:

Determining a collision time when the target object collides with the smart device according to a motion state of the target object;

A risk factor of the target object is determined according to the collision time.
The method according to any one of claims 1 to 6, characterized in that there are multiple target objects in the surrounding environment of the smart device, and determining the priority of the target objects according to the risk coefficient comprises:

sorting the multiple targets according to the risk factor of each target in the multiple targets;

The priority of each target object is determined according to the sorted multiple targets.
The method according to claim 7, characterized in that the step of determining the priority of each target object according to the sorted multiple targets comprises:

According to the preset number of priorities, the sorted multiple targets are divided into different priorities to obtain the priority of each target.
The method according to claim 8, characterized in that the method further comprises:

Displaying a first interface, wherein the first interface is used to input a priority number;

In response to the user's input operation, the priority number input by the user is obtained as the preset priority number.
The method according to claim 7, characterized in that the method further comprises:

According to the correspondence between the preset priority level and the color, and the priority of each target, the priority of each target is determined. The color corresponding to the marker;

The multiple objects are displayed based on the color corresponding to each object.
The method according to any one of claims 1-10 is characterized in that the correspondence between the preset priority levels and the prediction models includes: the first priority corresponds to the first prediction model, the second priority corresponds to the second prediction model, the first priority is higher than the second priority, and the computational complexity of the first prediction model is higher than the computational complexity of the second prediction model.
A smart device, characterized in that the smart device includes a memory and one or more processors; the memory and the processor are coupled; the memory is used to store computer program code, the computer program code includes computer instructions, and when the processor executes the computer instructions, the smart device executes the method described in any one of claims 1-11.
A vehicle-mounted device, characterized in that the vehicle-mounted device includes a memory and one or more processors; the memory and the processor are coupled; the memory is used to store computer program code, the computer program code includes computer instructions, and when the processor executes the computer instructions, the vehicle-mounted device executes the method described in any one of claims 1-11.
A vehicle, characterized in that the vehicle comprises an on-board device, and the on-board device executes the method as described in any one of claims 1-11.
A robot, characterized in that the robot includes a memory and one or more processors; the memory and the processor are coupled; the memory is used to store computer program code, the computer program code includes computer instructions, and when the processor executes the computer instructions, the robot executes the method as described in any one of claims 1-11.
A chip system, characterized in that the chip system is applied to an intelligent device; the chip system comprises one or more interface circuits and one or more processors; the interface circuit and the processor are interconnected via a line; the interface circuit is used to receive a signal from a memory of the intelligent device and send the signal to the processor, the signal comprising a computer instruction stored in the memory; when the processor executes the computer instruction, the intelligent device executes the method described in any one of claims 1 to 11.
A computer storage medium, characterized in that it includes computer instructions, and when the computer instructions are executed on an intelligent device, the intelligent device executes the method as described in any one of claims 1 to 11.
A computer program product, characterized in that when the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 11.