[go: up one dir, main page]
More Web Proxy on the site http://driver.im/

WO2024087456A1 - 确定朝向信息以及自动驾驶车辆 - Google Patents

确定朝向信息以及自动驾驶车辆 Download PDF

Info

Publication number
WO2024087456A1
WO2024087456A1 PCT/CN2023/080380 CN2023080380W WO2024087456A1 WO 2024087456 A1 WO2024087456 A1 WO 2024087456A1 CN 2023080380 W CN2023080380 W CN 2023080380W WO 2024087456 A1 WO2024087456 A1 WO 2024087456A1
Authority
WO
WIPO (PCT)
Prior art keywords
orientation
target obstacle
orientation information
autonomous driving
driving vehicle
Prior art date
Application number
PCT/CN2023/080380
Other languages
English (en)
French (fr)
Inventor
史磊
Original Assignee
北京三快在线科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京三快在线科技有限公司 filed Critical 北京三快在线科技有限公司
Publication of WO2024087456A1 publication Critical patent/WO2024087456A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects

Definitions

  • the present application relates to the field of autonomous driving technology, and in particular to a method for determining orientation information and an autonomous driving vehicle.
  • the orientation information of obstacles has an important impact on the decision-making and path planning of autonomous driving vehicles. Therefore, how to determine the orientation information of obstacles around autonomous driving vehicles has become a problem that needs to be solved urgently.
  • the point cloud data of obstacles can be obtained through the radar of the autonomous driving vehicle, and the direction information of the obstacle can be determined based on the point cloud data.
  • the embodiment of the present application provides a method for determining direction information and an autonomous driving vehicle, which can more accurately determine the direction of an obstacle.
  • the technical solution is as follows:
  • a method for determining orientation information comprising:
  • a candidate orientation set of the target obstacle Determine, based on the geometric features of the target obstacle, a candidate orientation set of the target obstacle, wherein the candidate orientation set includes a plurality of candidate orientation information, and the plurality of candidate orientation information indicates orientations of a plurality of faces of the target obstacle;
  • Second orientation information whose indicated orientation matches the orientation indicated by the first orientation information is determined from the candidate orientation set as the orientation information of the target obstacle.
  • a device for determining orientation information comprising:
  • An acquisition module is used to acquire point cloud data and image data taken by the autonomous driving vehicle for the target obstacle;
  • a feature extraction module configured to perform a first feature extraction on the point cloud data of the target obstacle to obtain a geometric feature of the target obstacle, and perform a second feature extraction on the image data of the target obstacle to obtain an apparent feature of the target obstacle;
  • a first determination module is used to determine a candidate orientation set of the target obstacle based on the geometric features of the target obstacle, wherein the candidate orientation set includes a plurality of candidate orientation information, and the plurality of candidate orientation information indicates orientations of a plurality of faces of the target obstacle;
  • a second determination module configured to determine first orientation information of the target obstacle based on the apparent characteristics of the target obstacle
  • a third determination module is configured to determine, from the candidate orientation set, second orientation information whose indicated orientation matches the orientation indicated by the first orientation information as the orientation information of the target obstacle.
  • the orientation information includes an orientation angle
  • the first determination module is used to determine the orientation angle of the target obstacle based on the geometric features
  • the orientation angle is increased by 90 degrees, 180 degrees and 270 degrees respectively to obtain multiple expanded orientation angles
  • the orientation angle and the multiple expanded orientation angles are determined as candidate orientation information to obtain the candidate orientation set.
  • the first determination module is configured to determine orientation information of multiple faces of the target obstacle based on the geometric features; and determine the orientation information of the multiple faces as a candidate orientation set of the target obstacle.
  • the image data is multi-frame image data
  • the multi-frame image data is image data obtained by the autonomous driving vehicle photographing the target obstacle at different times
  • the feature extraction module is used to perform a third feature extraction on each frame of image data in the multiple frames of image data of the target obstacle to obtain a first appearance feature corresponding to each frame of image data; and perform the following operations on each frame of image data in sequence according to the shooting order of the multiple frames of image data: fusing the first appearance feature of the current frame of image data with the first appearance feature of the previous frame of image data to obtain a second appearance feature of the current frame of image data; and determining the first orientation information of the target obstacle based on the second appearance feature of the last frame of image data.
  • the first orientation information is a first orientation category; the apparent features of the target obstacle and the first orientation category are determined by an orientation classification model, and the orientation classification model is used to determine an orientation category that matches the apparent features of the obstacle from a target number of orientation categories.
  • the target number of orientation categories includes front, right front, right, right rear, rear, left rear, left and left front; or, the target number of orientation categories includes 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees and 315 degrees.
  • the multiple candidate orientation information indicate the orientation of the target obstacle in the autonomous driving vehicle coordinate system
  • the first orientation information indicates the orientation of the target obstacle in the observer coordinate system
  • the observer coordinate system has a first ray pointing from the autonomous driving vehicle to the target obstacle as the vertical axis and a second ray passing through the center of the target obstacle and perpendicular to the first ray as the horizontal axis.
  • the third determination module is used to determine the angle between the autonomous driving vehicle coordinate system and the observer coordinate system; based on the angle, determine fourth orientation information corresponding to the first orientation information in the autonomous driving vehicle coordinate system, the fourth orientation information indicating the orientation of the target obstacle in the autonomous driving vehicle coordinate system; and determine second orientation information from the candidate orientation set whose indicated orientation matches the orientation indicated by the fourth orientation information as the orientation information of the target obstacle.
  • the third determination module is configured to determine, from the candidate orientation set, second orientation information whose indicated orientation has the smallest difference from the orientation indicated by the fourth orientation information as the orientation information of the target obstacle.
  • an autonomous driving vehicle which includes one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to implement the operations performed by the orientation information determination method in any possible implementation method as described above.
  • a computer-readable storage medium in which at least one program code is stored, and the at least one program code is loaded and executed by a processor to implement the operations performed by the orientation information determination method in any possible implementation manner as described above.
  • a computer program or a computer program product comprises: a computer program code, wherein when the computer program code is executed by a computer, the computer implements the operations performed by the method for determining the orientation information in any possible implementation manner as described above.
  • the method for determining orientation information and the autonomous driving vehicle provided in the embodiments of the present application can be based on the geometry of the target obstacle in the point cloud data.
  • the coarse-grained candidate orientation set is determined by the geometric information in the point cloud data and the appearance information in the image data.
  • One orientation information is selected from the candidate orientation set as the orientation information of the target obstacle according to the appearance information of the target obstacle in the image data. This is equivalent to jointly determining the orientation information of the target obstacle by integrating the geometric information in the point cloud data and the appearance information in the image data, which can more accurately determine the orientation of the target obstacle.
  • FIG1 is a schematic diagram of an implementation environment provided by an embodiment of the present application.
  • FIG2 is a flow chart of a method for determining orientation information provided by an embodiment of the present application.
  • FIG3 is a flow chart of a method for determining orientation information provided by an embodiment of the present application.
  • FIG4 is a schematic diagram of the structure of a direction classification model provided in an embodiment of the present application.
  • FIG5 is a schematic diagram of image data provided by an embodiment of the present application.
  • FIG6 is a schematic diagram of a coordinate system provided in an embodiment of the present application.
  • FIG7 is a schematic diagram of the structure of a device for determining orientation information provided in an embodiment of the present application.
  • FIG8 is a schematic diagram of the structure of an autonomous driving vehicle provided in an embodiment of the present application.
  • FIG. 9 is a schematic diagram of the structure of a server provided in an embodiment of the present application.
  • first, second, etc. used in this application can be used in this article to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, a first order can be called a second order, and a second order can be called a first order without departing from the scope of this application.
  • At least one includes one, two or more than two, multiple includes two or more than two, and each refers to each of the corresponding multiple, and any refers to any one of the multiple.
  • multiple corner points include 3 corner points, and each refers to each of the 3 corner points, and any refers to any one of the 3 corner points, which can be the first, the second, or the third.
  • the information including but not limited to user device information, user personal information, etc.
  • data including but not limited to data used for analysis, stored data, displayed data, etc.
  • signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.
  • the image data, point cloud data, etc. involved in this application are all obtained with full authorization.
  • the above information and data are processed and used in big data application scenarios, and cannot be identified to any natural person or have a specific association with him.
  • the method for determining the orientation information provided in the embodiments of the present application is performed by an autonomous vehicle.
  • the autonomous vehicle is any device with an automatic driving function.
  • the autonomous vehicle includes a vehicle traveling on the ground (e.g., a car, a truck, a bus, etc.), and may also include a vehicle traveling in the air (e.g., a drone, an airplane, a helicopter, etc.). It may also include vehicles traveling on or in water (e.g., ships, submarines, etc.).
  • the autonomous driving vehicle may or may not accommodate one or more passengers.
  • the autonomous driving vehicle may be applied to unmanned delivery fields, such as express logistics fields, takeaway food delivery fields, etc.
  • the method for determining the orientation information provided in the embodiments of the present application is performed by an autonomous driving vehicle and a server.
  • the server may be a single server, a server cluster consisting of a plurality of servers, or a cloud computing service center.
  • FIG1 is a schematic diagram of an implementation environment provided by an embodiment of the present application. As shown in FIG1 , the implementation environment includes an autonomous driving vehicle 101 and a server 102. The autonomous driving vehicle 101 and the server 102 are connected via a wireless or wired network.
  • the autonomous driving vehicle 101 takes a photo of the target obstacle to obtain point cloud data and image data of the target obstacle.
  • the autonomous driving vehicle 101 determines the orientation information of the target obstacle based on the point cloud data and image data, and sends the orientation information of the target obstacle to the server 102.
  • the server 102 plans the driving path of the autonomous driving vehicle 101 based on the orientation information of the target obstacle, and sends the driving path to the autonomous driving vehicle 101.
  • the autonomous driving vehicle 101 receives the driving path and drives according to the driving path.
  • the autonomous driving vehicle 101 takes a photo of the target obstacle to obtain the point cloud data and image data of the target obstacle, and the autonomous driving vehicle 101 sends the point cloud data and image data of the target obstacle to the server 102.
  • the server 102 determines the orientation information of the target obstacle based on the point cloud data and image data of the target obstacle.
  • the server 102 sends the orientation information of the target obstacle to the autonomous driving vehicle 101, and the autonomous driving vehicle 101 plans the driving path of the autonomous driving vehicle 101 based on the orientation information of the target obstacle; or, the server 102 plans the driving path of the autonomous driving vehicle 101 based on the orientation information of the target obstacle, sends the driving path to the autonomous driving vehicle 101, and the autonomous driving vehicle 101 drives according to the driving path.
  • the autonomous driving vehicle 101 can also complete the above process together with the server 102, and this application does not limit what the autonomous driving vehicle 101 and the server 102 do specifically.
  • FIG2 is a flow chart of a method for determining orientation information provided in an embodiment of the present application.
  • the present application embodiment is illustrated by taking an autonomous driving vehicle as an example.
  • the embodiment includes:
  • the autonomous driving vehicle obtains point cloud data and image data taken for the target obstacle.
  • the autonomous driving vehicle is any device with an automatic driving function.
  • the target obstacle is an object that may interfere with the autonomous driving vehicle during its automatic driving.
  • the target obstacle is other vehicles, pedestrians, etc. around the autonomous driving vehicle or in the lane where the autonomous driving vehicle is located.
  • the embodiments of the present application do not limit the target obstacle.
  • the autonomous driving vehicle is equipped with a radar and a camera.
  • the autonomous driving vehicle photographs the target obstacle through the radar to obtain point cloud data of the target obstacle; the autonomous driving vehicle photographs the target obstacle through the camera to obtain image data of the target obstacle.
  • the autonomous driving vehicle performs a first feature extraction on the point cloud data of the target obstacle to obtain a geometric feature of the target obstacle, and performs a second feature extraction on the image data of the target obstacle to obtain an apparent feature of the target obstacle.
  • Geometric features are used to describe the geometric shape of the target obstacle.
  • Appearance features are used to describe the texture, color and other information of the target obstacle.
  • the autonomous driving vehicle determines a set of candidate orientations of the target obstacle based on the geometric features of the target obstacle.
  • the set includes a plurality of candidate orientation information, where the plurality of candidate orientation information indicates orientations of a plurality of faces of the target obstacle.
  • the orientation of the target obstacle refers to the orientation of the front of the target obstacle.
  • the orientation of the vehicle is the orientation of the front of the vehicle.
  • the orientation of the pedestrian is the orientation of the face.
  • the geometric features of the front and the geometric features of the back or other faces in the point cloud data of some target obstacles are similar.
  • the orientation information of the target obstacle is determined based on the geometric features of the target obstacle, it is very likely that the determined orientation information is the orientation information of the back or other faces of the target obstacle, resulting in inaccurate orientation information.
  • the target obstacle is a large vehicle (for example, a bus, a coach, a truck, etc.), and the geometric shapes of the front and rear of the vehicle are similar.
  • the rear of the vehicle may be misjudged as the front of the vehicle, and the orientation information of the rear of the vehicle is determined as the orientation information of the vehicle, resulting in the determined orientation information being the opposite orientation information.
  • the precise orientation angle can be determined, but the determined orientation angle may not be the orientation angle corresponding to the front side but the orientation angle corresponding to the back side or other sides.
  • the autonomous driving vehicle when the autonomous driving vehicle determines the orientation information of the target obstacle based on the geometric features of the target obstacle, it determines the orientation information of multiple faces of the target obstacle and uses the orientation information of the multiple faces as candidates for the target obstacle orientation information.
  • the autonomous driving vehicle determines first orientation information of the target obstacle based on the apparent characteristics of the target obstacle.
  • the resolution of the image data is low, and it is difficult to determine accurate orientation information based on the apparent characteristics of the target obstacle.
  • the front of the target obstacle can be clearly determined. Therefore, the first orientation information of the target obstacle determined based on the apparent characteristics of the target obstacle has a certain reference value.
  • the autonomous driving vehicle determines, from the candidate orientation set, second orientation information whose indicated orientation matches the orientation indicated by the first orientation information as the orientation information of the target obstacle.
  • the accurate orientation angle can be determined, but the determined orientation angle may not be the orientation angle corresponding to the front side but the orientation angle corresponding to the back side or other sides. Based on the apparent features of the target obstacle, the front side of the target obstacle can be clearly determined. Therefore, the first orientation information can be referenced to select an orientation information from the candidate orientation set as the orientation information of the target obstacle.
  • the second orientation information whose indicated orientation matches the orientation indicated by the first orientation information can be determined from the candidate orientation set as the orientation information of the target obstacle. That is, the second orientation information that is most similar to the first orientation information can be determined from the candidate orientation set as the orientation information of the target obstacle.
  • the method for determining orientation information provided in the embodiment of the present application can determine a coarse-grained candidate orientation set based on the geometric information of the target obstacle in the point cloud data, and select one orientation information from the candidate orientation set as the orientation information of the target obstacle according to the apparent information of the target obstacle in the image data. This is equivalent to jointly determining the orientation information of the target obstacle by integrating the geometric information in the point cloud data and the apparent information in the image data, and can more accurately determine the orientation of the target obstacle.
  • FIG3 is a flow chart of a method for determining orientation information provided in an embodiment of the present application.
  • the present application embodiment is illustrated by taking an autonomous driving vehicle as an example.
  • the embodiment includes:
  • the autonomous driving vehicle obtains point cloud data and image data of the target obstacle.
  • the point cloud data and the image data are acquired at the same time, ensuring that the orientations of the target obstacles in the point cloud data and the image data are the same.
  • the autonomous driving vehicle identifies obstacles from the surrounding environment during driving, and executes the method steps provided in the embodiments of the present application for any identified obstacle.
  • the obstacle can be any movable object, so determining the direction of the obstacle can assist in determining the state of the obstacle at the next moment, which is conducive to the autonomous driving vehicle making more accurate decisions.
  • the autonomous driving vehicle performs a first feature extraction on the point cloud data of the target obstacle to obtain a geometric feature of the target obstacle.
  • the autonomous driving vehicle may use any feature extraction algorithm to perform a first feature extraction on the point cloud data of the target obstacle to obtain the geometric features of the target obstacle.
  • the autonomous driving vehicle performs a first feature extraction on the point cloud data of the target obstacle through a point cloud deep learning model to obtain the geometric features of the target obstacle.
  • the point cloud deep learning model is a model specifically used to process point clouds.
  • the point cloud deep learning model includes a first feature extraction layer, and the autonomous driving vehicle performs a first feature extraction on the point cloud data of the target obstacle through the first feature extraction layer to obtain the geometric features of the target obstacle.
  • the autonomous driving vehicle determines a candidate orientation set of the target obstacle based on the geometric features of the target obstacle, where the candidate orientation set includes multiple candidate orientation information indicating orientations of multiple faces of the target obstacle in the coordinate system of the autonomous driving vehicle.
  • the orientation information of the target obstacle can be determined based on the geometric features of the target obstacle, and the orientation information of multiple faces of the target obstacle can be expanded based on the orientation information of the target obstacle; or the orientation information of multiple faces of the target obstacle can be directly determined.
  • the embodiment of the present application does not limit the method of determining the candidate orientation set.
  • the autonomous driving vehicle determines the orientation information of the target obstacle based on the geometric features of the target obstacle, and based on the orientation information of the target obstacle, expands the orientation information of multiple faces of the target obstacle.
  • the orientation information includes an orientation angle.
  • the autonomous driving vehicle determines a candidate orientation set of the target obstacle based on the geometric features of the target obstacle, including: the autonomous driving vehicle determines the orientation angle of the target obstacle based on the geometric features; increases the orientation angle by 90 degrees, 180 degrees, and 270 degrees respectively to obtain a plurality of expanded orientation angles; determines the orientation angle and the plurality of expanded orientation angles as candidate orientation information to obtain the candidate orientation set.
  • the orientation angle of the target obstacle is the orientation angle of the front side of the target obstacle, after determining the orientation angle of the target obstacle, the orientation angle can be increased by 90 degrees, 180 degrees and 270 degrees respectively to obtain the orientation angles of other faces.
  • the autonomous driving vehicle processes the geometric features of the target obstacle through a point cloud deep learning model to obtain a candidate orientation set of the target obstacle.
  • the autonomous driving vehicle determines a candidate orientation set of the target obstacle based on the geometric features of the target obstacle, including: the autonomous driving vehicle processes the geometric features of the target obstacle through a point cloud deep learning model to obtain the orientation angle of the target obstacle; increases the orientation angle by 90 degrees, 180 degrees, and 270 degrees respectively to obtain multiple expanded orientation angles; determines the orientation angle and the multiple expanded orientation angles as candidate orientation information to obtain the candidate orientation set.
  • the point cloud deep learning model includes a direction determination layer
  • the autonomous driving vehicle processes the geometric features of the target obstacle through the point cloud deep learning model to obtain the direction angle of the target obstacle, including: the autonomous driving vehicle determines the direction The geometric features of the target obstacle are processed in the first layer to obtain the orientation angle of the target obstacle.
  • the point cloud deep learning model can be obtained through sample data training, and the sample data includes sample point cloud data and sample orientation information of sample obstacles.
  • the point cloud deep learning model is trained through the sample data, so that the point cloud deep learning model processes the sample point cloud data, and the error between the obtained orientation information and the sample orientation information converges.
  • the autonomous driving vehicle directly determines the orientation information of multiple faces of the target obstacle based on the geometric features of the target obstacle.
  • the autonomous driving vehicle determines the candidate orientation set of the target obstacle based on the geometric features of the target obstacle, including: the autonomous driving vehicle determines the orientation information of multiple faces of the target obstacle based on the geometric features; and determines the orientation information of the multiple faces as the candidate orientation set of the target obstacle.
  • the autonomous driving vehicle processes the geometric features of the target obstacle through a point cloud deep learning model to obtain a candidate orientation set of the target obstacle.
  • the autonomous driving vehicle determines a candidate orientation set of the target obstacle based on the geometric features of the target obstacle, including: the autonomous driving vehicle processes the geometric features of the target obstacle through a point cloud deep learning model to obtain orientation information of multiple faces of the target obstacle.
  • the point cloud deep learning model includes an orientation determination layer
  • the autonomous driving vehicle processes the geometric features of the target obstacle through the point cloud deep learning model to obtain the orientation information of multiple faces of the target obstacle, including: the autonomous driving vehicle processes the geometric features of the target obstacle through the orientation determination layer to obtain the orientation information of multiple faces of the target obstacle.
  • the point cloud deep learning model can be obtained through training with sample data, and the sample data includes sample point cloud data of sample obstacles and sample orientation information of multiple faces of the sample obstacles.
  • the point cloud deep learning model is trained with the sample data, so that the point cloud deep learning model processes the sample point cloud data, and the error between the orientation information of multiple faces obtained and the sample orientation information of the multiple faces converges.
  • the autonomous driving vehicle obtains multiple frames of point cloud data for the target obstacle during driving, and the autonomous driving vehicle determines the orientation information of the target obstacle based on the multiple frames of point cloud data.
  • the autonomous driving vehicle can fuse the geometric features of the target obstacle in the multiple frames of point cloud data.
  • the point cloud data is multiple frames of point cloud data, and the multiple frames of point cloud data are image data obtained by the autonomous driving vehicle shooting the target obstacle at different times.
  • the autonomous driving vehicle extracts the first feature of the point cloud data of the target obstacle to obtain the geometric features of the target obstacle, including: extracting the first feature of each frame of point cloud data in the multiple frames of point cloud data of the target obstacle to obtain the first geometric feature corresponding to each frame of point cloud data; and performing the following operations on each frame of point cloud data in sequence according to the shooting order of the multiple frames of point cloud data: fusing the first geometric feature of the current frame of point cloud data with the first geometric feature of the previous frame of point cloud data to obtain the second geometric feature of the current frame of point cloud data.
  • the autonomous driving vehicle determines the candidate orientation set of the target obstacle based on the geometric features of the target obstacle, including: the autonomous driving vehicle determines the candidate orientation set of the target obstacle based on the second geometric feature of the last frame of point cloud data.
  • the autonomous driving vehicle performs a second feature extraction on the image data of the target obstacle to obtain an apparent feature of the target obstacle; based on the apparent feature of the target obstacle, first orientation information of the target obstacle is determined, where the first orientation information indicates an orientation of the target obstacle in the observer coordinate system.
  • the image data of the target obstacle is obtained by shooting the autonomous driving vehicle, so the autonomous driving vehicle is the observer of the target obstacle.
  • the observer coordinate system takes the first ray pointed by the autonomous driving vehicle to the target obstacle as the vertical axis and the second ray passing through the center of the target obstacle and perpendicular to the first ray as the horizontal axis.
  • the autonomous driving vehicle may use any algorithm to extract the second feature of the image data of the target obstacle to obtain the apparent feature of the target obstacle, and determine the first orientation information of the target obstacle based on the apparent feature of the target obstacle.
  • the embodiment of the present application is not limited to this.
  • the image data is a plurality of frames of image data
  • the plurality of frames of image data is image data obtained by the autonomous driving vehicle shooting the target obstacle at different times.
  • the autonomous driving vehicle performs a second feature extraction on the image data of the target obstacle to obtain the apparent features of the target obstacle, including: performing a second feature extraction on each frame of image data in the plurality of frames of image data of the target obstacle to obtain the first apparent features corresponding to each frame of image data; and performing the following steps on each frame of image data in sequence according to the shooting order of the plurality of frames of image data: fusing the first apparent features of the current frame of image data with the first apparent features of the previous frame of image data to obtain the second apparent features of the current frame of image data.
  • the autonomous driving vehicle determines the first orientation information of the target obstacle based on the apparent features of the target obstacle, including: determining the first orientation information of the target obstacle based on the second apparent features of the last frame of image data.
  • the embodiment of the present application can refer to the appearance features in multiple frames of image data to determine the orientation information of the target obstacle, so that the determined orientation information is more accurate.
  • the first orientation information is a first orientation category; the apparent characteristics of the target obstacle and the first orientation category are determined by an orientation classification model, which is used to determine an orientation category that matches the apparent characteristics of the obstacle from a target number of orientation categories.
  • the target number can be any number, and the embodiment of the present application does not limit the target number. In some embodiments, the target number is 8.
  • the target number of orientation categories includes front, right front, right, right rear, rear, left rear, left and left front; or, the target number of orientation categories includes 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees and 315 degrees.
  • the embodiment of the present application is modeled with a pinhole imaging model, with the observer coordinate system as the reference coordinate system, and the obstacle orientation is divided into eight regions: front, right front, right, right rear, rear, left rear, left, and left front, corresponding to 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees, and 315 degrees, respectively.
  • the labeling personnel label the sample image data according to the standard, accumulate the sample data, and train the orientation classification model.
  • the orientation classification model includes a feature extraction layer, a fusion layer, and an orientation classification layer.
  • the autonomous driving vehicle extracts features from multiple frames of image data of the target obstacle through the feature extraction layer to obtain the appearance features corresponding to each frame of image data; through the fusion layer, in accordance with the shooting order of the multiple frames of image data, each frame of image data is sequentially executed: the first appearance feature of the current frame of image data is fused with the first appearance feature of the previous frame of image data to obtain the second appearance feature of the current frame of image data; through the orientation classification layer, the second appearance feature of the last frame of image data is classified to obtain the first orientation category of the target obstacle.
  • the autonomous driving vehicle determines an angle between the autonomous driving vehicle coordinate system and the observer coordinate system.
  • the autonomous driving vehicle coordinate system takes the center of the autonomous driving vehicle as the origin, the longitudinal direction of the autonomous driving vehicle as the longitudinal axis, and the lateral direction of the autonomous driving vehicle as the horizontal axis.
  • the observer coordinate system takes the center of the target obstacle as the origin, the first ray pointed by the autonomous driving vehicle to the target obstacle as the longitudinal axis, and the second ray passing through the center of the target obstacle and perpendicular to the first ray as the horizontal axis. Therefore, based on the position of the center point of the autonomous driving vehicle and the position of the center point of the target obstacle, the angle between the autonomous driving vehicle coordinate system and the observer coordinate system can be determined.
  • the sine function value of the angle is the ratio of the lateral distance between the autonomous driving vehicle and the target obstacle to the distance between the autonomous driving vehicle and the target obstacle.
  • the lateral distance between the autonomous driving vehicle and the target obstacle refers to the distance from the center point of the target obstacle to the autonomous driving vehicle.
  • the autonomous driving vehicle determines the angle between the autonomous driving vehicle coordinate system and the observer coordinate system, including: obtaining the position of the center point of the autonomous driving vehicle and the position of the center point of the target obstacle; based on the position of the center point of the autonomous driving vehicle and the position of the center point of the target obstacle, determining the lateral distance between the center point of the autonomous driving vehicle and the center point of the target obstacle, and also determining the distance between the center point of the autonomous driving vehicle and the center point of the target obstacle; obtaining the ratio of the distance and the lateral distance, and determining the angle between the autonomous driving vehicle coordinate system and the observer coordinate system based on the sine function and the ratio.
  • the autonomous driving vehicle determines fourth orientation information corresponding to the first orientation information in the autonomous driving vehicle coordinate system, where the fourth orientation information indicates the orientation of the target obstacle in the autonomous driving vehicle coordinate system.
  • the first orientation information and the candidate orientation information belong to different coordinate systems, the first orientation information can be mapped to the autonomous driving vehicle coordinate system to obtain the fourth orientation information.
  • the fourth orientation information determined based on the image data and the candidate orientation information determined based on the point cloud data are in the same coordinate system.
  • the autonomous driving vehicle determines, from the candidate orientation set, second orientation information whose indicated orientation matches the orientation indicated by the fourth orientation information as the orientation information of the target obstacle.
  • the indicated orientation matches the orientation indicated by the fourth orientation information, which means that the indicated orientation is similar to or close to the orientation indicated by the fourth orientation information.
  • the autonomous driving vehicle determines, from the candidate orientation set, second orientation information whose indicated orientation matches the orientation indicated by the fourth orientation information as the orientation information of the target obstacle, including: determining, from the candidate orientation set, second orientation information whose indicated orientation has the smallest difference with the orientation indicated by the fourth orientation information as the orientation information of the target obstacle.
  • Figure 5 is an obstacle whose direction needs to be determined.
  • the process of determining the direction of the obstacle can be shown in Figure 6.
  • the direction angle of the obstacle in the autonomous driving vehicle coordinate system is determined to be ⁇ + ⁇ , and the direction angle is expanded to obtain a candidate direction set ⁇ + ⁇ , ⁇ + ⁇ +90°, ⁇ + ⁇ +180°, ⁇ + ⁇ +270° ⁇ ;
  • the direction angle of the obstacle in the observer coordinate system is determined to be ⁇ ;
  • the angle between the autonomous driving vehicle coordinate system and the observer coordinate system is determined to be ⁇ , and the direction angle with the smallest difference from the ⁇ + ⁇ angle is determined from the candidate direction set as the direction angle of the autonomous driving vehicle.
  • multiple candidate orientation information indicates the orientation of the target obstacle in the autonomous driving vehicle coordinate system
  • the first orientation information indicates the orientation of the target obstacle in the observer coordinate system.
  • multiple candidate orientation information indicates the orientation of the target obstacle in the world coordinate system
  • the first orientation information indicates the orientation of the target obstacle in the world coordinate system.
  • the embodiment of the present application does not limit the coordinate system.
  • the method provided in the embodiment of the present application can reduce the error rate of vehicle orientation recognition by 20% and increase the accuracy of pedestrian orientation recognition by 50% compared with determining orientation information based only on point cloud data.
  • the method for determining orientation information provided in the embodiment of the present application can determine a coarse-grained candidate orientation set based on the geometric information of the target obstacle in the point cloud data, and select one orientation information from the candidate orientation set as the orientation information of the target obstacle according to the apparent information of the target obstacle in the image data. This is equivalent to jointly determining the orientation information of the target obstacle by integrating the geometric information in the point cloud data and the apparent information in the image data, and can more accurately determine the orientation of the target obstacle.
  • FIG. 7 is a schematic diagram of the structure of a device for determining orientation information provided in an embodiment of the present application.
  • the device includes:
  • An acquisition module 701 is used to acquire point cloud data and image data captured by the autonomous driving vehicle for a target obstacle;
  • a feature extraction module 702 is used to perform a first feature extraction on the point cloud data of the target obstacle to obtain a geometric feature of the target obstacle, and perform a second feature extraction on the image data of the target obstacle to obtain an apparent feature of the target obstacle;
  • a first determination module 703 is configured to determine a candidate orientation set of the target obstacle based on the geometric features of the target obstacle, wherein the candidate orientation set includes a plurality of candidate orientation information, and the plurality of candidate orientation information indicates orientations of a plurality of faces of the target obstacle;
  • a second determination module 704 configured to determine first orientation information of the target obstacle based on the apparent characteristics of the target obstacle
  • the third determination module 705 is configured to determine, from the candidate orientation set, second orientation information whose indicated orientation matches the orientation indicated by the first orientation information as the orientation information of the target obstacle.
  • the orientation information includes an orientation angle
  • the first determination module 703 is used to determine the orientation angle of the target obstacle based on the geometric features
  • the orientation angle is increased by 90 degrees, 180 degrees and 270 degrees respectively to obtain multiple expanded orientation angles
  • the orientation angle and the multiple expanded orientation angles are determined as candidate orientation information to obtain the candidate orientation set.
  • the first determination module 703 is configured to determine orientation information of multiple faces of the target obstacle based on the geometric features; and determine the orientation information of the multiple faces as a candidate orientation set of the target obstacle.
  • the image data is multi-frame image data
  • the multi-frame image data is image data obtained by the autonomous driving vehicle photographing the target obstacle at different times
  • the feature extraction module 702 is used to extract the third feature of each frame of image data in the multiple frames of image data of the target obstacle to obtain the first appearance feature corresponding to each frame of image data; and perform the following steps on each frame of image data in sequence according to the shooting order of the multiple frames of image data: fusing the first appearance feature of the current frame of image data with the first appearance feature of the previous frame of image data to obtain the second appearance feature of the current frame of image data;
  • the second determination module 704 is configured to determine first orientation information of the target obstacle based on a second appearance feature of the last frame of image data.
  • the first orientation information is a first orientation category; the apparent features of the target obstacle and the first orientation category are determined by an orientation classification model, and the orientation classification model is used to determine an orientation category that matches the apparent features of the obstacle from a target number of orientation categories.
  • the target number of orientation categories includes front, right front, right, right rear, rear, left rear, left and left front; or, the target number of orientation categories includes 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees and 315 degrees.
  • the multiple candidate orientation information indicate the orientation of the target obstacle in the autonomous driving vehicle coordinate system
  • the first orientation information indicates the orientation of the target obstacle in the observer coordinate system
  • the observer coordinate system has a first ray pointing from the autonomous driving vehicle to the target obstacle as the vertical axis and a second ray passing through the center of the target obstacle and perpendicular to the first ray as the horizontal axis.
  • the third determination module 705 is used to determine the angle between the autonomous driving vehicle coordinate system and the observer coordinate system; based on the angle, determine fourth orientation information corresponding to the first orientation information in the autonomous driving vehicle coordinate system, the fourth orientation information indicating the orientation of the target obstacle in the autonomous driving vehicle coordinate system; determine second orientation information from the candidate orientation set whose indicated orientation matches the orientation indicated by the fourth orientation information as the target obstacle The direction information of the marked obstacle.
  • the third determination module 705 is configured to determine, from the candidate orientation set, second orientation information whose indicated orientation has the smallest difference from the orientation indicated by the fourth orientation information as the orientation information of the target obstacle.
  • the orientation information determination device provided in the above embodiment only uses the division of the above functional modules as an example when calibrating.
  • the above functions can be assigned to different functional modules as needed, that is, the internal structure of the autonomous driving vehicle is divided into different functional modules to complete all or part of the functions described above.
  • the orientation information determination device provided in the above embodiment and the orientation information determination method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.
  • FIG8 shows a block diagram of an autonomous driving vehicle 800 provided by an exemplary embodiment of the present application.
  • the autonomous driving vehicle 800 includes: a processor 801 and a memory 802 .
  • the processor 801 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc.
  • the processor 801 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array).
  • the processor 801 may also include a main processor and a coprocessor.
  • the main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state.
  • the processor 801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen.
  • the processor 801 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.
  • AI Artificial Intelligence
  • the memory 802 may include one or more computer-readable storage media, which may be non-transitory.
  • the memory 802 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices.
  • the non-transitory computer-readable storage medium in the memory 802 is used to store at least one program code, which is executed by the processor 801 to implement the orientation information determination method provided in the method embodiment of the present application.
  • the autonomous driving vehicle 800 may also optionally include: a peripheral device interface 803 and at least one peripheral device.
  • the processor 801, the memory 802 and the peripheral device interface 803 may be connected via a bus or a signal line.
  • Each peripheral device may be connected to the peripheral device interface 803 via a bus, a signal line or a circuit board.
  • the peripheral device includes: at least one of a radio frequency circuit 804, a display screen 805, a camera assembly 806, an audio circuit 807, a positioning assembly 808 and a power supply 809.
  • the peripheral device interface 803 may be used to connect at least one peripheral device related to I/O (Input/Output) to the processor 801 and the memory 802.
  • the processor 801, the memory 802, and the peripheral device interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 801, the memory 802, and the peripheral device interface 803 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.
  • the RF circuit 804 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals.
  • the RF circuit 804 communicates with the communication network and other communication devices through electromagnetic signals.
  • the RF circuit 804 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals.
  • the RF circuit 804 includes: an antenna system, an RF transceiver,
  • the RF circuit 804 may include a device, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like.
  • the RF circuit 804 may communicate with other autonomous driving vehicles through at least one wireless communication protocol.
  • the wireless communication protocol includes, but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and/or a WiFi (Wireless Fidelity) network.
  • the RF circuit 804 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.
  • the display screen 805 is used to display a UI (User Interface).
  • the UI may include graphics, text, icons, videos, and any combination thereof.
  • the display screen 805 also has the ability to collect touch signals on the surface or above the surface of the display screen 805.
  • the touch signal may be input as a control signal to the processor 801 for processing.
  • the display screen 805 may also be used to provide virtual buttons and/or virtual keyboards, also known as soft buttons and/or soft keyboards.
  • the display screen 805 may be one, and the front panel of the autonomous driving vehicle 800 may be set; in other embodiments, the display screen 805 may be at least two, and they may be set on different surfaces of the autonomous driving vehicle 800 or in a folding design; in still other embodiments, the display screen 805 may be a flexible display screen, and may be set on a curved surface or a folding surface of the autonomous driving vehicle 800. Even more, the display screen 805 may be set to a non-rectangular irregular shape, that is, a special-shaped screen.
  • the display screen 805 may be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).
  • the camera assembly 806 is used to capture images or videos.
  • the camera assembly 806 includes a front camera and a rear camera.
  • the front camera is set on the front panel of the autonomous driving vehicle, and the rear camera is set on the back of the autonomous driving vehicle.
  • there are at least two rear cameras which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions.
  • the camera assembly 806 may also include a flash.
  • the flash can be a single-color temperature flash or a dual-color temperature flash.
  • a dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.
  • the audio circuit 807 may include a microphone and a speaker.
  • the microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 801 for processing, or input them into the RF circuit 804 to achieve voice communication.
  • the microphone may also be an array microphone or an omnidirectional acquisition microphone.
  • the speaker is used to convert the electrical signal from the processor 801 or the RF circuit 804 into sound waves.
  • the speaker may be a traditional film speaker or a piezoelectric ceramic speaker.
  • the speaker When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging.
  • the audio circuit 807 may also include a headphone jack.
  • the positioning component 808 is used to locate the current geographic location of the autonomous driving vehicle 800 to achieve navigation or LBS (Location Based Service).
  • the positioning component 808 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system, Russia's Grenas system, or the European Union's Galileo system.
  • the power supply 809 is used to power various components in the autonomous driving vehicle 800.
  • the power supply 809 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery.
  • the rechargeable battery can support wired charging or wireless charging.
  • the rechargeable battery can also be used to support fast charging technology.
  • the autonomous driving vehicle 800 further includes one or more sensors 810.
  • the one or more sensors 810 Including but not limited to: an acceleration sensor 811 , a gyroscope sensor 812 , a pressure sensor 813 , a fingerprint sensor 814 , an optical sensor 815 and a proximity sensor 816 .
  • the acceleration sensor 811 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the autonomous driving vehicle 800.
  • the acceleration sensor 811 can be used to detect the components of gravity acceleration on the three coordinate axes.
  • the processor 801 can control the display screen 805 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 811.
  • the acceleration sensor 811 can also be used to collect motion data of games or users.
  • the gyro sensor 812 can detect the body direction and rotation angle of the autonomous driving vehicle 800, and the gyro sensor 812 can cooperate with the acceleration sensor 811 to collect the user's 3D actions on the autonomous driving vehicle 800.
  • the processor 801 can implement the following functions based on the data collected by the gyro sensor 812: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.
  • the pressure sensor 813 can be set on the side frame of the autonomous driving vehicle 800 and/or the lower layer of the display screen 805.
  • the pressure sensor 813 can detect the user's grip signal of the autonomous driving vehicle 800, and the processor 801 performs left and right hand recognition or quick operation according to the grip signal collected by the pressure sensor 813.
  • the processor 801 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 805.
  • the operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.
  • the fingerprint sensor 814 is used to collect the user's fingerprint, and the processor 801 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 814, or the fingerprint sensor 814 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 801 authorizes the user to perform relevant sensitive operations, which include unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings.
  • the fingerprint sensor 814 can be set on the front, back, or side of the autonomous driving vehicle 800. When a physical button or manufacturer logo is set on the autonomous driving vehicle 800, the fingerprint sensor 814 can be integrated with the physical button or manufacturer logo.
  • the optical sensor 815 is used to collect the ambient light intensity.
  • the processor 801 can control the display brightness of the display screen 805 according to the ambient light intensity collected by the optical sensor 815. Specifically, when the ambient light intensity is high, the display brightness of the display screen 805 is increased; when the ambient light intensity is low, the display brightness of the display screen 805 is reduced.
  • the processor 801 can also dynamically adjust the shooting parameters of the camera assembly 806 according to the ambient light intensity collected by the optical sensor 815.
  • the proximity sensor 816 also called a distance sensor, is disposed on the front panel of the autonomous driving vehicle 800.
  • the proximity sensor 816 is used to collect the distance between the user and the front of the autonomous driving vehicle 800.
  • the processor 801 controls the display screen 805 to switch from the screen-on state to the screen-off state; when the proximity sensor 816 detects that the distance between the user and the front of the autonomous driving vehicle 800 is gradually increasing, the processor 801 controls the display screen 805 to switch from the screen-off state to the screen-on state.
  • FIG. 8 does not constitute a limitation on the autonomous driving vehicle 800 , and may include more or fewer components than shown, or combine certain components, or adopt a different component arrangement.
  • FIG9 is a schematic diagram of the structure of a server provided in an embodiment of the present application.
  • the server 900 may have relatively large differences due to different configurations or performances, and may include one or more processors 901, such as CPUs (Central Processing Units) and one or more memories 902, wherein the memory 902 stores at least one program code. At least one program code is loaded and executed by the processor 901 to implement the methods provided by the above-mentioned various method embodiments.
  • the server may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output.
  • the server may also include other components for implementing device functions, which will not be described in detail here.
  • the server 900 is used to execute the steps executed by the server in the above method embodiment.
  • a computer-readable storage medium such as a memory including a program code, and the program code can be executed by a processor in a computer device to complete the orientation information determination method in the above embodiment.
  • the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
  • a computer program or a computer program product is further provided.
  • the computer program or the computer program product includes a computer program code.
  • the computer program code When the computer program code is executed by a computer, the computer implements the method for determining the orientation information in the above embodiment.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)

Abstract

本申请公开了一种朝向信息确定方法，其包括：获取自动驾驶车辆针对目标障碍物拍摄的点云数据和图像数据；对目标障碍物的点云数据进行第一特征提取，得到目标障碍物的几何特征，对目标障碍物的图像数据进行第二特征提取，得到目标障碍物的表观特征；基于目标障碍物的几何特征，确定目标障碍物的候选朝向集合，候选朝向集合包括多个候选朝向信息，多个候选朝向信息指示目标障碍物多个面的朝向；基于目标障碍物的表观特征，确定目标障碍物的第一朝向信息；从候选朝向集合中确定所指示朝向与第一朝向信息所指示朝向匹配的第二朝向信息作为目标障碍物的朝向信息。

Description

确定朝向信息以及自动驾驶车辆 技术领域
本申请涉及自动驾驶技术领域，特别涉及一种朝向信息确定方法以及自动驾驶车辆。
背景技术
自动驾驶应用场景下，障碍物的朝向信息对于自动驾驶车辆的决策和路径规划有着重要的影响，因此，如何确定自动驾驶车辆周围障碍物的朝向信息成为目前亟需解决的问题。
目前，可以通过自动驾驶车辆的雷达获取障碍物的点云数据，基于该点云数据，确定该障碍物的朝向信息。
发明内容
本申请实施例提供了一种朝向信息确定方法以及自动驾驶车辆，能够更加准确地确定障碍物的朝向。该技术方案如下：
一方面，提供了一种朝向信息确定方法，所述方法包括：
获取自动驾驶车辆针对目标障碍物拍摄的点云数据和图像数据；
对所述目标障碍物的点云数据进行第一特征提取，得到所述目标障碍物的几何特征，对所述目标障碍物的图像数据进行第二特征提取，得到所述目标障碍物的表观特征；
基于所述目标障碍物的几何特征，确定所述目标障碍物的候选朝向集合，所述候选朝向集合包括多个候选朝向信息，所述多个候选朝向信息指示所述目标障碍物多个面的朝向；
基于所述目标障碍物的表观特征，确定所述目标障碍物的第一朝向信息；
从所述候选朝向集合中确定所指示朝向与所述第一朝向信息所指示朝向匹配的第二朝向信息作为所述目标障碍物的朝向信息。
一方面，提供了一种朝向信息确定装置，所述装置包括：
获取模块，用于获取自动驾驶车辆针对目标障碍物拍摄的点云数据和图像数据；
特征提取模块，用于对所述目标障碍物的点云数据进行第一特征提取，得到所述目标障碍物的几何特征，对所述目标障碍物的图像数据进行第二特征提取，得到所述目标障碍物的表观特征；
第一确定模块，用于基于所述目标障碍物的几何特征，确定所述目标障碍物的候选朝向集合，所述候选朝向集合包括多个候选朝向信息，所述多个候选朝向信息指示所述目标障碍物多个面的朝向；
第二确定模块，用于基于所述目标障碍物的表观特征，确定所述目标障碍物的第一朝向信息；
第三确定模块，用于从所述候选朝向集合中确定所指示朝向与所述第一朝向信息所指示朝向匹配的第二朝向信息作为所述目标障碍物的朝向信息。
在一种可能的实现方式中，所述朝向信息包括朝向角度；所述第一确定模块，用于基于所述几何特征，确定所述目标障碍物的朝向角度；将所述朝向角度分别增加90度、180度和270度，得到多个扩充的朝向角度；将所述朝向角度和所述多个扩充的朝向角度确定为候选朝向信息，得到所述候选朝向集合。
在一种可能的实现方式中，所述第一确定模块，用于基于所述几何特征，确定所述目标障碍物多个面的朝向信息；将所述多个面的朝向信息，确定为所述目标障碍物的候选朝向集合。
在一种可能的实现方式中，所述图像数据为多帧图像数据，所述多帧图像数据为所述自动驾驶车辆在不同时刻拍摄所述目标障碍物得到的图像数据；
所述特征提取模块，用于对所述目标障碍物的多帧图像数据中每帧图像数据进行第三特征提取，得到所述每一帧图像数据对应的第一表观特征；按照所述多帧图像数据的拍摄顺序，对每帧图像数据依次执行：将当前帧图像数据的第一表观特征与上一帧图像数据的第一表观特征进行融合，得到所述当前帧图像数据的第二表观特征；基于最后一帧图像数据的第二表观特征，确定所述目标障碍物的第一朝向信息。
在一种可能的实现方式中，所述第一朝向信息为第一朝向类别；所述目标障碍物的表观特征和所述第一朝向类别由朝向分类模型确定，所述朝向分类模型用于从目标数量个朝向类别中确定与障碍物的表观特征匹配的朝向类别。
在一种可能的实现方式中，所述目标数量个朝向类别包括前、右前、右、右后、后、左后、左和左前；或者，所述目标数量个朝向类别包括0度、45度、90度、135度、180度、225度、270度和315度。
在一种可能的实现方式中，所述多个候选朝向信息指示所述目标障碍物在自动驾驶车辆坐标系中的朝向，所述第一朝向信息指示所述目标障碍物在观察者坐标系中的朝向，所述观察者坐标系以所述自动驾驶车辆指向所述目标障碍物的第一射线为纵轴，以经过所述目标障碍物中心且垂直于所述第一射线的第二射线为横轴。
在一种可能的实现方式中，所述第三确定模块，用于确定所述自动驾驶车辆坐标系与所述观察者坐标系的夹角；基于所述夹角，确定所述第一朝向信息在所述自动驾驶车辆坐标系中对应的第四朝向信息，所述第四朝向信息指示所述目标障碍物在所述自动驾驶车辆坐标系中的朝向；从所述候选朝向集合中确定所指示朝向与所述第四朝向信息所指示朝向匹配的第二朝向信息作为所述目标障碍物的朝向信息。
在一种可能的实现方式中，所述第三确定模块，用于从所述候选朝向集合中确定所指示朝向与所述第四朝向信息所指示朝向相差最小的第二朝向信息作为所述目标障碍物的朝向信息。
一方面，提供了一种自动驾驶车辆，所述自动驾驶车辆包括一个或多个处理器和一个或多个存储器，所述一个或多个存储器中存储有至少一条程序代码，所述至少一条程序代码由所述一个或多个处理器加载并执行以实现如上述任一种可能实现方式的朝向信息确定方法所执行的操作。
一方面，提供了一种计算机可读存储介质，该存储介质中存储有至少一条程序代码，该至少一条程序代码由处理器加载并执行以实现如上述任一种可能实现方式的朝向信息确定方法所执行的操作。
一方面，提供了一种计算机程序或计算机程序产品，所述计算机程序或计算机程序产品包括：计算机程序代码，所述计算机程序代码被计算机执行时，使得所述计算机实现如上述任一种可能实现方式的朝向信息确定方法所执行的操作。
本申请实施例提供的朝向信息确定方法以及自动驾驶车辆，可以基于点云数据中目标障碍物的几 何信息确定粗粒度的候选朝向集合，根据图像数据中目标障碍物的表观信息从候选朝向集合中择优选择一个朝向信息作为目标障碍物的朝向信息，相当于综合点云数据中的几何信息和图像数据中的表观信息共同确定目标障碍物的朝向信息，能够更加准确地确定目标障碍物的朝向。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的一种实施环境的示意图；
图2是本申请实施例提供的一种朝向信息确定方法的流程图；
图3是本申请实施例提供的一种朝向信息确定方法的流程图；
图4是本申请实施例提供的一种朝向分类模型的结构示意图；
图5是本申请实施例提供的一种图像数据的示意图；
图6是本申请实施例提供的一种坐标系的示意图；
图7是本申请实施例提供的一种朝向信息确定装置的结构示意图；
图8是本申请实施例提供的自动驾驶车辆的结构示意图；
图9是本申请实施例提供的服务器的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
可以理解，本申请所使用的术语“第一”、“第二”等可在本文中用于描述各种概念，但除非特别说明，这些概念不受这些术语限制。这些术语仅用于将一个概念与另一个概念区分。举例来说，在不脱离本申请的范围的情况下，可以将第一订单称为第二订单，将第二订单称为第一订单。
本申请所使用的术语“至少一个”、“多个”、“每个”、“任一”，至少一个包括一个、两个或者两个以上，多个包括两个或者两个以上，而每个是指对应的多个中的每一个，任一是指多个中的任意一个，举例来说，多个角点包括3个角点，而每个是指这3个角点中的每一个角点，任一是指这3个角点中的任意一个，可以是第一个，也可以是第二个，还可以是第三个。
需要说明的是，本申请所涉及的信息(包括但不限于用户设备信息、用户个人信息等)、数据(包括但不限于用于分析的数据、存储的数据、展示的数据等)以及信号，均为经用户授权或者经过各方充分授权的，且相关数据的收集、使用和处理需要遵守相关国家和地区的相关法律法规和标准。例如，本申请中涉及到的图像数据、点云数据等都是在充分授权的情况下获取的。且上述信息、数据经过加工处理后使用于大数据应用场景，无法识别至任意自然人或与其产生特定关联。
在一些实施例中，本申请实施例提供的朝向信息确定方法由自动驾驶车辆执行。该自动驾驶车辆是任一具有自动行驶功能的设备。在一些实施例中，该自动驾驶车辆包括在地面上行驶的车辆(例如，汽车、卡车、公交车等)，也可以包括在空中行驶的车辆(例如，无人机、飞机、直升机等)， 也可以包括在水上或者水中行驶的车辆(例如，船、潜艇等)。该自动驾驶车辆可以容纳或者不容纳一个或者多个乘客。另外，该自动驾驶车辆可以应用于无人配送领域，例如，快递物流领域、外卖送餐领域等。
在另一些实施例中，本申请实施例提供的朝向信息确定方法由自动驾驶车辆和服务器执行。该服务器可以为一台服务器，或者由若干服务器组成的服务器集群，或者是一个云计算服务中心。
需要说明的是，本申请实施例对朝向信息确定方法的执行主体不做限定。
图1是本申请实施例提供的一种实施环境的示意图，如图1所示，该实施环境包括自动驾驶车辆101和服务器102，该自动驾驶车辆101与服务器102之间通过无线或者有线网络连接。
在一些实施例中，自动驾驶车辆101针对目标障碍物进行拍摄，得到目标障碍物的点云数据和图像数据，自动驾驶车辆101基于点云数据和图像数据确定目标障碍物的朝向信息，向服务器102发送该目标障碍物的朝向信息。服务器102基于该目标障碍物的朝向信息规划自动驾驶车辆101的行驶路径，向自动驾驶车辆101发送该行驶路径。自动驾驶车辆101接收该行驶路径，按照该行驶路径进行行驶。
在一些实施例中，自动驾驶车辆101针对目标障碍物进行拍摄，得到目标障碍物的点云数据和图像数据，自动驾驶车辆101向服务器102发送该目标障碍物的点云数据和图像数据。服务器102基于该目标障碍物的点云数据和图像数据，确定该目标障碍物的朝向信息。服务器102向该自动驾驶车辆101发送该目标障碍物的朝向信息，由自动驾驶车辆101基于该目标障碍物的朝向信息规划自动驾驶车辆101的行驶路径；或者，服务器102基于该目标障碍物的朝向信息规划自动驾驶车辆101的行驶路径，向自动驾驶车辆101发送该行驶路径，由自动驾驶车辆101按照该行驶路径进行行驶。当然，自动驾驶车辆101还可以和服务器102共同完成上述过程，本申请对自动驾驶车辆101和服务器102具体做什么不做限定。
图2是本申请实施例提供的一种朝向信息确定方法的流程图。本申请实施例以执行主体为自动驾驶车辆为例进行示例性说明，该实施例包括：
201、自动驾驶车辆获取针对目标障碍物拍摄的点云数据和图像数据。
其中，自动驾驶车辆是任一具有自动行驶功能的设备。目标障碍物是自动驾驶车辆自动行驶过程中可能对自动驾驶车辆产生干扰的物体。例如，目标障碍物是自动驾驶车辆周围或自动驾驶车辆所在车道上的其他车辆、行人等，本申请实施例对目标障碍物不做限定。
在一些实施例中，自动驾驶车辆安装有雷达和相机。自动驾驶车辆通过雷达对目标障碍物进行拍摄，得到目标障碍物的点云数据；自动驾驶车辆通过相机对目标障碍物进行拍摄，得到目标障碍物的图像数据。
202、自动驾驶车辆对目标障碍物的点云数据进行第一特征提取，得到目标障碍物的几何特征，对目标障碍物的图像数据进行第二特征提取，得到目标障碍物的表观特征。
几何特征是用于描述目标障碍物几何形状的特征。表观特征是用于描述目标障碍物的纹理、颜色等信息的特征。
203、自动驾驶车辆基于目标障碍物的几何特征，确定目标障碍物的候选朝向集合，该候选朝向 集合包括多个候选朝向信息，该多个候选朝向信息指示目标障碍物多个面的朝向。
除非特别说明，目标障碍物的朝向是指目标障碍物正面的朝向。以目标障碍物为车辆为例，车辆的朝向是车头的朝向。以目标障碍物为行人为例，行人的朝向是人脸的朝向。但是，一些目标障碍物的点云数据中正面的几何特征和背面或其他面的几何特征是相似的，基于目标障碍物的几何特征确定目标障碍物的朝向信息时，很有可能确定出的朝向信息是目标障碍物背面或者其他面的朝向信息，导致得到的朝向信息不准确。例如，目标障碍物为大型车辆(例如，公交车、大巴、卡车等)，车头和车尾的几何形状相似，基于该车辆的几何特征确定该车辆的朝向信息时，可能出现将车尾误判为车头，从而将车尾的朝向信息确定为车辆的朝向信息，导致确定出的朝向信息是相反的朝向信息。
需要说明的是，基于目标障碍物的几何特征确定朝向信息时，可以确定出精确的朝向角度，只是确定出的朝向角度可能不是正面对应的朝向角度而是背面或其他面对应的朝向角度。
为此，本申请实施例中，自动驾驶车辆基于目标障碍物的几何特征，确定目标障碍物的朝向信息时，确定目标障碍物多个面的朝向信息，将该多个面的朝向信息作为目标障碍物朝向信息的候选。
204、自动驾驶车辆基于目标障碍物的表观特征，确定目标障碍物的第一朝向信息。
需要说明的是，图像数据的分辨率较低，基于目标障碍物的表观特征，难以确定出精准的朝向信息。但是基于目标障碍物的表观特征，可以明确确定目标障碍物的正面，因此，基于目标障碍物的表观特征确定的目标障碍物的第一朝向信息具有一定的参考性。
例如，基于车辆的表观特征，可以明确确定车头以及车头大致朝东，但是无法准确地确定出车头朝正东，还是车头与正东存在一定角度，存在多大角度。
205、自动驾驶车辆从候选朝向集合中确定所指示朝向与第一朝向信息所指示朝向匹配的第二朝向信息作为目标障碍物的朝向信息。
由于基于目标障碍物的几何特征确定朝向信息时，可以确定出精确的朝向角度，只是确定出的朝向角度可能不是正面对应的朝向角度而是背面或其他面对应的朝向角度。而基于目标障碍物的表观特征，可以明确确定目标障碍物的正面，因此，可以参考第一朝向信息从候选朝向集合中择优选择一个朝向信息作为目标障碍物的朝向信息。
虽然第一朝向信息不够准确，但是大致指向是正确的，因此，可以从候选朝向集合中确定所指示朝向与第一朝向信息所指示朝向匹配的第二朝向信息作为目标障碍物的朝向信息，也即是，从候选朝向集合中确定与第一朝向信息最相似的第二朝向信息作为目标障碍物的朝向信息。
本申请实施例提供的朝向信息确定方法，可以基于点云数据中目标障碍物的几何信息确定粗粒度的候选朝向集合，根据图像数据中目标障碍物的表观信息从候选朝向集合中择优选择一个朝向信息作为目标障碍物的朝向信息，相当于综合点云数据中的几何信息和图像数据中的表观信息共同确定目标障碍物的朝向信息，能够更加准确地确定目标障碍物的朝向。
图3是本申请实施例提供的一种朝向信息确定方法的流程图。本申请实施例以执行主体为自动驾驶车辆为例进行示例性说明，该实施例包括：
301、自动驾驶车辆获取针对目标障碍物拍摄的点云数据和图像数据。
在一些实施例中，点云数据和图像数据是同一时刻获取的，保证了点云数据和图像数据中目标障碍物的朝向是相同的。
在一些实施例中，自动驾驶车辆在行驶过程中，从周围环境中识别障碍物，针对任一识别到的障碍物执行本申请实施例所提供的方法步骤。其中，障碍物可以是任一能够移动的物体，这样，确定障碍物的朝向可以辅助确定障碍物下一刻的状态，有利于自动驾驶车辆做出更加精准的决策。
302、自动驾驶车辆对目标障碍物的点云数据进行第一特征提取，得到目标障碍物的几何特征。
本申请实施例中，自动驾驶车辆可以采用任一种特征提取算法对目标障碍物的点云数据进行第一特征提取，得到目标障碍物的几何特征。在一些实施例中，自动驾驶车辆通过点云深度学习模型对目标障碍物的点云数据进行第一特征提取，得到目标障碍物的几何特征。其中，点云深度学习模型是专门用于对点云进行处理的模型。
可选地，该点云深度学习模型包括第一特征提取层，自动驾驶车辆通过第一特征提取层对目标障碍物的点云数据进行第一特征提取，得到目标障碍物的几何特征。
303、自动驾驶车辆基于目标障碍物的几何特征，确定目标障碍物的候选朝向集合，该候选朝向集合包括多个候选朝向信息，该多个候选朝向信息指示目标障碍物多个面在自动驾驶车辆坐标系中的朝向。
本申请实施例中，自动驾驶车辆基于目标障碍物的几何特征，确定目标障碍物的候选朝向集合时，可以基于目标障碍物的几何特征，确定目标障碍物的朝向信息，基于该目标障碍物的朝向信息，扩展出目标障碍物多个面的朝向信息；也可以直接确定目标障碍物多个面的朝向信息。本申请实施例对确定候选朝向集合的方式不做限定。
在第一种可能的实现方式中，自动驾驶车辆基于目标障碍物的几何特征，确定目标障碍物的朝向信息，基于该目标障碍物的朝向信息，扩展出目标障碍物多个面的朝向信息。
可选地，朝向信息包括朝向角度。自动驾驶车辆基于目标障碍物的几何特征，确定目标障碍物的候选朝向集合，包括：自动驾驶车辆基于几何特征，确定目标障碍物的朝向角度；将该朝向角度分别增加90度、180度和270度，得到多个扩充的朝向角度；将该朝向角度和多个扩充的朝向角度确定为候选朝向信息，得到该候选朝向集合。
其中，目标障碍物大多是长方体，由于目标障碍物的朝向角度是目标障碍物正面的朝向角度，因此，在确定目标障碍物的朝向角度之后，可以将该朝向角度分别增加90度、180度和270度，即可得到其他面的朝向角度。
可选地，自动驾驶车辆通过点云深度学习模型对目标障碍物的几何特征进行处理，得到目标障碍物的候选朝向集合。在一些实施例中，自动驾驶车辆基于目标障碍物的几何特征，确定目标障碍物的候选朝向集合，包括：自动驾驶车辆通过点云深度学习模型对目标障碍物的几何特征进行处理，得到目标障碍物的朝向角度；将该朝向角度分别增加90度、180度和270度，得到多个扩充的朝向角度；将该朝向角度和多个扩充的朝向角度确定为候选朝向信息，得到该候选朝向集合。
在一些实施例中，该点云深度学习模型包括朝向确定层，自动驾驶车辆通过点云深度学习模型对目标障碍物的几何特征进行处理，得到目标障碍物的朝向角度，包括：自动驾驶车辆通过朝向确定 层对目标障碍物的几何特征进行处理，得到目标障碍物的朝向角度。
其中，点云深度学习模型可以通过样本数据训练得到，该样本数据包括样本障碍物的样本点云数据和样本朝向信息，通过样本数据对点云深度学习模型进行训练，使得点云深度学习模型对样本点云数据进行处理，得到的朝向信息与样本朝向信息的误差收敛。
在第二种可能的实现方式中，自动驾驶车辆基于目标障碍物的几何特征直接确定目标障碍物多个面的朝向信息。自动驾驶车辆基于目标障碍物的几何特征，确定该目标障碍物的候选朝向集合，包括：自动驾驶车辆基于该几何特征，确定该目标障碍物多个面的朝向信息；将该多个面的朝向信息，确定为目标障碍物的候选朝向集合。
可选地，自动驾驶车辆通过点云深度学习模型对目标障碍物的几何特征进行处理，得到目标障碍物的候选朝向集合。在一些实施例中，自动驾驶车辆基于目标障碍物的几何特征，确定目标障碍物的候选朝向集合，包括：自动驾驶车辆通过点云深度学习模型对目标障碍物的几何特征进行处理，得到该目标障碍物多个面的朝向信息。
在一些实施例中，该点云深度学习模型包括朝向确定层，自动驾驶车辆通过点云深度学习模型对目标障碍物的几何特征进行处理，得到该目标障碍物多个面的朝向信息，包括：自动驾驶车辆通过朝向确定层对目标障碍物的几何特征进行处理，得到该目标障碍物多个面的朝向信息。
其中，点云深度学习模型可以通过样本数据训练得到，该样本数据包括样本障碍物的样本点云数据和该样本障碍物多个面的样本朝向信息，通过样本数据对点云深度学习模型进行训练，使得点云深度学习模型对样本点云数据进行处理，得到的多个面的朝向信息与该多个面的样本朝向信息的误差收敛。
在一种可能的实现方式中，自动驾驶车辆在行驶过程中针对目标障碍物拍摄得到了多帧点云数据，自动驾驶车辆基于多帧点云数据，确定目标障碍物的朝向信息。可选地，为了使得得到的几何特征更加完善，自动驾驶车辆可以融合多帧点云数据中目标障碍物的几何特征。在一些实施例中，点云数据为多帧点云数据，多帧点云数据为自动驾驶车辆在不同时刻拍摄目标障碍物得到的图像数据。自动驾驶车辆对目标障碍物的点云数据进行第一特征提取，得到目标障碍物的几何特征，包括：对目标障碍物的多帧点云数据中每帧点云数据进行第一特征提取，得到每帧点云数据对应的第一几何特征；按照多帧点云数据的拍摄顺序，对每帧点云数据依次执行：将当前帧点云数据的第一几何特征与上一帧点云数据的第一几何特征进行融合，得到当前帧点云数据的第二几何特征。自动驾驶车辆基于目标障碍物的几何特征，确定目标障碍物的候选朝向集合，包括：自动驾驶车辆基于最后一帧点云数据的第二几何特征，确定目标障碍物的候选朝向集合。
304、自动驾驶车辆对目标障碍物的图像数据进行第二特征提取，得到目标障碍物的表观特征；基于目标障碍物的表观特征，确定目标障碍物的第一朝向信息，第一朝向信息指示目标障碍物在观察者坐标系中的朝向。
本申请实施例中，针对目标障碍物的图像数据是自动驾驶车辆拍摄得到的，因此，自动驾驶车辆就是目标障碍物的观察者。观察者坐标系以自动驾驶车辆指向目标障碍物的第一射线为纵轴，以经过目标障碍物中心且垂直于第一射线的第二射线为横轴。
本申请实施例中，自动驾驶车辆可以采用任一种算法对目标障碍物的图像数据进行第二特征提取，得到目标障碍物的表观特征，基于目标障碍物的表观特征，确定目标障碍物的第一朝向信息。本申请实施例对此不做限定。
在一种可能的实现方式中，图像数据为多帧图像数据，多帧图像数据为自动驾驶车辆在不同时刻拍摄目标障碍物得到的图像数据。自动驾驶车辆对目标障碍物的图像数据进行第二特征提取，得到目标障碍物的表观特征，包括：对目标障碍物的多帧图像数据中每帧图像数据进行第二特征提取，得到每帧图像数据对应的第一表观特征；按照多帧图像数据的拍摄顺序，对每帧图像数据依次执行：将当前帧图像数据的第一表观特征与上一帧图像数据的第一表观特征进行融合，得到当前帧图像数据的第二表观特征。自动驾驶车辆基于目标障碍物的表观特征，确定目标障碍物的第一朝向信息，包括：基于最后一帧图像数据的第二表观特征，确定目标障碍物的第一朝向信息。
为了避免一帧图像数据中目标障碍物的表观特征不够丰富，本申请实施例可以参考多帧图像数据中的表观特征来确定目标障碍物的朝向信息，使得确定出的朝向信息更加准确。
在一些实施例中，第一朝向信息为第一朝向类别；目标障碍物的表观特征和第一朝向类别由朝向分类模型确定，该朝向分类模型用于从目标数量个朝向类别中确定与障碍物的表观特征匹配的朝向类别。
其中，目标数量可以是任一数量，本申请实施例对目标数量不做限定。在一些实施例中，目标数量为8。目标数量个朝向类别包括前、右前、右、右后、后、左后、左和左前；或者，该目标数量个朝向类别包括0度、45度、90度、135度、180度、225度、270度和315度。
例如，本申请实施例以小孔成像模型进行建模，以观察者坐标系为基准坐标系，将障碍物朝向分为八个区域：前、右前、右、右后、后、左后、左、左前，分别对应0度、45度、90度、135度、180度、225度、270度、315度。由标注人员按照该标准对样本图像数据进行标注，并积累样本数据，对朝向分类模型进行训练。
在一些实施例中，如图4所示，该朝向分类模型包括特征提取层、融合层和朝向分类层。自动驾驶车辆通过特征提取层，对目标障碍物的多帧图像数据进行特征提取，得到每帧图像数据对应的表观特征；通过融合层，按照多帧图像数据的拍摄顺序，对每帧图像数据依次执行：将当前帧图像数据的第一表观特征与上一帧图像数据的第一表观特征进行融合，得到当前帧图像数据的第二表观特征；通过朝向分类层，对最后一帧图像数据的第二表观特征进行分类，得到目标障碍物的第一朝向类别。
305、自动驾驶车辆确定自动驾驶车辆坐标系与观察者坐标系的夹角。
其中，自动驾驶车辆坐标系是以自动驾驶车辆的中心作为原点，以自动驾驶车辆的纵向为纵轴，以自动驾驶车辆的横向为横轴。观察者坐标系以目标障碍物的中心为原点，以自动驾驶车辆指向目标障碍物的第一射线为纵轴，以经过目标障碍物中心且垂直于第一射线的第二射线为横轴。因此，基于自动驾驶车辆中心点的位置和目标障碍物中心点的位置，可以确定自动驾驶车辆坐标系与观察者坐标系的夹角。该夹角的正弦函数值为自动驾驶车辆与目标障碍物的横向距离和自动驾驶车辆与目标障碍物的距离的比值。自动驾驶车辆与目标障碍物的横向距离是指目标障碍物中心点到自动驾 驶车辆坐标系的纵轴之间的距离。因此，自动驾驶车辆确定自动驾驶车辆坐标系与观察者坐标系的夹角，包括：获取自动驾驶车辆中心点的位置和目标障碍物中心点的位置；基于自动驾驶车辆中心点的位置和目标障碍物中心点的位置，确定自动驾驶车辆中心点和目标障碍物中心点的横向距离，还确定自动驾驶车辆中心点和目标障碍物中心点的距离；获取该距离和该横向距离的比值，基于正弦函数和该比值，确定自动驾驶车辆坐标系与观察者坐标系的夹角。
306、自动驾驶车辆基于该夹角，确定第一朝向信息在自动驾驶车辆坐标系中对应的第四朝向信息，该第四朝向信息指示目标障碍物在自动驾驶车辆坐标系中的朝向。
由于第一朝向信息与候选朝向信息属于不同的坐标系，因此，可以将第一朝向信息映射到自动驾驶车辆坐标系中得到第四朝向信息，这样，基于图像数据确定的第四朝向信息和基于点云数据确定的候选朝向信息就处于同一坐标系了。
307、自动驾驶车辆从候选朝向集合中确定所指示朝向与第四朝向信息所指示朝向匹配的第二朝向信息作为目标障碍物的朝向信息。
其中，所指示朝向与第四朝向信息所指示朝向匹配是指：所指示的朝向与第四朝向信息所指示的朝向相似或者接近。在一些实施例中，自动驾驶车辆从候选朝向集合中确定所指示朝向与第四朝向信息所指示朝向匹配的第二朝向信息作为目标障碍物的朝向信息，包括：从候选朝向集合中确定所指示朝向与第四朝向信息所指示朝向相差最小的第二朝向信息作为目标障碍物的朝向信息。
例如，图5为需要确定朝向的障碍物，确定该障碍物朝向的过程可以如图6所示，基于障碍物的点云数据，确定障碍物在自动驾驶车辆坐标系中的朝向角度为α+β，将该朝向角度进行扩展，得到候选朝向集合{α+β，α+β+90°，α+β+180°，α+β+270°}；基于障碍物的图像数据，确定障碍物在观察者坐标系中的朝向角度为γ；确定自动驾驶车辆坐标系和观察者坐标系的夹角为β，从候选朝向集合中确定与γ+β角度相差最小的朝向角度作为自动驾驶车辆的朝向角度。
需要说明的是，本申请实施例仅是以多个候选朝向信息指示目标障碍物在自动驾驶车辆坐标系中的朝向，第一朝向信息指示目标障碍物在观察者坐标系中的朝向为例进行示例性说明，而在另一实施例中，多个候选朝向信息指示目标障碍物在世界坐标系中的朝向，第一朝向信息指示目标障碍物在世界坐标系中的朝向，本申请实施例对坐标系不做限定。
需要说明的另一点是，本申请实施例提供的方法相对于仅基于点云数据确定朝向信息而言，可以将车辆朝向识别的错误率降低20％，将行人朝向识别的准确率提高50％。
本申请实施例提供的朝向信息确定方法，可以基于点云数据中目标障碍物的几何信息确定粗粒度的候选朝向集合，根据图像数据中目标障碍物的表观信息从候选朝向集合中择优选择一个朝向信息作为目标障碍物的朝向信息，相当于综合点云数据中的几何信息和图像数据中的表观信息共同确定目标障碍物的朝向信息，能够更加准确地确定目标障碍物的朝向。
图7是本申请实施例提供的一种朝向信息确定装置的结构示意图，参见图7，该装置包括：
获取模块701，用于获取自动驾驶车辆针对目标障碍物拍摄的点云数据和图像数据；
特征提取模块702，用于对所述目标障碍物的点云数据进行第一特征提取，得到所述目标障碍物的几何特征，对所述目标障碍物的图像数据进行第二特征提取，得到所述目标障碍物的表观特征；
第一确定模块703，用于基于所述目标障碍物的几何特征，确定所述目标障碍物的候选朝向集合，所述候选朝向集合包括多个候选朝向信息，所述多个候选朝向信息指示所述目标障碍物多个面的朝向；
第二确定模块704，用于基于所述目标障碍物的表观特征，确定所述目标障碍物的第一朝向信息；
第三确定模块705，用于从所述候选朝向集合中确定所指示朝向与所述第一朝向信息所指示朝向匹配的第二朝向信息作为所述目标障碍物的朝向信息。
在一种可能的实现方式中，所述朝向信息包括朝向角度；所述第一确定模块703，用于基于所述几何特征，确定所述目标障碍物的朝向角度；将所述朝向角度分别增加90度、180度和270度，得到多个扩充的朝向角度；将所述朝向角度和所述多个扩充的朝向角度确定为候选朝向信息，得到所述候选朝向集合。
在一种可能的实现方式中，所述第一确定模块703，用于基于所述几何特征，确定所述目标障碍物多个面的朝向信息；将所述多个面的朝向信息，确定为所述目标障碍物的候选朝向集合。
在一种可能的实现方式中，所述图像数据为多帧图像数据，所述多帧图像数据为所述自动驾驶车辆在不同时刻拍摄所述目标障碍物得到的图像数据；
所述特征提取模块702，用于对所述目标障碍物的多帧图像数据中每帧图像数据进行第三特征提取，得到所述每一帧图像数据对应的第一表观特征；按照所述多帧图像数据的拍摄顺序，对每帧图像数据依次执行：将当前帧图像数据的第一表观特征与上一帧图像数据的第一表观特征进行融合，得到所述当前帧图像数据的第二表观特征；
所述第二确定模块704，用于基于最后一帧图像数据的第二表观特征，确定所述目标障碍物的第一朝向信息。
在一种可能的实现方式中，所述第一朝向信息为第一朝向类别；所述目标障碍物的表观特征和所述第一朝向类别由朝向分类模型确定，所述朝向分类模型用于从目标数量个朝向类别中确定与障碍物的表观特征匹配的朝向类别。
在一种可能的实现方式中，所述目标数量个朝向类别包括前、右前、右、右后、后、左后、左和左前；或者，所述目标数量个朝向类别包括0度、45度、90度、135度、180度、225度、270度和315度。
在一种可能的实现方式中，所述多个候选朝向信息指示所述目标障碍物在自动驾驶车辆坐标系中的朝向，所述第一朝向信息指示所述目标障碍物在观察者坐标系中的朝向，所述观察者坐标系以所述自动驾驶车辆指向所述目标障碍物的第一射线为纵轴，以经过所述目标障碍物中心且垂直于所述第一射线的第二射线为横轴。
在一种可能的实现方式中，所述第三确定模块705，用于确定所述自动驾驶车辆坐标系与所述观察者坐标系的夹角；基于所述夹角，确定所述第一朝向信息在所述自动驾驶车辆坐标系中对应的第四朝向信息，所述第四朝向信息指示所述目标障碍物在所述自动驾驶车辆坐标系中的朝向；从所述候选朝向集合中确定所指示朝向与所述第四朝向信息所指示朝向匹配的第二朝向信息作为所述目 标障碍物的朝向信息。
在一种可能的实现方式中，所述第三确定模块705，用于从所述候选朝向集合中确定所指示朝向与所述第四朝向信息所指示朝向相差最小的第二朝向信息作为所述目标障碍物的朝向信息。
需要说明的是：上述实施例提供的朝向信息确定装置在进行标定时，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将自动驾驶车辆的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。另外，上述实施例提供的朝向信息确定装置与朝向信息确定方法实施例属于同一构思，其具体实现过程详见方法实施例，这里不再赘述。
图8示出了本申请一个示例性实施例提供的自动驾驶车辆800的结构框图。自动驾驶车辆800包括有：处理器801和存储器802。
处理器801可以包括一个或多个处理核心，比如4核心处理器、8核心处理器等。处理器801可以采用DSP(Digital Signal Processing，数字信号处理)、FPGA(Field－Programmable Gate Array，现场可编程门阵列)、PLA(Programmable Logic Array，可编程逻辑阵列)中的至少一种硬件形式来实现。处理器801也可以包括主处理器和协处理器，主处理器是用于对在唤醒状态下的数据进行处理的处理器，也称CPU(Central Processing Unit，中央处理器)；协处理器是用于对在待机状态下的数据进行处理的低功耗处理器。在一些实施例中，处理器801可以在集成有GPU(Graphics Processing Unit，图像处理器)，GPU用于负责显示屏所需要显示的内容的渲染和绘制。一些实施例中，处理器801还可以包括AI(Artificial Intelligence，人工智能)处理器，该AI处理器用于处理有关机器学习的计算操作。
存储器802可以包括一个或多个计算机可读存储介质，该计算机可读存储介质可以是非暂态的。存储器802还可包括高速随机存取存储器，以及非易失性存储器，比如一个或多个磁盘存储设备、闪存存储设备。在一些实施例中，存储器802中的非暂态的计算机可读存储介质用于存储至少一个程序代码，该至少一个程序代码用于被处理器801所执行以实现本申请中方法实施例提供的朝向信息确定方法。
在一些实施例中，自动驾驶车辆800还可选包括有：外围设备接口803和至少一个外围设备。处理器801、存储器802和外围设备接口803之间可以通过总线或信号线相连。各个外围设备可以通过总线、信号线或电路板与外围设备接口803相连。具体地，外围设备包括：射频电路804、显示屏805、摄像头组件806、音频电路807、定位组件808和电源809中的至少一种。
外围设备接口803可被用于将I/O(Input/Output，输入/输出)相关的至少一个外围设备连接到处理器801和存储器802。在一些实施例中，处理器801、存储器802和外围设备接口803被集成在同一芯片或电路板上；在一些其他实施例中，处理器801、存储器802和外围设备接口803中的任意一个或两个可以在单独的芯片或电路板上实现，本实施例对此不加以限定。
射频电路804用于接收和发射RF(Radio Frequency，射频)信号，也称电磁信号。射频电路804通过电磁信号与通信网络以及其他通信设备进行通信。射频电路804将电信号转换为电磁信号进行发送，或者，将接收到的电磁信号转换为电信号。可选地，射频电路804包括：天线系统、RF收发 器、一个或多个放大器、调谐器、振荡器、数字信号处理器、编解码芯片组、用户身份模块卡等等。射频电路804可以通过至少一种无线通信协议来与其它自动驾驶车辆进行通信。该无线通信协议包括但不限于：城域网、各代移动通信网络(2G、3G、4G及5G)、无线局域网和/或WiFi(Wireless Fidelity，无线保真)网络。在一些实施例中，射频电路804还可以包括NFC(Near Field Communication，近距离无线通信)有关的电路，本申请对此不加以限定。
显示屏805用于显示UI(User Interface，用户界面)。该UI可以包括图形、文本、图标、视频及其它们的任意组合。当显示屏805是触摸显示屏时，显示屏805还具有采集在显示屏805的表面或表面上方的触摸信号的能力。该触摸信号可以作为控制信号输入至处理器801进行处理。此时，显示屏805还可以用于提供虚拟按钮和/或虚拟键盘，也称软按钮和/或软键盘。在一些实施例中，显示屏805可以为一个，设置自动驾驶车辆800的前面板；在另一些实施例中，显示屏805可以为至少两个，分别设置在自动驾驶车辆800的不同表面或呈折叠设计；在再一些实施例中，显示屏805可以是柔性显示屏，设置在自动驾驶车辆800的弯曲表面上或折叠面上。甚至，显示屏805还可以设置成非矩形的不规则图形，也即异形屏。显示屏805可以采用LCD(Liquid Crystal Display，液晶显示屏)、OLED(Organic Light-Emitting Diode,有机发光二极管)等材质制备。
摄像头组件806用于采集图像或视频。可选地，摄像头组件806包括前置摄像头和后置摄像头。前置摄像头设置在自动驾驶车辆的前面板，后置摄像头设置在自动驾驶车辆的背面。在一些实施例中，后置摄像头为至少两个，分别为主摄像头、景深摄像头、广角摄像头、长焦摄像头中的任意一种，以实现主摄像头和景深摄像头融合实现背景虚化功能、主摄像头和广角摄像头融合实现全景拍摄以及VR(Virtual Reality，虚拟现实)拍摄功能或者其它融合拍摄功能。在一些实施例中，摄像头组件806还可以包括闪光灯。闪光灯可以是单色温闪光灯，也可以是双色温闪光灯。双色温闪光灯是指暖光闪光灯和冷光闪光灯的组合，可以用于不同色温下的光线补偿。
音频电路807可以包括麦克风和扬声器。麦克风用于采集用户及环境的声波，并将声波转换为电信号输入至处理器801进行处理，或者输入至射频电路804以实现语音通信。出于立体声采集或降噪的目的，麦克风可以为多个，分别设置在自动驾驶车辆800的不同部位。麦克风还可以是阵列麦克风或全向采集型麦克风。扬声器则用于将来自处理器801或射频电路804的电信号转换为声波。扬声器可以是传统的薄膜扬声器，也可以是压电陶瓷扬声器。当扬声器是压电陶瓷扬声器时，不仅可以将电信号转换为人类可听见的声波，也可以将电信号转换为人类听不见的声波以进行测距等用途。在一些实施例中，音频电路807还可以包括耳机插孔。
定位组件808用于定位自动驾驶车辆800的当前地理位置，以实现导航或LBS(Location Based Service，基于位置的服务)。定位组件808可以是基于美国的GPS(Global Positioning System，全球定位系统)、中国的北斗系统、俄罗斯的格雷纳斯系统或欧盟的伽利略系统的定位组件。
电源809用于为自动驾驶车辆800中的各个组件进行供电。电源809可以是交流电、直流电、一次性电池或可充电电池。当电源809包括可充电电池时，该可充电电池可以支持有线充电或无线充电。该可充电电池还可以用于支持快充技术。
在一些实施例中，自动驾驶车辆800还包括有一个或多个传感器810。该一个或多个传感器810 包括但不限于：加速度传感器811、陀螺仪传感器812、压力传感器813、指纹传感器814、光学传感器815以及接近传感器816。
加速度传感器811可以检测以自动驾驶车辆800建立的坐标系的三个坐标轴上的加速度大小。比如，加速度传感器811可以用于检测重力加速度在三个坐标轴上的分量。处理器801可以根据加速度传感器811采集的重力加速度信号，控制显示屏805以横向视图或纵向视图进行用户界面的显示。加速度传感器811还可以用于游戏或者用户的运动数据的采集。
陀螺仪传感器812可以检测自动驾驶车辆800的机体方向及转动角度，陀螺仪传感器812可以与加速度传感器811协同采集用户对自动驾驶车辆800的3D动作。处理器801根据陀螺仪传感器812采集的数据，可以实现如下功能：动作感应(比如根据用户的倾斜操作来改变UI)、拍摄时的图像稳定、游戏控制以及惯性导航。
压力传感器813可以设置在自动驾驶车辆800的侧边框和/或显示屏805的下层。当压力传感器813设置在自动驾驶车辆800的侧边框时，可以检测用户对自动驾驶车辆800的握持信号，由处理器801根据压力传感器813采集的握持信号进行左右手识别或快捷操作。当压力传感器813设置在显示屏805的下层时，由处理器801根据用户对显示屏805的压力操作，实现对UI界面上的可操作性控件进行控制。可操作性控件包括按钮控件、滚动条控件、图标控件、菜单控件中的至少一种。
指纹传感器814用于采集用户的指纹，由处理器801根据指纹传感器814采集到的指纹识别用户的身份，或者，由指纹传感器814根据采集到的指纹识别用户的身份。在识别出用户的身份为可信身份时，由处理器801授权该用户执行相关的敏感操作，该敏感操作包括解锁屏幕、查看加密信息、下载软件、支付及更改设置等。指纹传感器814可以被设置自动驾驶车辆800的正面、背面或侧面。当自动驾驶车辆800上设置有物理按键或厂商Logo时，指纹传感器814可以与物理按键或厂商Logo集成在一起。
光学传感器815用于采集环境光强度。在一个实施例中，处理器801可以根据光学传感器815采集的环境光强度，控制显示屏805的显示亮度。具体地，当环境光强度较高时，调高显示屏805的显示亮度；当环境光强度较低时，调低显示屏805的显示亮度。在另一个实施例中，处理器801还可以根据光学传感器815采集的环境光强度，动态调整摄像头组件806的拍摄参数。
接近传感器816，也称距离传感器，设置在自动驾驶车辆800的前面板。接近传感器816用于采集用户与自动驾驶车辆800的正面之间的距离。在一个实施例中，当接近传感器816检测到用户与自动驾驶车辆800的正面之间的距离逐渐变小时，由处理器801控制显示屏805从亮屏状态切换为息屏状态；当接近传感器816检测到用户与自动驾驶车辆800的正面之间的距离逐渐变大时，由处理器801控制显示屏805从息屏状态切换为亮屏状态。
本领域技术人员可以理解，图8中示出的结构并不构成对自动驾驶车辆800的限定，可以包括比图示更多或更少的组件，或者组合某些组件，或者采用不同的组件布置。
图9是本申请实施例提供的一种服务器的结构示意图，该服务器900可因配置或性能不同而产生比较大的差异，可以包括一个或一个以上处理器901，例如CPU(Central Processing Units，中央处理单元)和一个或一个以上的存储器902，其中，该存储器902中存储有至少一条程序代码，该 至少一条程序代码由该处理器901加载并执行以实现上述各个方法实施例提供的方法。当然，该服务器还可以具有有线或无线网络接口、键盘以及输入输出接口等部件，以便进行输入输出，该服务器还可以包括其他用于实现设备功能的部件，在此不做赘述。
该服务器900用于执行上述方法实施例中服务器所执行的步骤。
在示例性实施例中，还提供了一种计算机可读存储介质，例如包括程序代码的存储器，上述程序代码可由计算机设备中的处理器执行以完成上述实施例中的朝向信息确定方法。例如，该计算机可读存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。
在示例性实施例中，还提供了一种计算机程序或计算机程序产品，该计算机程序或计算机程序产品包括计算机程序代码，该计算机程序代码被计算机执行时，使得计算机实现上述实施例中的朝向信息确定方法。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成，也可以通过程序来指令相关的硬件完成，该程序可以存储于一种计算机可读存储介质中，上述提到的存储介质可以是只读存储器，磁盘或光盘等。
以上所述仅为本申请的可选实施例，并不用以限制本申请，凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (13)

  1. 一种朝向信息确定方法，其中，所述方法包括：
    获取自动驾驶车辆针对目标障碍物拍摄的点云数据和图像数据；
    对所述目标障碍物的点云数据进行第一特征提取，得到所述目标障碍物的几何特征，对所述目标障碍物的图像数据进行第二特征提取，得到所述目标障碍物的表观特征；
    基于所述目标障碍物的几何特征，确定所述目标障碍物的候选朝向集合，所述候选朝向集合包括多个候选朝向信息，所述多个候选朝向信息指示所述目标障碍物多个面的朝向；
    基于所述目标障碍物的表观特征，确定所述目标障碍物的第一朝向信息；
    从所述候选朝向集合中确定所指示朝向与所述第一朝向信息所指示朝向匹配的第二朝向信息作为所述目标障碍物的朝向信息。
  2. 根据权利要求1所述的方法，其中，所述朝向信息包括朝向角度；所述基于所述目标障碍物的几何特征，确定所述目标障碍物的候选朝向集合，包括：
    基于所述几何特征，确定所述目标障碍物的朝向角度；
    将所述朝向角度分别增加90度、180度和270度，得到多个扩充的朝向角度；
    将所述朝向角度和所述多个扩充的朝向角度确定为候选朝向信息，得到所述候选朝向集合。
  3. 根据权利要求1所述的方法，其中，所述基于所述目标障碍物的几何特征，确定所述目标障碍物的候选朝向集合，包括：
    基于所述几何特征，确定所述目标障碍物多个面的朝向信息；
    将所述多个面的朝向信息，确定为所述目标障碍物的候选朝向集合。
  4. 根据权利要求1所述的方法，其中，所述图像数据为多帧图像数据，所述多帧图像数据为所述自动驾驶车辆在不同时刻拍摄所述目标障碍物得到的图像数据；所述对所述目标障碍物的图像数据进行第二特征提取，得到所述目标障碍物的表观特征，包括：
    对所述目标障碍物的多帧图像数据中每帧图像数据进行第二特征提取，得到所述每帧图像数据对应的第一表观特征；
    按照所述多帧图像数据的拍摄顺序，对所述每帧图像数据依次执行：将当前帧图像数据的第一表观特征与上一帧图像数据的第一表观特征进行融合，得到所述当前帧图像数据的第二表观特征；
    所述基于所述目标障碍物的表观特征，确定所述目标障碍物的第一朝向信息，包括：
    基于最后一帧图像数据的第二表观特征，确定所述目标障碍物的第一朝向信息。
  5. 根据权利要求1或4所述的方法，其中，所述第一朝向信息为第一朝向类别；所述目标障碍物的表观特征和所述第一朝向类别由朝向分类模型确定，所述朝向分类模型用于从目标数量个朝向类别中确定与障碍物的表观特征匹配的朝向类别。
  6. 根据权利要求5所述的方法，其中，所述目标数量个朝向类别包括前、右前、右、右后、后、左后、左和左前；或者，所述目标数量个朝向类别包括0度、45度、90度、135度、180度、225度、270度和315度。
  7. 根据权利要求1所述的方法，其中，所述多个候选朝向信息指示所述目标障碍物在自动驾驶车辆坐标系中的朝向，所述第一朝向信息指示所述目标障碍物在观察者坐标系中的朝向，所述观察者坐标系以所述自动驾驶车辆指向所述目标障碍物的第一射线为纵轴，以经过所述目标障碍物中心且垂直于所述第一射线的第二射线为横轴，所述自动驾驶车辆坐标系是以所述自动驾驶车辆的中心作为原点，以所述自动驾驶车辆的纵向为纵轴，以所述自动驾驶车辆的横向为横轴。
  8. 根据权利要求7所述的方法，其中，所述从所述候选朝向集合中确定所指示朝向与所述第一朝向信息所指示朝向匹配的第二朝向信息作为所述目标障碍物的朝向信息，包括：
    确定所述自动驾驶车辆坐标系与所述观察者坐标系的夹角；
    基于所述夹角，确定所述第一朝向信息在所述自动驾驶车辆坐标系中对应的第四朝向信息，所述第四朝向信息指示所述目标障碍物在所述自动驾驶车辆坐标系中的朝向；
    从所述候选朝向集合中确定所指示朝向与所述第四朝向信息所指示朝向匹配的第二朝向信息作为所述目标障碍物的朝向信息。
  9. 根据权利要求8所述的方法，其中，所述从所述候选朝向集合中确定所指示朝向与所述第四朝向信息所指示朝向匹配的第二朝向信息作为所述目标障碍物的朝向信息，包括：
    从所述候选朝向集合中确定所指示朝向与所述第四朝向信息所指示朝向相差最小的第二朝向信息作为所述目标障碍物的朝向信息。
  10. 根据权利要求1所述的方法，其中，所述点云数据为多帧点云数据，所述多帧点云数据为所述自动驾驶车辆在不同时刻拍摄所述目标障碍物得到的图像数据，
    对所述目标障碍物的点云数据进行第一特征提取，得到所述目标障碍物的几何特征，包括：
    对所述目标障碍物的多帧点云数据中每帧点云数据进行第一特征提取，得到所述每帧点云数据对应的第一几何特征；
    按照所述多帧点云数据的拍摄顺序，对所述每帧点云数据依次执行：将当前帧点云数据的第一几何特征与上一帧点云数据的第一几何特征进行融合，得到当前帧点云数据的第二几何特征；
    基于所述目标障碍物的几何特征，确定所述目标障碍物的候选朝向集合，包括：
    基于最后一帧点云数据的第二几何特征，确定所述目标障碍物的候选朝向集合。
  11. 一种自动驾驶车辆，其中，所述自动驾驶车辆包括一个或多个处理器和一个或多个存储器，所述一个或多个存储器中存储有至少一条程序代码，所述至少一条程序代码由所述一个或多个处理器加载并执行以实现如权利要求1至10任一项所述的朝向信息确定方法所执行的操作。
  12. 一种计算机可读存储介质，其中，所述存储介质中存储有至少一条程序代码，所述至少一条程序代码由处理器加载并执行以实现如权利要求1-10任一项所述的方法所执行的操作。
  13. 一种计算机程序或计算机程序产品，其中，所述计算机程序或计算机程序产品包括：计算机程序代码，所述计算机程序代码被计算机执行时，使得所述计算机实现如权利要求1-10任一项所述的方法所执行的操作。
PCT/CN2023/080380 2022-10-26 2023-03-09 确定朝向信息以及自动驾驶车辆 WO2024087456A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211321542.XA CN117975404B (zh) 2022-10-26 2022-10-26 朝向信息确定方法以及自动驾驶车辆
CN202211321542.X 2022-10-26

Publications (1)

Publication Number Publication Date
WO2024087456A1 true WO2024087456A1 (zh) 2024-05-02

Family

ID=90829855

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/080380 WO2024087456A1 (zh) 2022-10-26 2023-03-09 确定朝向信息以及自动驾驶车辆

Country Status (2)

Country Link
CN (1) CN117975404B (zh)
WO (1) WO2024087456A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109116374A (zh) * 2017-06-23 2019-01-01 百度在线网络技术（北京）有限公司 确定障碍物距离的方法、装置、设备及存储介质
CN111401208A (zh) * 2020-03-11 2020-07-10 北京百度网讯科技有限公司 一种障碍物检测方法、装置、电子设备及存储介质
CN111753765A (zh) * 2020-06-29 2020-10-09 北京百度网讯科技有限公司 感知设备的检测方法、装置、设备及存储介质
CN114973193A (zh) * 2022-05-18 2022-08-30 广州小马慧行科技有限公司 识别车辆与障碍物距离的方法、装置、设备和存储介质
CN115019511A (zh) * 2022-06-29 2022-09-06 九识(苏州)智能科技有限公司 基于自动驾驶车辆的识别机动车违规变道的方法和装置

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4052496B2 (ja) * 1998-08-07 2008-02-27 富士通テン株式会社 車両用障害物検知システム
CN109901567B (zh) * 2017-12-08 2020-06-23 百度在线网络技术（北京）有限公司 用于输出障碍物信息的方法和装置
CN111028330B (zh) * 2019-11-15 2023-04-07 腾讯科技（深圳）有限公司 三维表情基的生成方法、装置、设备及存储介质
CN111192327B (zh) * 2020-01-03 2023-09-29 北京百度网讯科技有限公司 用于确定障碍物朝向的方法和装置
CN111324115B (zh) * 2020-01-23 2023-09-19 北京百度网讯科技有限公司 障碍物位置检测融合方法、装置、电子设备和存储介质
CN111812649A (zh) * 2020-07-15 2020-10-23 西北工业大学 基于单目相机与毫米波雷达融合的障碍物识别与定位方法
CN114140760A (zh) * 2020-08-13 2022-03-04 长沙智能驾驶研究院有限公司 智能驾驶中障碍物检测去干扰方法、装置和计算机设备
CN111736616A (zh) * 2020-08-27 2020-10-02 北京奇虎科技有限公司 扫地机器人的避障方法、装置、扫地机器人及可读介质
CN113985405A (zh) * 2021-09-16 2022-01-28 森思泰克河北科技有限公司 障碍物检测方法、应用于车辆的障碍物检测设备
CN115100426B (zh) * 2022-06-23 2024-05-24 高德软件有限公司 信息确定方法、装置、电子设备及计算机程序产品

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109116374A (zh) * 2017-06-23 2019-01-01 百度在线网络技术（北京）有限公司 确定障碍物距离的方法、装置、设备及存储介质
CN111401208A (zh) * 2020-03-11 2020-07-10 北京百度网讯科技有限公司 一种障碍物检测方法、装置、电子设备及存储介质
CN111753765A (zh) * 2020-06-29 2020-10-09 北京百度网讯科技有限公司 感知设备的检测方法、装置、设备及存储介质
CN114973193A (zh) * 2022-05-18 2022-08-30 广州小马慧行科技有限公司 识别车辆与障碍物距离的方法、装置、设备和存储介质
CN115019511A (zh) * 2022-06-29 2022-09-06 九识(苏州)智能科技有限公司 基于自动驾驶车辆的识别机动车违规变道的方法和装置

Also Published As

Publication number Publication date
CN117975404A (zh) 2024-05-03
CN117975404B (zh) 2024-10-25

Similar Documents

Publication Publication Date Title
US11221726B2 (en) Marker point location display method, electronic device, and computer-readable storage medium
EP3779883B1 (en) Method and device for repositioning in camera orientation tracking process, and storage medium
CN111126182B (zh) 车道线检测方法、装置、电子设备及存储介质
WO2020108647A1 (zh) 车载摄像头和车载雷达联动的目标检测方法、装置及系统
WO2021128777A1 (en) Method, apparatus, device, and storage medium for detecting travelable region
CN110148178B (zh) 相机定位方法、装置、终端及存储介质
WO2022213733A1 (zh) 获取飞行航线的方法、装置、计算机设备及可读存储介质
CN113205515B (zh) 目标检测方法、装置、及计算机存储介质
CN113343457B (zh) 自动驾驶的仿真测试方法、装置、设备及存储介质
CN112991439B (zh) 定位目标物体的方法、装置、电子设备及介质
CN111538009B (zh) 雷达点的标记方法和装置
CN111325701B (zh) 图像处理方法、装置及存储介质
CN111862148A (zh) 实现视觉跟踪的方法、装置、电子设备及介质
CN111179628B (zh) 自动驾驶车辆的定位方法、装置、电子设备及存储介质
CN113255906A (zh) 一种自动驾驶中回归障碍物3d角度信息方法、装置、终端及存储介质
CN111754564B (zh) 视频展示方法、装置、设备及存储介质
WO2024087456A1 (zh) 确定朝向信息以及自动驾驶车辆
CN112734346B (zh) 航线覆盖范围的确定方法、装置、设备及可读存储介质
CN117911520A (zh) 相机内参标定方法和自动驾驶设备
CN112817337B (zh) 获取路径的方法、装置、设备和可读存储介质
CN118135255A (zh) 图像匹配模型的训练方法、图像匹配的方法及计算机设备
CN115965936A (zh) 边缘位置标注方法及设备
CN113033590B (zh) 图像特征匹配方法、装置、图像处理设备及存储介质
WO2019233299A1 (zh) 地图构建方法、装置及计算机可读存储介质
CN111583339A (zh) 获取目标位置的方法、装置、电子设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23881059

Country of ref document: EP

Kind code of ref document: A1