JP2023126893A - Method for creating universally usable feature map - Google Patents

Abstract

The present invention provides a method for generating a digital map that can be used for general purposes while reducing data consumption. A method for generating a digital map by a control device, which receives environmental measurement data during a measurement run, executes SLAM based on the received measurement data to identify the trajectory of the measurement run, and A method is disclosed in which measured data is converted into a trajectory coordinate system, the converted measured data is used to generate an intensity map, and features are extracted from the intensity map and stored in the feature map. Additionally, a method, controller, computer program product, and machine-readable storage medium for performing self-localization is disclosed. [Selection diagram] Figure 1

Description

The present invention relates to a method of generating a digital map and a method of performing self-localization. Furthermore, the present invention relates to a control device, a computer program, and a machine-readable recording medium.

Localization is an essential functional element for autonomous driving of cars and robots. Self-localization allows vehicles and robots to know their exact location on a map or in the environment. Based on the identified location, control commands can be generated to, for example, travel on a track or perform a task.

Particularly in applications that do not access GNSS data, self-location and mapping (map generation) can be performed simultaneously using so-called SLAM. For this purpose, measurement data, for example from LIDAR sensors, are collected and analyzed in order to generate a map. In a subsequent step, the location within the map can be determined.

The problem with SLAM is its application in dynamic or semi-static environments. Such environments may exist, for example, in storage areas, construction sites, intralogistics, or container ports. Due to regular movement of objects, the generated maps lose their validity after a short time. Periodic updating of such maps requires a great deal of effort in measurement and analysis. In particular, regularly updated maps need to be available to all users, which requires an infrastructure to provide large amounts of data.

It is believed that the underlying problem of the present invention is to propose a method for generating a universally usable digital map with reduced data consumption.

This problem is solved by the subject matter of the independent claims. Advantageous embodiments of the invention are the subject of the respective dependent claims.

According to one aspect of the invention, a method is provided for generating a digital map by a controller. One step is to receive environmental measurement data during a measurement drive. Here, the measurement run may be any run. Preferably, measurement data can also be collected by at least one sensor when the vehicle is stationary or parked. Subsequently, the corresponding measurement data can be received and processed by the control device.

Based on the received measurement data, SLAM is performed to determine the trajectory of the measurement trip. Here, self-localization is performed based on a series of measurement data, and each position during the measurement run forms a trajectory.

A further step is to transform the received measurement data into the coordinate system of the orbit. The received measurement data may include, for example, a relative position and/or a relative distance to the sensor. This relative coordinate can then be transformed into the absolute coordinate system of the trajectory, for example. Such a coordinate system may be, for example, a Cartesian (orthogonal) coordinate system.

The transformed measurement data is used to generate an intensity map. For example, the intensity of reflected light from one or more LIDAR or radar sensors may be measured and stored in the form of a map of received light intensity.

Next, features are extracted from the intensity map and saved in a feature map. The features can preferably be detected from the intensity map. This process can be performed, for example, by a pattern recognition algorithm. Pattern recognition may be performed by a neural network that has been appropriately trained in advance. Pattern recognition may be performed manually, for example by a person performing the processing, or may be performed automatically. Furthermore, the automatically performed pattern recognition may be canceled or confirmed by the person performing the processing.

The pattern map is preferably universally usable. In particular, since the pattern map can be used sensor-independent or sensor-dominantly, features can be extracted from different measured measurement data and used for self-localization based on the feature map.

According to a further aspect of the invention, a method is provided for performing self-localization, particularly in a control device. One step is to receive measurement data and feature maps of the environment. Measurement data may be measured by one or more sensors. Such a sensor may be, for example, a camera sensor, a lidar sensor, a radar sensor, an ultrasonic sensor, or the like. In particular, the sensor may be different from the sensor used to generate the feature map.

A further step is to recognize and extract features of the received measurement data. Next, the position is determined by comparing the extracted at least one feature with the feature stored in the feature map. When the at least one characteristic is successfully matched, the sensor or the position of the vehicle from which the sensor is to be measured can be determined.

According to a further aspect of the invention, a controller is provided, the controller being configured to implement the method described above. Here, the control device may be an in-vehicle control device built into a vehicle control device for executing an automatic driving function, or a device connectable to the vehicle control device. Alternatively or additionally, the control device may be configured as a control device external to the vehicle, such as, for example, a server device or cloud technology.

Furthermore, according to one aspect of the invention, there is provided a computer program comprising a plurality of instructions which, when executed by a computer or a control device, cause the computer or control device to carry out the method according to the invention. According to a further aspect of the invention, there is provided a machine-readable storage medium storing a computer program according to the invention.

In this case, the control device may be built into the vehicle. In particular, at least one measuring drive can be carried out in a vehicle equipped with a control device. At this time, the vehicle can operate in assisted, partially automated, highly automated, and/or fully automated mode, or without a driver, based on the BASt standard. According to alternative or additional forms, the vehicle may be a robot, a drone, a ship, etc. This allows the method to be used on roads such as highways, country roads, urban areas, and off-road. The method can be used in particular in buildings or halls, underground spaces, parking lots and garages, tunnels, etc.

The at least one sensor for measuring the measurement data may be a component of a surrounding environment sensor system or at least one sensor of a vehicle. In particular, the at least one sensor may be a lidar sensor, a radar sensor, an ultrasound sensor, a camera sensor, an odometer, an acceleration sensor, a position sensor, etc. In particular, the sensors may be used alone or in combination with each other. Furthermore, sensors such as acceleration sensors, wheel sensors, lidar sensors, ultrasonic distance sensors, cameras, etc. may be used to perform the odometric method.

The method according to the invention allows in particular static features of the environment to be identified and extracted. Such features may be, for example, road signs, building geometry, curbs, roads, traffic light placement and position, delineators, lane boundaries, buildings, containers, etc. Such features can be detected by multiple different sensors and used for self-localization. For example, features extracted from measurement data of a lidar sensor may also be detected by a camera sensor and compared with each other for self-localization. Therefore, it is possible to generate a universally usable feature map that can be used in different vehicles and machines. For example, such feature maps can be utilized for accurate self-localization by personal mobility, transport units, manipulators, etc.

Preferably, a marking map is generated in a first step and can subsequently be used for the task of self-localization. In particular, marking maps can be used for self-localization and control tasks in autonomous vehicles and robots.

Since the features of the feature map can be geometric figures, straight lines, points, etc., they can be stored in the form of coordinates or vectors with a minimum data size. This makes it possible to reduce the amount of data required to provide feature maps to vehicles and robots.

According to embodiments, the feature map is saved as a digital map or as a map layer of a digital map. This allows the feature map to be used particularly flexibly. In particular, existing maps can be upgraded with feature maps or implemented as digital maps with minimal storage requirements.

According to a further embodiment, the received measurement data are present as a point cloud and are assigned to a raster of cells. Preferably, the average value of the measurement data of each cell is calculated to generate the intensity map. A cell of the digital map may be, for example, a pixel, a group of pixels, a polygon, or the like. By calculating the average value, it is possible to compensate for local discrepancies and fluctuations in the measured values. The measurements may in particular be constituted by reflected beams from radar and/or lidar sensors or by beams detected after backscatter.

According to a further embodiment, an altitude map is generated from the received measurement data, and a weighted average value is calculated from the measurement data of each cell and neighboring cells to generate the altitude map. Thereby, additional information can be extracted from the measured measurement data and used to generate a feature map.

According to a further embodiment, information is received from the generated altitude map and stored in the feature map to determine the altitude of the extracted features. Here, the altitude map and the feature map can be superimposed and the corresponding attributes and information of the altitude map can be transmitted to the feature map. In addition, for example, the height or intensity gradient at the feature position can be inherited from each feature. This process is preferably performed automatically, matching each cell of the elevation map with each cell of the feature map.

According to a further embodiment, the extracted features are stored in a feature map as universally identifiable features. According to an advantageous embodiment, the features are extracted and stored as geometric shapes, lines, points and/or point clouds, etc. In this way, objects, landmarks, shapes with distinctive shapes or characteristics can be extracted and used from the environmental measurement data to perform self-localization. In particular, this allows the identification of large numbers of static features even in dynamic environments and can be used to accurately determine location. Here, since the features can basically be identified without depending on the sensor, the marking map can be used for general purposes.

According to a further embodiment, the received measurement data is realized as position data and stored in a position diagram. Preferably, the identified location is stored as a new measurement in the location diagram when the feature is successfully matched. Feature maps can be used, for example, to locate vehicles or robots. At this time, each current position is specified along the route, for example, at predetermined time intervals, and stored in a position diagram or a position map. Position diagrams allow you to display distance traveled and trajectories. If the feature map has a feature, a position in the feature map can be assigned to the vehicle or sensor that measured the measurement data. This position is then saved as a separate measurement in the position diagram.

According to a further embodiment, the measurement data is measured from at least one sensor different from the at least one sensor used to generate the feature map. The extracted features preferably exist in an abstracted form and may thus be universally readable or comparable. The form of such features may be present, for example, as coordinates in text form. In particular, features within the coordinates may include starting points, ending points, intermediate points, directions, lengths, heights, etc. This information can be stored with particularly low storage requirements and used for verification purposes.

In the following, preferred embodiments of the invention will be explained in detail using highly simplified schematic diagrams.

FIG. 2 is a schematic diagram of an arrangement for explaining the method according to the present invention. FIG. 2 is a schematic diagram for explaining a method of generating a digital map according to an embodiment. FIG. 2 is a schematic diagram for explaining a method of performing self-location according to an embodiment. It is a figure showing a typical intensity map. It is a figure showing a typical altitude map. FIG. 3 is a perspective view of a feature map.

FIG. 1 is a schematic diagram of an arrangement 1 for explaining methods 2, 4 according to the invention.

Arrangement 1 has two vehicles 6,8. Alternatively or additionally, the arrangement 1 may have robots and/or further vehicles. According to the illustrated embodiment, the first vehicle 6 serves to carry out the method 2 for generating a digital map, in particular a marking map. The second vehicle 8 is shown schematically to clarify the method 4 of performing self-localization within the digital map.

The first vehicle 6 has a control device 10 connected in data communication with a machine-readable memory 12 and a sensor 14 . The sensor 14 may be a LIDAR sensor 14, for example.

The lidar sensor 14 allows the first vehicle 6 to scan the environment U and generate measurement data. Subsequently, the measured measurement data can be received and analyzed by the control device 10. The feature map generated by the control device 10 can be provided to other road users or vehicles 8 via the communication link 16 . The feature map may be stored on a machine-readable storage medium 12.

The second vehicle 8 likewise has a control device 11 . The control device 11 is connected to a machine-readable storage medium 13 and a sensor 15 for data communication. According to this embodiment, the sensor 15 is a camera sensor 15, which can similarly measure measurement data of the environment U and transmit it to the control device 11. The control device 11 can extract features from the measurement data of the environment U and match them with features from a feature map received from the control device 11 via the communication link 16.

FIG. 2 is a schematic diagram for explaining method 2 for generating a digital map according to the embodiment.

In a first step 18 , measurement data of the environment U are measured during a measurement drive of the first vehicle 6 and are received by the control device 10 . According to the embodiment, the environment U is scanned by the lidar sensor 14.

In the following step 19, SLAM is performed using the measurement data received during the measurement run. SLAM determines the trajectory of the first vehicle 6.

The received measurement data is transformed into an orbital coordinate system (20). Alternatively, the trajectory may be transformed into the coordinate system of the measured data. For example, the common coordinate system may be a Cartesian (orthogonal) coordinate system.

An intensity map 30 is generated using the converted measurement data (21). Such an intensity map 30 is shown in FIG. In particular, the measurement data may be present as a grid map with a plurality of cells 31, 32. The cells 31 and 32 may be configured, for example, as pixels or as a group of pixels. Each cell 31, 32 may have a local assignment, for example GPS coordinates, corresponding to a coordinate system.

Subsequently, the intensity is calculated for each cell 31, 32. For this purpose, an average value is calculated for all measured values in each cell 31, 32. An intensity map 30 is generated from the average value thus calculated (21).

Furthermore, an altitude map 40 is generated (22). An altitude map 40 is generated from the weighted average values and is shown in FIG. A weighted average is calculated for the measured values within each cell 31 and the measured data of the corresponding neighboring cell 32.

A further step 23 extracts features from the intensity map 30. This can be done, for example, by an automated pattern recognition algorithm or manually by an operator. For example, transitions between bright and dark areas in the intensity map 30 are considered as possible patterns. Based on the elevation map 40, each feature can be assigned a profile.

The identified features are stored in feature map 60 according to their location within intensity map 30 (24). The feature map 60 is schematically shown in FIG. Here, a plurality of features 62, 64, 66 are overlaid with an exemplary lidar scan. Features 62, 64, 66 are exemplary roadway markings 62, roadway boundaries 64, and other markings 66 on the ground. Feature map 60 may be stored on machine-readable storage medium 12 and provided via communication link 16, for example.

FIG. 3 is a schematic diagram for explaining a method 4 for performing self-localization according to an embodiment. Method 4 is exemplarily carried out by the control device 11 of the second vehicle 8.

In step 25 , measurement data of the environment U is measured by the sensor 15 and transmitted to the control device 11 . Furthermore, a feature map 60 is received by the control device 11 via the communication link 16 . This can be implemented by means of a position diagram locator (position measuring device) implemented in the control device 11.

At this time, the measurement data can be acquired continuously or at predetermined time intervals and received by the control device 11. Furthermore, the control device 11 may receive odometric (integrated) measurement data.

A further step 26 is to extract features 62, 64, 66 from the received measurement data. At this time, the features 62, 64, 66 are compared with the received feature map 60 (27). During matching, an attempt is made to locate the features 62, 64, 66 detected within the vehicle on the feature map 60. At this time, the measurement data measured by odometry (integration) can limit the search area within the feature map 60. Since the feature map 60 has abstracted and thus universally usable features 62, 64, 66, measurement data measured using the camera sensor 15 can also be used for self-localization.

If a match is found between the features identified on the vehicle side and the features 62, 64, 66 of the feature map 60, the position of the vehicle 8 can be corrected or updated (28).

Claims (12)

A method (2) of generating a digital map by a control device (10), comprising:
Receive measurement data of the environment (U) during the measurement run (18),
performing SLAM based on the received measurement data to identify the trajectory of the measurement run (19);
converting the received measurement data into a coordinate system of the orbit (20);
using the converted measurement data to generate (21) an intensity map (30);
A method, wherein features (62, 64, 66) are extracted (23) from said intensity map (30) and stored (24) in a feature map (60). The method of claim 1, wherein the feature map (60) is saved as a digital map or as a map layer of a digital map. The received measurement data is present as a point cloud and is assigned to a raster consisting of a plurality of cells (31, 32), in order to generate the intensity map (30), the measurement data of each cell (31, 32) is The method according to claim 1 or 2, wherein an average value of is calculated. An altitude map (40) is generated from the received measurement data, and a weighted average value is calculated from the measurement data of each cell (31) and adjacent cells (32) to generate the altitude map (40). 4. The method according to claim 3. 5. Information according to claim 4, wherein information is received from the generated altitude map (40) and stored in the feature map (60) to determine the altitude of the extracted feature (62, 64, 66). The method described. The extracted features (62, 64, 66) are stored in the feature map (60) as universally identifiable features (62, 64, 66), and the features (62, 64, 66) are 6. A method according to any one of claims 1 to 5, wherein the geometric shapes, lines, points and/or point clouds are extracted and stored. In particular, a method (4) for performing self-localization by a control device (11), comprising:
receiving (25) measurement data and a feature map (60) of the environment (U);
recognizing and extracting features of the received measurement data (26);
A method (27) of identifying a position by comparing at least one extracted feature with features (62, 64, 66) stored in the feature map. The received measurement data is realized as position data and stored in the position diagram, and the identified position is added to the position diagram as a new measurement when the features (62, 64, 66) are successfully matched. 8. The method of claim 7, wherein the method is stored in (28). Method according to claim 7 or 8, wherein the measurement data is measured from at least one sensor (15) different from the at least one sensor (14) used to generate the feature map (60). A control device (10, 11) configured to implement at least one of the methods (2, 4) according to any one of claims 1 to 9. A plurality of instructions which, when executed by a computer or a control device (10, 11), cause the computer or a control device to carry out at least one of the methods (2, 4) according to any one of claims 1 to 9. computer programs, including; A machine-readable storage medium (12, 13) on which a computer program according to claim 11 is stored.

Images