DE102020210421A1 - Method and system for creating and teaching in a map of the surroundings for a trained parking process - Google Patents

Abstract

The invention relates to a method for creating and teaching in a map of the surroundings for a trained parking process with the following steps: requesting (S1) a new teaching process for a trained parking process; starting (S2) following a trajectory; localizing (S3) the ego - vehicle;- retrieval (S4) of at least one aerial image based on a current position of the ego vehicle;- extraction (S5) of features from the at least one aerial image;- starting (S6) of a learning process;- recording (S7) of an environment of the Ego vehicle with at least one surroundings detection sensor (2) of the ego vehicle; - Matching (S8) the features of the aerial image with the detected features of the at least one surroundings detection sensor (2); - Evaluating (S9) the features of the at least one surroundings detection sensor (2) as relevant or not relevant;- selecting (S10) the features of the detection of the at least one environment detection sensor (2) that are evaluated as relevant and linking them r features with the features of the aerial photograph for a map of the surroundings; - checking (S11) whether a destination point of the trajectory has been reached; - storing (S12) the trajectory covered and the map of the surroundings when the destination point is reached.

Description

The invention relates to a method and a system for creating and teaching in a map of the surroundings for a trained parking process.

Among other things, the assistance function of trained parking or “trained parking” is known from the prior art. A map of the surroundings and a trajectory are created and taught in, which enable a vehicle to drive automatically from a starting point to an end point (e.g. parking lot, garage, etc.) without the driver having to intervene or check it.

During the learning process, a map of the surroundings is created using the sensors installed on the vehicle (camera, radar, ultrasound, laser), which fulfills two functions. On the one hand, the map is used to localize the ego vehicle on the map if the automated parking maneuver is to be carried out after the training. On the other hand, it serves to identify obstacles (e.g. garbage can, flower pot, etc.) that were not present during training and which have to be avoided (= new path planning and trajectory that avoids a collision with the obstacle, but still allows the automated parking maneuver to be continued allows.

A problem of the prior art is that the ego sensor system can generate false detections (false positives) and it is sometimes not possible to recognize and differentiate which detections have a very important meaning, in particular for a later re-planning of the trajectory , and which may only be of a temporary nature.

It is therefore an object of the present invention to provide a method and a system by means of which a learning process of a card for trained parking can be improved and thus increase the robustness and reliability of the information present in the stored card.

This object is solved by the subject matter of independent claims 1 and 6.

• - Anfordern eines neuen Einlernvorgangs für einen trainierten Parkvorgang;
• - Starten eines Abfahrens einer Trajektorie;
• - Lokalisieren des Ego-Fahrzeugs;
• - Abrufen zumindest eines Luftbildes basierend auf einer aktuellen Position des Ego-Fahrzeugs;
• - Extraktion von Merkmalen aus dem zumindest einen Luftbild;
• - Starten eines Einlernvorgangs;
• - Aufzeichnen eines Umfelds des Ego-Fahrzeugs mit zumindest einem Umfelderfassungssensor des Ego-Fahrzeugs;
• - Matching der Merkmale des Luftbilds mit den detektierten Merkmalen des zumindest einen Umfelderfassungssensors;
• - Bewerten der Merkmale des zumindest einen Umfelderfassungssensors als relevant oder nicht relevant;
• - Selektion der als relevant bewerteten Merkmale der Detektion des zumindest einen Umfelderfassungssensors und Verknüpfen dieser Merkmale mit den Merkmalen des Luftbilds zu einer Umgebungskarte;
• - Überprüfen ob ein Zielpunkt der Trajektorie erreicht wurde;
• - Abspeichern der zurückgelegten Trajektorie sowie der Umgebungskarte bei Erreichen des Zielpunkts.

According to the invention, a method for creating and teaching in a map of the surroundings for a trained parking process is provided with the following steps:

• - Requesting a new learning process for a trained parking process;
• - Starting a trajectory traversing;
• - locating the ego vehicle;
• - retrieving at least one aerial image based on a current position of the ego vehicle;
• - Extraction of features from the at least one aerial image;
• - Start a learning process;
• - recording an environment of the ego vehicle with at least one environment detection sensor of the ego vehicle;
• - Matching the features of the aerial image with the detected features of the at least one environment detection sensor;
• - Evaluating the features of the at least one environment detection sensor as relevant or not relevant;
• - Selection of the features of the detection of the at least one surroundings detection sensor rated as relevant and linking these features with the features of the aerial image to form a map of the surroundings;
• - Check whether a target point of the trajectory has been reached;
• - Saving the trajectory covered and the map of the area when the destination is reached.

The ego vehicle can be localized using GPS or GNSS, for example. At least one aerial image is then retrieved based on the position. Several aerial photos can also be called up, depending on the route to be covered. These aerial images are advantageously satellite images of the earth's surface. (e.g. Google Maps/ Google Earth or similar). This aerial photograph or aerial photographs are then used to generate a map for trained parking. The aerial images can, for example, already be stored in a storage device for the trajectory to be driven. It would also be conceivable to retrieve an up-to-date aerial photo for the trajectory directly from the Internet. For this it would be conceivable for the vehicle to establish a connection to the Internet using a corresponding module. This is particularly advantageous in the case of new trajectories or a newly trained parking scenario. If an aerial image is already stored for the trajectory, the aerial image can primarily be used for future travel along the same trajectory. For further processing of the aerial image, the section or area relevant to the trajectory is considered and further analyzed. In this way, features are extracted in this area, which simplifies training with ego sensors, since the features recorded by means of the environment detection sensor are then matched with the features of the aerial image. This means that the features are compared with each other and checked for consistency. So a better assignment of the features of the aerial image to the detections of the at least one environment detection sensor and thus to real objects can advantageously be made possible.

The request for the learning process can be triggered by the driver, for example.

The at least one environment detection sensor is, for example, a mono camera, a stereo camera, a radar sensor, a lidar sensor or an ultrasonic sensor. The use of several cameras would also be conceivable, for example as part of a surround view system. It would also be conceivable to use a plurality of sensors or a combination of the aforementioned sensors. This is advantageous since a number of sensors and also different sensors detect different objects better and the detections can be verified via a sensor data fusion.

The features of the at least one environment detection sensor can be evaluated as relevant or not relevant. This is advantageous because not all features have to be used for the trajectory planning, but only the features evaluated as relevant are used, which in turn contributes to a reduction in the computing effort. Furthermore, the features of the detection of the at least one environment detection sensor that are evaluated as relevant are selected and linked to the features of the aerial image to form an environment map. By combining these two pieces of information, a more reliable and robust map can be created. It is then checked whether the target point of the trajectory has been reached. If the target point has not yet been reached, the procedural steps are repeated after the start of the teach-in process until the target point has been reached. This is advantageous because it ensures that the entire route can be saved with all relevant features. Finally, the trajectory covered and the map of the area are saved when the destination is reached.

In a preferred embodiment, it is provided that the features of the aerial image include landmarks and structures.

In a further particularly preferred refinement, the assessment as relevant or not relevant is carried out based on the result of the matching of the features. In this way, a more robust map for trained parking can advantageously be created. Furthermore, faulty detections of the surroundings detection sensors can also be recognized. For example, detections of walls and buildings can be better assigned to them and they receive a greater weight/confidence value and thus greater relevance. The data obtained by means of the at least one environment detection sensor can thus be labeled according to the features extracted from the aerial image. Furthermore, it is possible to determine whether in relevant areas, e.g. house corner, garage entrance, extracted from the aerial photo, there are sufficiently many and reliable detections of the at least one environment detection sensor, so that unwanted "white spots" in the environment map for the trained parking avoided will.

In a further preferred embodiment, in addition to the assessment as relevant or not relevant, a mobility hypothesis value based on the result of the matching of the features is undertaken. Accordingly, the value is defined when features are indeed detected by the environment detection sensor, but do not or not always appear in the aerial image. The features can be objects that are present while the trajectory is being followed or the trajectory is being taught in, but may not appear on the aerial image because the objects are only present temporarily. This can be garbage cans, for example, or other parked vehicles, parked bicycles or motorbikes. These features can, for example, be marked as potentially moving obstacles in order to differentiate between these features and permanently static obstacles such as house walls. The mobility hypothesis value thus describes a parameter that specifies that a feature only occurs at certain times or only possibly at the detected position.

This hypothetical value is preferably also entered in the map of the surroundings, so that this value is noted for subsequent journeys along this trajectory. This is advantageous because in this way the areas of the map of the surroundings provided with such a value can be regarded as potentially occupied. Furthermore, such a hypothetical value is advantageous since non-detection of such features when the trajectory is traversed again does not lead to uncertainties or errors in the map, since the hypothetical value indicates that the features are not permanently present.

Furthermore, in a further embodiment, height information of a soil profile is additionally extracted from the aerial image. This information is particularly relevant for curb detection or when the trained parking trajectory has a strong vertical profile, such as a steep ramp lying on a portion of the trajectory. With this knowledge the environment map for trained parking can be designed in such a way that it is taken into account that the ego sensors perceive different features of the environment while driving on the ramp than in front of or behind the ramp.

Furthermore, the aerial image is preferably automatically updated at definable time intervals. A comparison can be made here as to whether the updated aerial photo reveals better/more precise/new features that were not present in the old version. For example, a time interval of 1 month would be conceivable. Longer time intervals would also be conceivable.

Furthermore, in a preferred embodiment of the invention, a confidence value for the map of the environment is determined before the map of the environment is stored. This confidence value is determined by checking the map of the surroundings for completeness. The more complete the map, the higher the confidence value. Areas without detections or without features from the aerial image lead to incompleteness and thus to a lower confidence value. Local low-confidence areas can be used during the learning process to control the type and representation of sensor readings and environment modeling (e.g. storing denser sensor readings or even raw values, which would not be possible for the whole map). When applying and executing a parking process based on the stored map, the confidence values can be used for trajectory planning, so that the trajectory is planned in such a way that local areas with low confidence are deliberately avoided or bypassed.

In general, the method is advantageous since normal navigation maps and HD maps for driver assistance systems do not cover private areas on which trained parking would be very relevant. Areas that are not visible from the public road (e.g. backyards) are also visible on the aerial photo. The aerial photos are available worldwide and have a very high resolution and accuracy in cities and built-up areas (e.g. Google Maps / Google Earth).

Relevant structures and information (vegetation, curb, building) can be automatically extracted from the lift images using state-of-the-art machine image processing methods. This can even be done in the vehicle if only the pixel-based aerial image is transmitted to the vehicle.

Furthermore, according to the invention, a system is provided which is designed to carry out the method according to the invention, comprising at least one environment detection sensor for recording the environment of an ego vehicle, a localization module for locating the position of an ego vehicle, a computing device, by means of which detections of the at least one Environment detection sensor and features of an aerial image can be evaluated and an environment map can be created, a control device, which is designed to control the ego vehicle according to a trajectory of a trained parking process, and a memory device, which is designed to store the created environment map and a driven trajectory of the ego to save the vehicle.

The system can also have a module that enables a connection to the Internet in order to call up the corresponding aerial photos. In the light of the invention, a localization module is understood to mean a module which is designed for position determination using GPS or GNSS.

• 1: ein schematisches Ablaufdiagramm einer Ausführungsform der Erfindung;
• 2: eine schematische Darstellung eines Systems gemäß einer Ausführungsform der Erfindung.

Further advantageous refinements and embodiments result from the drawings. Show in it:

• 1 : a schematic flow diagram of an embodiment of the invention;
• 2 1: a schematic representation of a system according to an embodiment of the invention.

In 1 a schematic flow diagram of an embodiment of the invention is shown. In step S1 we request a new learning process for a trained parking process. Following this or at the same time, a trajectory is started to be followed in step S2. The ego vehicle is localized in S3 shortly after or at the same time as the starting of the departure. After localization, at least one aerial image based on a current position of the ego vehicle is retrieved in step S4. Subsequently, in a step S5, features are extracted from the at least one aerial image. After this extraction, a learning process is started in step S7. In step S7, surroundings of the ego vehicle are now recorded with at least one surroundings detection sensor of the ego vehicle. Then, in step S8, the features of the aerial image are matched with the detected features of the at least one environment detection sensor. Then, in step S9, the features of the at least one environment detection sensor are evaluated as relevant or not relevant. In step S10, the features of the detection of the at least one environment detection sensor that are evaluated as relevant are selected and linked to the features of the aerial image to form an environment map. Then, in step S11, it is checked whether a target point of the trajectory has been reached. If the destination has been reached, the trajectory covered and the map of the surroundings are stored in a step S12. If the destination has not yet been reached, steps S7 to S11 are repeated until it is determined in step S11 that the destination has been reached.

2 shows a schematic representation of a system 1 according to an embodiment of the invention. The system 1 comprises at least one environment detection sensor 2 for recording the environment of an ego vehicle, a localization module 4 for locating the position of an ego vehicle, a computing device 3, by means of which detections of the at least one environment detection sensor 2 and features of an aerial image can be evaluated and a map of the surroundings can be created, a control device 5, which is designed to control the ego vehicle according to a trajectory of a trained parking process, and a storage device 6, which is designed to store the created map of the surroundings and a trajectory driven by the ego vehicle. Furthermore, a data connection D between the modules 2, 4, 5, 6 and the computing device 3 is provided. This data connection D can be wired or wireless, for example with Bluetooth or WLAN.

Reference List

1

system

2

Environment detection sensor

3

computing device

4

localization module

5

control device

6

storage device

D

Data Connection

S1-S12

process steps

Claims (8)

Procedure for creating and teaching an environment map for a trained parking process with the following steps: - Requesting (S1) a new learning process for a trained parking process; - Starting (S2) a traversing of a trajectory; - Locating (S3) the ego vehicle; - retrieving (S4) at least one aerial image based on a current position of the ego vehicle; - Extraction (S5) of features from the at least one aerial image; - Starting (S6) a learning process; - Recording (S7) an environment of the ego vehicle with at least one environment detection sensor (2) of the ego vehicle; - Matching (S8) of the features of the aerial image with the detected features of the at least one environment detection sensor (2); - Evaluation (S9) of the features of the at least one surroundings detection sensor (2) as relevant or not relevant; - selection (S10) of the features of the detection of the at least one environment detection sensor (2) that are rated as relevant and linking these features to the features of the aerial image to form an area map; - Checking (S11) whether a target point of the trajectory has been reached; - Saving (S12) the trajectory covered and the map of the surroundings when the destination is reached. procedure after claim 1 , characterized in that the features of the aerial image include landmarks and structures. procedure after claim 1 , characterized in that the assessment is made as relevant or irrelevant based on the result of the matching of the characteristics. procedure after claim 1 , characterized in that in addition to the assessment as relevant or not relevant, a mobility hypothesis value based on the result of the matching of the features is made. procedure after claim 1 or 2 , characterized in that additional height information of a soil profile is extracted from the aerial photograph. procedure after claim 1 or 2 , characterized in that the aerial photograph is automatically updated at predeterminable time intervals. procedure after claim 1 , characterized in that a confidence value for the map of the environment is determined before the map of the environment is stored. System (1) which for carrying out the method according to one of Claims 1 until 7 is configured, comprising at least one environment detection sensor (2) for recording the environment of an ego vehicle, a localization module (4) for locating the position of an ego vehicle, a computing device (3) by means of which detections of the at least one environment detection sensor (2 ) and features of an aerial image can be evaluated and a map of the surroundings can be created, a control device (5) which is designed for the ego vehicle according to a Trajek to control the theory of a trained parking process and a memory device (6), which is designed to store the created map of the surroundings and a driven trajectory of the ego vehicle.

Images