WO2024089080A1

WO2024089080A1 - Method for recognising and classifying elevated road markings

Info

Publication number: WO2024089080A1
Application number: PCT/EP2023/079734
Authority: WO
Inventors: Annette Frederiksen; Zoltan Ersek; Axel Fink; Mustafa Kamil; Mario Lietz; Alf Neustadt; Arne Josten
Original assignee: Robert Bosch Gmbh
Priority date: 2022-10-26
Filing date: 2023-10-25
Publication date: 2024-05-02
Also published as: DE102022211343A1

Abstract

The invention relates to a method for recognising and classifying elevated road markings (14) by means of a lidar system (36) of a vehicle (10). The lidar system (36) comprises a transmitting unit (38), a receiving unit (44), at least one laser beam source (46) and at least one detector (48). The following method steps are carried out individually or in combination: a) comparing a map, which contains information about the roadway (20), with possible elevated road markings (14) to be recognised along the roadway (20) by the lidar system (36), b) determining normal road markings from at least one intensity-coded or background-light-coded 3D point cloud (28) and comparing same with possible elevated road markings (14) to be recognised along the roadway (20) by the lidar system (36), c) determining context-sensitive information from an accompanying infrastructure or from a traffic-planning relevance of installation locations of elevated road markings (14) and/or d) analysing the intensity-coded or background-light-coded 3D point cloud (28) for regular patterns (78) of an arrangement (60) of elevated road markings (14).

Description

Method for detecting and classifying raised road markings

Technical area

The invention relates to a method for detecting and classifying raised road markings using a LiDAR system of a vehicle. The LiDAR system comprises a transmitting unit, a receiving unit, at least one laser beam source and at least one detector. Furthermore, the invention relates to the use of the method in a LiDAR system for the reliable detection and drivability classification of raised road markings.

State of the art

DE 10 2011 082 477 A1 relates to a method and a system for creating a digital image of a vehicle's surroundings. The system represents a driver assistance system for autonomous driving. For this purpose, measurement data on the vehicle's surroundings are recorded using a vehicle-based first sensor system and at least one optically detectable marking that is fixedly arranged in the vehicle's surroundings is recorded using a vehicle-based video system. At least one optically detectable characteristic of the at least one marking is recognized from the recording data of the video system and the relative position and orientation of the marking to the vehicle's surroundings is determined.

EP 3 529 561 Bl relates to a system and a method for generating digital road models from aerial or satellite images and from vehicle-recorded data. At least one trajectory of a vehicle for the at least one road section is received in a vehicle-external database. At least one image is received which contains at least shows parts of the at least one road section, wherein the image has a perspective that corresponds to an image taken from an elevated position substantially vertically downwards. Then the at least one trajectory is superimposed with the at least one image such that the at least one trajectory corresponds to the course of a road in the at least one image. The at least one image is then analyzed in a corridor that extends along the trajectory and encloses it, and driving or position-relevant features of the road section in the corridor are recognized.

DE 10 2019 106 213 A1 relates to a method for determining at least one item of position information of at least one object in a surveillance area using an optical detection device, as well as to an optical detection device. A method for determining at least one item of position information of at least one object in a surveillance area using an optical detection device, as well as the detection device, are described here. In the method, at least one optical transmission signal with a main transmission signal propagation axis is sent into the surveillance area. At least part of the at least one transmission signal is reflected as a reception signal at an object present in the surveillance area, and at least part of the reception signal, the initial signal propagation main axis of which runs parallel to the main transmission signal propagation axis at least on the side facing the object, is varied.

Current LiDAR sensor systems use SPAD detectors, which can have disadvantages related to a dynamic range for objects with different reflectivity at different distances. The LiDAR sensor system has the task of reliably detecting objects with particularly low reflectivity (e.g. 5%) at a great distance, such as 200 m, as well as retroreflectors with particularly high reflectivity (e.g. 10,000%) at a short distance (e.g. a few meters). The resulting dynamic factor between the darkest and the brightest object to be detected is up to 2000 at the same distance; in addition, there is the part of the signal intensity that scales quadratically with the distance (has a degrading effect). If the LiDAR sensor system is set up according to functional reliability on If the detection of weakly reflective objects at the greatest possible distance is limited, the probability of optical saturation of the detector with scattered light-induced false detections, also known as "crosstalk", increases. This is a significant effect, especially for close and highly reflective objects. The challenge here is to design the optical design within a LiDAR sensor system consisting of transmit and receive lenses, laser diodes and detector pixels in such a way that the dynamic range can be reliably recorded.

Description of the invention

According to the invention, a method for detecting and classifying raised road markings by means of a LiDAR system of a vehicle is proposed, wherein the LiDAR system comprises a transmitting unit, a receiving unit, at least one laser beam source and at least one detector, with at least the following method steps, which are carried out individually or in combination with one another: a) carrying out a comparison between a map containing information on the course of the road and possible raised road markings that can be detected along the course of the road by the LiDAR system, b) determining normal road markings from at least one intensity-coded or background-light-coded 3D point cloud and comparing them with possible raised road markings that can be detected along the course of the road by the LiDAR system, c) determining context-sensitive information from an accompanying infrastructure or from a traffic planning relevance of installation locations of raised road markings and d) analyzing the intensity-coded or background-light-coded 3D point cloud for regular patterns of an arrangement of raised road markings.

Advantageously, the method proposed according to the invention can achieve a reliable distinction between false detections produced by raised road markings and objects that cannot actually be driven over. In an advantageous development of the solution proposed according to the invention, according to method step a), a map comparison is carried out based on a geoposition of the vehicle with the map or observations of vehicles driving ahead in a cloud-based database.

In an advantageous development of the method proposed according to the invention, an approximation of an analytically describable curve or a geometric plane is carried out according to method step d), which is carried out in such a way that the possible raised road markings to be recognized are suitably connected to one another and a continuous check for plausibility is carried out.

Within the scope of the method proposed according to the invention, a regular repetition of possible raised road markings to be detected can advantageously be carried out according to method step d), each at the same distance from one another along the analytically describable curve or the geometric plane.

In the method proposed according to the invention, according to method step d), a circle can be drawn around possible raised road markings to be recognized, such that intersection points of adjacent circles form horizontal, vertical or mutually orthogonal straight lines. Arrangement patterns of road markings can be extrapolated using the horizontal, vertical or mutually orthogonal straight lines.

In an advantageous development of the method proposed according to the invention, geometric plausibility checks are carried out according to method step d) to determine a geometric regularity of the occurrence of raised road markings for demarcation against individual objects lying on the road.

In an advantageous development of the method proposed according to the invention, according to method step d), a targeted forecast is carried out in such a way that a targeted extrapolation to further raised road markings is carried out from the intensity-coded 3D point cloud on the basis of less visible raised road markings as a continuation of an edge geometric regularity. In the method proposed according to the invention, a comparison is made between the intensity-coded 3D point clouds on the one hand and the background-light-coded 3D point clouds on the other. An intensity-coded 3D point cloud is a cloud of 3D points with a brightness value corresponding to the received signal of the reflected laser pulse. A background-light-coded 3D point cloud is a cloud of 3D points that have a brightness value that corresponds to the brightness of the received background noise signal.

Finally, the invention relates to the use of the method in a LiDAR system for the reliable detection and overridability classification of raised road markings.

Advantages of the invention

The solution proposed according to the invention in the form of the method according to the invention enables a reliable detection and classification of the ability to drive over raised road markings. This enables an accurate distinction to be made between these and objects that cannot actually be driven over, for example warning beacons, traffic signs or the like, which have a not inconsiderable height so that the vehicle and its occupants would inevitably be damaged. The solution proposed according to the invention enables a reliable detection and classification of raised road markings by analyzing geometric structures, installation models, number, distance and drop in the optical crosstalk intensity, over distance and angle as well as other recurring regularities. Of crucial importance in the method proposed according to the invention is that a raised road marking rarely appears alone and without context. This means that a raised road marking is generally used in conjunction with other raised road markings and is associated with accompanying infrastructure or traffic planning relevance with regard to the installation location. Examples of danger spots include, for example, merging lanes, the proximity an airport, motorway junctions, construction sites or the like. The solution proposed according to the invention can be used to reliably distinguish between false detection induced by raised road markings and objects that cannot actually be driven over, for example in the form of warning beacons extending in a vertical direction. Furthermore, the method proposed according to the invention can achieve a not inconsiderable improvement in the processed point cloud quality after optical crosstalk filtering and thus a considerable improvement in the performance of a LiDAR sensor system. The solution proposed according to the invention can effectively compensate for optical crosstalk that occurs after it has been detected.

As a result, the method proposed according to the invention can ensure that the LiDAR sensor of a LiDAR sensor system generates a crosstalk-free point cloud when raised road markings occur. This makes it easy and reliable to identify a key source of the optical crosstalk that occurs.

By using the method proposed according to the invention, incorrect braking in automated driving functions can also be prevented; furthermore, missing braking in front of relevant objects can be prevented, thereby increasing road safety. Furthermore, by using the method proposed according to the invention, the occurrence of undesired braking on non-relevant objects can be prevented, which is also extremely beneficial to road safety. The method proposed according to the invention can support data integrity within the SD point cloud.

Short description of the drawings

Embodiments of the invention are explained in more detail with reference to the drawings and the following description.

Show it: Figure 1 a vehicle with a LiDAR system in front of a raised lane marking, a lane course and a 3D point cloud derived from the lane course,

Figure 1.1 the components of a LiDAR system,

Figure 2 shows a raised road marking, emerging crosstalk areas and identified crosstalk areas and a tolerance range limited by them,

Figure 3 is an exemplary representation of raised road markings arranged in regular patterns along lanes of a curved roadway,

Figure 4 shows a perspective view of a narrowing of the roadway, bordered by pairs of raised road markings on both sides, as well as an extrapolation of this scene and

Figure 5 a circle made up of a number of raised road markings, bounded by two line patterns.

Embodiments of the invention

In the following description of the embodiments of the invention, identical or similar elements are designated by identical reference numerals, whereby a repeated description of these elements is omitted in individual cases. The figures only represent the subject matter of the invention schematically.

Figure 1 shows a vehicle 10 that is equipped with a LiDAR system 36. In front of the vehicle 10, there is a roadway 12 that has a raised road marking 14 that is in the field of view of the LiDAR system 36. This raised road marking 14 has a height 16 that is usually a few centimeters. Above the raised Crosstalk areas 18, also referred to as crosstalk areas, extend from the road marking 14.

Figure 1 also shows a field of view of the scene extending in front of the vehicle 10. A roadway 20 is shown in this scene as a straight line 22. On the roadway 12 there are warning beacons 26 which extend in a vertical direction from the surface of the roadway 12. Furthermore, next to the two warning beacons 26 there are two raised road markings 14 at a distance from each other.

Below the scene of the roadway 12 in front of the vehicle 10, an intensity-coded or background-light-coded 3D point cloud 28 is shown, which represents the recording of the scene according to the course of the roadway 20 by the LiDAR system 36. From the representation of the intensity-coded or background-light-coded 3D point cloud 28 according to Figure 1, it can be seen that the warning beacons 26 extending in the vertical direction are shown as objects 30 classified as non-driveable. In contrast, the raised road markings 14 on the roadway 12, which are spaced apart from one another in the scene, are shown as objects 32 extending in the horizontal direction, and thus as objects 34 that are reproduced in a distorted manner. Despite the relatively low construction height 16 of the raised road markings 14, they appear as extended objects 32 in the intensity-coded or background-light-coded 3D point cloud 28 and cannot be clearly classified as drivable due to their crosstalk properties.

The representation according to Figure 1.1 schematically shows the components of the LiDAR system 36 that are installed in the vehicle 10 shown in Figure 1. For example, the LiDAR system 36 comprises a transmitting unit 38 that includes at least one laser beam source 46. The LiDAR system 36 also comprises a receiving unit 44 that contains a detector 48 onto which radiation reflected from an object detected by the LiDAR system 36 is reflected. Said LiDAR system 36 can also comprise other components that are not shown in detail here and is generally arranged in the front area of the vehicle 10 to detect the scene in front of the vehicle 10. Figure 2 shows a representation of a raised road marking 14 and crosstalk areas 18 emanating from it. Figure 2 shows that the raised road marking 14 has a first retroreflective surface 40 and a second retroreflective surface 42 opposite it. Said retroreflective surfaces 40, 42 are recorded by a LiDAR system 36 as an intensity-coded or background-light-coded 3D point cloud 28. Above and below the raised road marking 14 there are crosstalk areas 18, which can be divided into a tolerance range 50 and identified crosstalk areas 52. The identified crosstalk areas 52 are areas identified as safe crosstalk areas 52. Based on the crosstalk areas 18, it cannot be determined whether the raised road marking 14 can be correctly classified as being drivable. Therefore, such detection by the LiDAR system 36 would result in an unjustified false braking or an unjustified abort of a lane change with the resulting disadvantages for driving experience, trajectory planning and accident safety.

With reference to Figure 2, it should be noted that outside the tolerance range 50 around the raised road marking 14, a crosstalk point 54 can be misclassified as an object point 56, which will occur with a probability of, for example, less than 5%. Reference number 56 designates an object point that is misclassified as optical crosstalk, although the object point 56 is part of the raised road marking 14. Figure 2 uses the object point 56 to visualize a key performance requirement, i.e. the object point 56 must be recognized by the LiDAR system 36. By implementing the method proposed according to the invention in a LiDAR system 36 of the vehicle 10, reliable detection and overridability classification of raised road markings 14 is provided, so that a distinction can be made with certainty between objects 30 that cannot actually be driven over, such as the warning beacons 26 shown in Figure. A reliable detection and reliable overridability classification of raised road markings 14 can be carried out, which is achieved by an analysis of geometric structures, installation models, number, distance and drop of the Crosstalk intensity can be measured over distance and angle as well as other regularities. The decisive factor in this context is the fact that a raised road marking 14 rarely occurs alone or free of context, such as other raised road markings 14, an accompanying infrastructure or traffic planning relevance of installation locations, such as near an airport, in merging areas, at danger points or the like.

The method proposed according to the invention is based on the fundamental approach that raised road markings 14 rarely appear alone, but that they appear in arrangements 60 of regular patterns 78, as shown in Figures 3, 4 and 5.

In principle, 14 different information sources can be used to detect and reliably classify raised road markings, either individually or in combination in the sense of information fusion.

Thus, according to a method step a), a comparison can be made between a map with information on the course of the road and raised road markings 14 that can possibly be recognized on a curve. As a rule, a map comparison is carried out based on the geoposition of the vehicle 10 with the map or by observing vehicles driving ahead, for example via a cloud-based database.

According to a method step b), a road marking is used from an intensity-coded or a background-light-coded 3D point cloud 28 and a comparison is made with raised road markings 14 that may be recognizable on a curve 24, as shown for example in Figure 3. According to a method step c), context-sensitive information of an accompanying infrastructure or traffic planning relevance of installation locations of infrastructure components, for example for danger spots, to indicate proximity to an airport or hospital, or as an indication of lane junctions, government buildings, tunnels, etc. As part of the detection and reliable classification of raised road markings 14, the intensity-coded or background-light-coded 3D point cloud 28 can also be used to detect the occurrence of regular patterns 78 of arrangements 60 of raised road markings 14.

To analyze the intensity-coded or background-light-coded 3D point cloud 28 shown in Figure 1, for example, a search is carried out for the occurrence of regular patterns 78 within which the raised road markings 14 are located at certain positions. For example, an approximation of an analytically describable curve 62 can be made, as shown in Figure 3, or a geometric plane 65 can be shown (see Figure 3), within which the possible raised road markings 14 to be recognized are suitably connected to one another and a continuous check for plausibility is carried out. The intensity-coded or background-light-coded 3D point cloud 28 can also be examined for regularly occurring repetitions of possible raised road markings 14 to be recognized, which each have an equal distance 76 from one another, wherein the regular repetition of possible raised road markings 14 to be recognized with the same distance 76 from one another can be found along curves 24 or straight lines 22 according to the roadway course 20.

Furthermore, for example, in the context of the analysis of the intensity-coded or background-light-coded 3D point cloud 28, a circle 70 can be mentally drawn around a possible raised road marking 14 to be recognized, which circle has a sufficiently large radius, so that the resulting intersection points of the circles 70 result in straight lines that are orthogonal to one another, both horizontally and vertically and relative to one another, and run parallel to one another.

Within the framework of the above-mentioned process steps a), b), c) and d), a large number of geometric plausibility checks can be applied to the intensity-coded 3D point cloud 28. The aim is to prove the occurrence of a geometric regularity as a distinguishing feature for example, to a single lost load lying on lane 12.

Using pattern recognition and/or matching algorithms, setup models of raised road markings 14 can be generated as clear indications of these as man-made, artificial infrastructure measures. For example, a speed warning can be surrounded by a circular collection of raised road markings 14, as shown in Figure 5, for example, so that for recognition only a correlation of the intensity-coded or background-light-coded 3D point cloud 28 with a circular pattern (see position 70 in Figure 5) would have to be carried out.

Optical crosstalk 18 itself can also provide reliable information for detecting raised road markings 14. The intensity of false detections decreases with the angle of view in the LiDAR system 36 and over the distance in a non-linear but analytically describable manner. Not only can currently available information sources be used to make the classification decision, but it is also possible to look ahead in a targeted manner, resulting in an extrapolation 68, as shown in connection with Figure 4. For the analysis of the intensity-coded or background-light-coded 3D point cloud 28, the extrapolation 68 is derived from a few visible raised road markings 14, so that a continuation of a once recognized geometric regularity, in the case of Figure 4 a regular distance 76 between the individual raised road markings 14 and a gradual reduction in the distance between them, can be concluded, so that the extrapolation 68 of the further course of the roadway 12 indicated in Figure 4 results.

It is possible to compare the background light-coded 3D point cloud with the intensity-coded 3D point cloud 28. Since retroreflective surfaces 40, 42 basically reflect the incident light back in the same direction from which it entered, retroreflective surfaces 40, 42 appear less intense in background light-coded point clouds 28 than in intensity-coded 3D point clouds 28, since the background light of the first-mentioned coding is not measured. The Sunlight is not reflected to the LiDAR sensor, but back into the sky.

Figure 3 shows a representation of raised road markings 14 which are arranged, for example, along lanes 80 of the roadway 12 in regular patterns 78. Figure 3 shows that the individual raised road markings 14 have a very low construction height 16, i.e. represent objects that can be driven over. In the embodiment according to Figure 3, the raised road markings 14 serve as boundaries of lanes 80 of a multi-lane roadway 12. The individual raised road markings 14 are provided in an arrangement 60 which, on the one hand, is adapted to the curve 24 of the individual lanes 80 of the roadway 12 and, on the other hand, has a regular distance 76 from one another. In the vertical direction, the individual raised road markings 14 are evenly spaced from one another and in this context serve as boundaries of the individual lanes 80 of the roadway 12. The arrangement 60 of the individual raised road markings 14 is shown in Figure 3 as an analytically describable curve 62. The roadway 12 with a number of lanes 80, which are separated from one another by the regular patterns 78 of raised road markings 14, is delimited by a road edge 64 on the right-hand side.

In contrast to the representation according to Figure 3, in the representation according to Figure 4 a geometric plane 65 is designated which represents a straight course 22 of the roadway 12. Figure 4 shows that the raised road markings 14 arranged in pairs on the right and left in the geometric plane 65 are arranged at identical distances 76 from one another. From the regular pattern 78 according to Figure 4, an extrapolation 68 can be obtained from the geometric plane 65, which continues the recognized regular pattern 78 of the arrangement 60 according to the straight course 22 along the roadway 12. In this way, further information regarding the route further ahead of the vehicle 10, i.e. the roadway 12, can be obtained from a few visible raised road markings 14 by means of extrapolation 68.

Figure 5 shows a geometric plane 65 which is defined by a first line pattern 72 on the left side and by a second line pattern 74 on the right side is limited. From the illustration according to Figure 5 it can be seen that the individual raised road markings 14 are arranged along the first line pattern 72 and the second line pattern 74 in regular patterns 78 with an identical distance 76 from one another. In the middle, for example between the two line patterns 72, 74, a circle 70 made up of individual raised road markings 14 arranged in a regular pattern 78 is shown. In relation to the pattern of the circle 70, the individual raised road markings 14 have an identical distance 76 from one another.

The invention is not limited to the embodiments described here and the aspects highlighted therein. Rather, a large number of modifications are possible within the scope specified by the claims, which are within the scope of expert action.

Claims

Expectations

1. Method for detecting and classifying raised road markings (14) by means of a LiDAR system (36) of a vehicle (10), wherein the LiDAR system (36) comprises a transmitting unit (38), a receiving unit (44), at least one laser beam source (46) and at least one detector (48), with at least the following method steps, which are carried out individually or in combination with one another: a) carrying out a comparison between a map containing information on the course of the road (20) and raised road markings (14) that can be detected along the course of the road (20) by the LiDAR system (36), b) determining normal road markings from at least one intensity-coded or background-coded 3D point cloud (28) and comparing them with raised road markings (14) that can be detected along the course of the road (20) by the LiDAR system (36), c) determining context-sensitive information from an accompanying infrastructure or from a traffic planning relevance of Installation locations of raised road markings (14) and/or d) analysis of the intensity-coded or background-light-coded 3D point cloud (28) for regular patterns of an arrangement (60) of raised road markings (14).

2. Method according to claim 1, characterized in that according to method step a) a map comparison is carried out on the basis of a geoposition of the vehicle (10) with the map or with observations of preceding vehicles in a cloud-based database.

3. Method according to claims 1 and 2, characterized in that according to method step d) an approximation of an analytically describable curve (62) or a geometric plane (65) is carried out in such a way that the possible increased Road markings (14) are suitably connected to one another and a continuous check for plausibility is carried out. Method according to claims 1 to 3, characterized in that according to method step d) a regular repetition of possible raised road markings (14) to be recognized is carried out at equal distances (76) from one another along the analytically describable curve (62) or the geometric plane (65). Method according to claims 1 to 4, characterized in that according to method step d) circles (70) are drawn around possible raised road markings (14) to be recognized in such a way that intersection points of adjacent circles (70) form horizontal, vertical and mutually orthogonal straight lines (72, 74). Method according to claims 1 to 5, characterized in that according to method step d) geometric plausibility checks are carried out to determine a geometric regularity in the occurrence of raised road markings (14) and a demarcation from individual objects lying on the road (12) is carried out. Method according to claims 1 to 6, characterized in that according to method step d) a targeted forecast is carried out in such a way that a targeted extrapolation to further raised road markings (14) is carried out from the intensity-coded or background-light-coded 3D point cloud (28) on the basis of less visible raised road markings (14) as a continuation of a recognized geometric regularity. Method according to claims 1 to 7, characterized in that a comparison of the intensity-coded 3D point cloud (28) is carried out with a background-light-coded 3D point cloud (28). Method according to claims 1 to 8, characterized in that by comparing background light-coded point clouds with intensity-coded point clouds (28) in the case of signs of an infrastructure, crosstalk effects (18) are minimized. Use of the method according to one of claims 1 to 9, characterized in that a LiDAR system (36) is used for reliable detection and crossing classification of raised road markings (14).