CN110188687B - Method, system, device and storage medium for identifying terrain of automobile - Google Patents
Method, system, device and storage medium for identifying terrain of automobile Download PDFInfo
- Publication number
- CN110188687B CN110188687B CN201910462467.0A CN201910462467A CN110188687B CN 110188687 B CN110188687 B CN 110188687B CN 201910462467 A CN201910462467 A CN 201910462467A CN 110188687 B CN110188687 B CN 110188687B
- Authority
- CN
- China
- Prior art keywords
- information
- terrain
- point cloud
- cloud data
- topographic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000005070 sampling Methods 0.000 claims abstract description 49
- 238000007781 pre-processing Methods 0.000 claims abstract description 20
- 238000000605 extraction Methods 0.000 claims abstract description 18
- 239000013598 vector Substances 0.000 claims abstract description 15
- 238000012549 training Methods 0.000 claims description 25
- 238000004422 calculation algorithm Methods 0.000 claims description 17
- 238000013528 artificial neural network Methods 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 15
- 238000007637 random forest analysis Methods 0.000 claims description 15
- 239000004576 sand Substances 0.000 claims description 5
- 230000003044 adaptive effect Effects 0.000 claims description 4
- 238000012876 topography Methods 0.000 claims description 4
- 238000003066 decision tree Methods 0.000 description 9
- 238000009434 installation Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000001914 filtration Methods 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 238000002360 preparation method Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides a terrain recognition method, a terrain recognition system, equipment and a storage medium for an automobile, wherein the method comprises the following steps: s10, point cloud data output by at least one vehicle-mounted laser radar and image information output by at least one vehicle-mounted camera are obtained; s20, preprocessing the sampling points in the point cloud data to obtain second point cloud data; preprocessing the image information to obtain second image information; s30, clustering the second point cloud data to obtain a point cluster; s40, classifying the point clusters by using a first classifier to obtain first terrain information; s50, performing feature extraction on the second image information to obtain an image feature vector; s60, classifying the image feature vectors to obtain second topographic information; s70, obtaining map information of the position of the vehicle, wherein the map information comprises third topographic information; s80, fusing the terrain information to obtain a terrain recognition result; the invention realizes accurate identification of the terrain where the vehicle is about to pass, and is convenient for the vehicle to prepare in advance.
Description
Technical Field
The present invention relates to the field of automotive electronics, and in particular, to a method, a system, a device, and a storage medium for identifying a topography of an automobile.
Background
Some of the cars currently on the market are equipped with an all-terrain mode control system, and when driving a car equipped with an all-terrain control system, the driver can visually determine the type of terrain over which the vehicle is traveling, such as: ordinary roads, sand, muddy ground or snow, and then manually switch between the various terrain modes to select the corresponding terrain mode, thereby enabling the respective driving assistance devices of the automobile, such as: an Electronic Power Steering (EPS), a Transmission Control Unit (TCU), a four-wheel drive (Torque On Demand), an Electronic Differential lock (EDS), or a vehicle body Stability System (ESP) enters an operating mode corresponding to the terrain mode, and each auxiliary driving device can maximally improve the Stability and safety of the vehicle under the terrain condition.
However, the driving experience of the driver is not facilitated by the manual input mode of the driver, and the possibility of untimely reaction also exists; some manufacturers have implemented sensors in the body of the vehicle to obtain certain characteristics of the vehicle subsystems, such as wheel acceleration, wheel slip or steering force estimates, of the terrain over which the vehicle is traveling, and based on these measurements, control algorithms built into the electronic control unit determine the terrain type that is most likely to meet these characteristics. However, this method does not provide advance prevention for the terrain over which the vehicle will travel, i.e., does not provide advance preparation for the poor road surface that may be present, which is not favorable for the driver's driving experience.
On the other hand, the automatic topographic identification technology adopted by some vehicle types in the market at present is basically used for acquiring point cloud data through a laser radar for identification, and the problem of low identification accuracy exists.
Disclosure of Invention
The invention aims to provide a method, a system, equipment and a storage medium for identifying the terrain of an automobile, which aim to realize accurate identification of the terrain where the automobile is going to run, facilitate the preparation of each subsystem of the automobile in advance and improve the reliability of an all-terrain control system of the automobile.
In order to achieve the above object, the present invention provides a terrain recognition method for an automobile, the method comprising the steps of:
s10, point cloud data output by at least one vehicle-mounted laser radar and image information output by at least one vehicle-mounted camera are obtained;
s20, preprocessing the sampling points in the point cloud data to obtain second point cloud data; preprocessing the image information to obtain second image information;
s30, clustering the second point cloud data to obtain a point cluster corresponding to the second point cloud data;
s40, classifying the point clusters corresponding to the second point cloud data by using a first classifier to obtain first terrain information;
s50, extracting the features of the second image information based on a pre-trained deep neural network to obtain an image feature vector corresponding to the second image information;
s60, classifying the image feature vectors corresponding to the second image information by using a second classifier to obtain second topographic information;
s70, obtaining map information of the position of the vehicle, wherein the map information comprises third topographic information;
and S80, fusing the first topographic information, the second topographic information and the third topographic information to obtain a topographic identification result.
Preferably, the point cloud data is a set consisting of a plurality of sampling points, and the attribute information corresponding to each sampling point at least comprises laser reflection intensity; in step S20, preprocessing the sampling points in the point cloud data to obtain second point cloud data, which specifically includes:
removing the sampling points of which the laser reflection intensity is lower than an intensity threshold value in the point cloud data;
and taking the rest sampling points in the point cloud data as second point cloud data.
Preferably, the point cloud data is a set consisting of a plurality of sampling points, and the attribute information corresponding to each sampling point at least comprises laser reflection intensity; in step S20, preprocessing the sampling points in the point cloud data to obtain second point cloud data, which specifically includes:
removing the sampling points of which the laser reflection intensity is lower than an intensity threshold value in the point cloud data;
processing the rest sampling points in the point cloud data by using a median filter;
and taking the rest sampling points obtained after the processing as second point cloud data.
Preferably, in step S20, the image information is subjected to adaptive wiener filtering to obtain second image information.
Preferably, in step S30, sampling points in the second point cloud data having a density greater than a preset density threshold within a preset search radius are classified into one class, so as to obtain a point cluster corresponding to the second point cloud data.
Preferably, the pre-trained deep neural network in step S50 is obtained by training as follows:
inputting a training sample into a pre-established initial image feature extraction network based on a deep neural network, wherein the training sample comprises a sample image and a mark for representing the terrain of the sample image;
and taking the sample image as input, taking the mark for representing the terrain of the sample image as expected output, training the initial image feature extraction network, and obtaining the trained image feature extraction network.
Preferably, in step S80, when the first topographic information and the second topographic information are the same, the first topographic information is used as a topographic identification result;
when the first terrain information and the third terrain information are the same, taking the first terrain information as a terrain identification result;
when the second topographic information and the third topographic information are the same, taking the second topographic information as a topographic identification result;
and when the first topographic information, the second topographic information and the third topographic information are different from each other, sending prompt information for manually selecting a topographic mode to a display screen.
Preferably, the first topographic information, the second topographic information and the third topographic information each include at least a road, a snow land, a mud land, a sand land or a desert.
Preferably, the first classifier and the second classifier both use a random forest classification algorithm.
Preferably, the preset search radius is 30 cm and the preset density threshold is 50.
To achieve the above object, the present invention also provides a terrain recognition system for an automobile, the system comprising:
the point cloud data and image information acquisition module is used for acquiring point cloud data output by at least one vehicle-mounted laser radar and image information output by at least one vehicle-mounted camera;
the preprocessing module is used for preprocessing the sampling points in the point cloud data to obtain second point cloud data; preprocessing the image information to obtain second image information;
the point cloud clustering module is used for clustering the second point cloud data to obtain a point cluster corresponding to the second point cloud data;
the first topographic information output module is used for classifying the point clusters corresponding to the second point cloud data by using a first classifier to obtain first topographic information;
the image feature extraction module is used for extracting features of the second image information based on a pre-trained deep neural network to obtain an image feature vector corresponding to the second image information;
the second topographic information output module is used for classifying the image feature vectors corresponding to the second image information by using a second classifier to obtain second topographic information;
the map information acquisition module is used for acquiring map information of the position of the vehicle, and the map information comprises third topographic information;
and the terrain result output module is used for fusing the first terrain information, the second terrain information and the third terrain information to obtain a terrain recognition result.
In order to achieve the above object, the present invention also provides a terrain recognition apparatus for an automobile, comprising:
a processor;
a memory having stored therein executable instructions of the processor;
wherein the processor is configured to perform the steps of any of the above-described methods of terrain identification for automobiles via execution of the executable instructions.
The present invention also provides a computer readable storage medium storing a program which, when executed, performs the steps of any of the above-described methods for terrain recognition for a vehicle.
Compared with the prior art, the invention has the following advantages and prominent effects:
the method, the system, the equipment and the storage medium for recognizing the terrain of the automobile provided by the invention utilize a vehicle-mounted laser radar to obtain point cloud data consisting of a plurality of terrain surface information sampling points, and the point cloud data is processed to obtain first terrain information; acquiring image information of a terrain surface by using a camera, and preprocessing, characteristic extraction and classification are carried out on the image information to obtain second terrain information; the position of the vehicle is used for acquiring third terrain information, and the final terrain recognition result is obtained by fusing the three terrain information, so that accurate recognition of the terrain where the vehicle is going to run is realized, each subsystem related to the vehicle can be prepared in advance, the vehicle can stably and safely pass through dangerous terrain, and the reliability of the all-terrain vehicle control system is improved; on the other hand, the user does not need to manually switch the terrain mode, and driving comfort is improved.
Drawings
Other features, objects and advantages of the present invention will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, with reference to the accompanying drawings.
FIG. 1 is a schematic flow chart illustrating a method for terrain recognition of an automobile according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of a terrain recognition system for an automobile according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a terrain recognition apparatus for a vehicle according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a computer-readable storage medium according to an embodiment of the disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar structures, and thus their repetitive description will be omitted.
As shown in fig. 1, the embodiment of the invention discloses a terrain recognition method for an automobile, which comprises the following steps:
and S10, acquiring point cloud data output by at least one vehicle-mounted laser radar and image information output by at least one vehicle-mounted camera. Specifically, at least one laser radar and at least one camera are mounted at least one location of the host vehicle for collecting topographical surface information. The camera may also be a camera. The lidar includes but is not limited to a 4-line lidar, a 16-line lidar or a 64-line lidar, and in this embodiment, the installation parameters of the lidar and the camera on the vehicle body are not specifically limited, and the installation parameters include but are not limited to an installation position, an installation angle or a height from the ground, and may be set accordingly as required in specific implementation. For example, the installation position may be the left side, the right side or the center position of the automobile head, and the installation angle may be defined as: the included angle between the front face of the laser radar and the front of the vehicle is 60 degrees, and the height of the laser radar from the ground can be set to be 30-50 centimeters.
In addition, the present invention does not specifically limit the acquisition parameters of the lidar, where the acquisition parameters include, but are not limited to, an acquisition period or the number of point cloud data packets to be acquired in a unit acquisition period, and the acquisition period of the lidar may be preset according to hardware parameters of the lidar, for example, if the time of one rotation of the lidar is 200 milliseconds, the acquisition period of the lidar is set to 200 milliseconds.
In this embodiment, the number of point cloud data packets to be acquired in a unit acquisition period may be preset for each laser radar of the vehicle body, and the number of point cloud data packets to be acquired in the unit acquisition period is the number of point cloud data packets that can be normally acquired by the laser radar in each acquisition period. The number of point cloud data packets to be acquired by the laser radar in a unit acquisition period can also be preset according to hardware parameters of the laser radar, for example, for a 64-line laser radar, each line of laser radar can acquire 6 point cloud data packets under a normal condition in each acquisition period, and then the 64-line laser radar normally acquires 64 × 6-384 point cloud data packets in the unit acquisition period, so that 384 is the number of point cloud data packets to be acquired in the unit acquisition period of the 64-line laser radar.
The point cloud data output by the laser radar is a set consisting of a plurality of sampling points, the attribute information corresponding to each sampling point at least comprises laser reflection intensity, and in other embodiments, the attribute information corresponding to each sampling point also can comprise three-dimensional coordinates, elevations and the like of the sampling points; the point cloud data is stored in a file in LAS format, i.e., LAS file format. The image information output by the camera is composed of a plurality of image frames.
S20, preprocessing the sampling points in the point cloud data collected in the step S10 to obtain second point cloud data; and pre-processes the image information collected in step S10 to obtain second image information. Specifically, for the sample points in the point cloud data, the sample points in which the laser reflection intensity is lower than the intensity threshold are removed, and then the remaining sample points are taken as the second point cloud data.
For example, the intensity threshold in this embodiment is 10. Then the sampling points with the laser reflection intensity lower than 10 are removed, and the remaining sampling points are retained as the second point cloud data, because the sampling points with the laser reflection intensity lower than the intensity threshold belong to noise points, which may be external environmental factors such as moving vehicles, pedestrians and trees obstructing or obscuring the scanned terrain, and will affect the accuracy of terrain identification, so that the noise points need to be removed.
For the image information acquired by the camera, in this embodiment, an image frame in the image information is acquired at any time and stored in the memory, and then the image frame is subjected to adaptive wiener filtering preprocessing for removing noise points in the image frame to obtain second image information, that is, a preprocessed image frame. It can be understood that the adaptive wiener filtering preprocessing algorithm is an algorithm widely studied in the prior art, and is not described herein again.
S30, clustering the second point cloud data to obtain a point cluster corresponding to the second point cloud data; specifically, based on the sampling points in the second point cloud data obtained in step S20, the sampling points within the preset search radius and having a density greater than the preset density threshold are classified into one class, and a clustering result, that is, a point cluster corresponding to the second point cloud data is formed. In this embodiment, the preset search radius is 30 cm, and the preset density threshold is 50. The present invention does not specifically limit the above parameters, and in other embodiments, the parameters may be set as needed.
And S40, classifying the point clusters corresponding to the second point cloud data by using a first classifier to obtain first terrain information. The first classifier in this embodiment adopts a random forest classification algorithm, the number of trees in the random forest classification algorithm is 200, the depth of the trees is 40, and the dimension of the splitting attribute randomly selected by each splitting node is the number rounded by the arithmetic square root of the total dimension of the original feature. Before the point clusters corresponding to the second point cloud data are classified, the random forest classification algorithm completes training by using training samples, namely, the training samples are input into the random forest classification algorithm, and the training samples comprise sample point clusters formed by sample point cloud data and marks for representing the landforms of the sample point clusters; and taking the sample point cluster as input, taking the mark for representing the terrain of the sample point cluster as expected output, and training the random forest classification algorithm. And after the training is finished, testing the trained random forest classification algorithm by using the test sample.
The random forest classification algorithm is a multi-classifier composed of a plurality of mutually independent decision trees, each decision tree is equivalent to an independent classifier, and the training process of a single decision tree is as follows: carrying out N times of random sampling in the sample point cluster with the size of N according to the replaced bagging sampling rule to obtain a sample point cluster set { theta }k,MAnd i k is 1,2, …, N, wherein k represents the current sample, and the sample point cluster is used as the training sample of the root node of the decision tree. The decision tree randomly selects M (M) in the M-dimensional characteristic attributes of the current node<M) dimension, and calculating their index of degree of uncertainty of the degree of accuracy, choose the degree of uncertainty of the degree of uncertainty using the criterion of minimum degree of uncertainty of the degree of accuracy of the degree of uncertainty of the degree of accuracyAnd the characteristic of the minimum index is taken as the splitting attribute of the node, and the current tree is divided into a left sub-tree and a right sub-tree at the node by a splitting function, so that the process is circulated until the current tree can not be split any more or the leaf node is reached. Because the training of the decision tree is a process of two random choices, the depth of the decision tree can reach the maximum, and the overfitting cannot be carried out in the training process. The characteristic attributes comprise point cluster density, point cluster elevation difference and point cluster elevation standard difference.
And after the point clusters corresponding to the second point cloud data are input into a random forest classification algorithm, giving a category judgment to each decision tree, finally, integrating the single classification results of all the decision trees by the random forest, and taking the category with the largest occurrence frequency of the single classification result as the classification result of the point cluster.
S50, feature extraction is performed on the second image information based on the pre-trained deep neural network, and an image feature vector corresponding to the second image information is obtained.
The pre-trained deep neural network is obtained by training in the following way:
inputting a training sample into a pre-established initial image feature extraction network based on a deep neural network, wherein the training sample comprises a sample image and a mark for representing the terrain of the sample image;
and taking the sample image as input, taking the mark for representing the terrain of the sample image as expected output, training the initial image feature extraction network, and obtaining the trained image feature extraction network.
In this embodiment, 50 layers of residual error neural networks are adopted to extract a feature vector of second image information, the extracted image features include local height difference, RGB intensity and an average value of each component of a color space, the 50 layers of residual error neural networks specifically include a convolutional layer, a pooling layer and a full-link layer, in the training process, the training is finished when the number of training iterations is greater than a preset number of iterations, and convolutional layer parameters and pooling layer parameters of the 50 layers of residual error neural networks are stored as an image feature extraction network model. In this embodiment, the preset iteration number is 1000, and it should be noted that the preset iteration number is not limited in the present invention, and may be set as needed in specific implementation.
S60, the second classifier is used to classify the image feature vector corresponding to the second image information, and second topographic information is obtained. The second classifier also adopts a random forest classification algorithm, the number of trees of the random forest classification algorithm is 200, and the depth of the trees is 80. The specific process of the random forest classification algorithm classification is described in step S40, and is not described herein again, but only when the random forest classification algorithm is trained, the training sample includes sample image information and a label for representing a terrain to which the sample image information belongs, and the sample image information is used as an input, and the label for representing the terrain to which the sample image information belongs is used as an expected output.
S70, map information of the position of the host vehicle is acquired, and the map information includes third topographic information. Specifically, map information of the position of the host vehicle is obtained from an onboard GPS positioning system, and the map information indicates third topographic information of the position of the host vehicle, such as a road, a snow land, a mud land, a sand land, or a desert.
And S80, fusing the first topographic information, the second topographic information and the third topographic information to obtain a topographic identification result.
Specifically, when the first topographic information and the second topographic information are the same, the first topographic information is used as a topographic identification result;
when the first topographic information and the third topographic information are the same, taking the first topographic information as a topographic identification result;
when the second topographic information and the third topographic information are the same, taking the second topographic information as a topographic identification result;
and when the first topographic information, the second topographic information and the third topographic information are different from each other, sending prompt information for manually selecting a topographic mode to a display screen.
The first topographic information, the second topographic information and the third topographic information include at least road, snow, mud, sand or desert. The topographic information in the present invention is not limited to the above five types of topographic information, and other topographic information also belongs to the protection scope of the present invention.
In some optional implementation manners of this embodiment, the preprocessing operation performed on the sampling points in the collected point cloud data in step S20 specifically includes removing sampling points in which the laser reflection intensity is lower than the intensity threshold, processing the remaining sampling points by using a median filter, and then taking the remaining sampling points after processing as the second point cloud data. Since some noise points may still remain in the second point cloud data after removing the sampling points with the laser reflection intensity lower than the intensity threshold, the present embodiment performs a secondary filtering process by using a median filter, effectively filters the remaining noise, and retains useful edge information of the original data, so that the surrounding data can be closer to the true value, thereby facilitating the subsequent processing. The window width of the median filter in this embodiment is 7. The specific process of filtering by the median filter can be realized by using the prior art, and the embodiment will not be described in detail.
In some optional implementations of this embodiment, step S70 further includes: the map information further includes weather information, and it is determined whether the second topographic information and the third topographic information satisfy a preset constraint output condition, and if the preset constraint output condition is satisfied, in step S80, an output result in the preset constraint output condition is directly used as a topographic identification result.
The preset constraint output conditions in this embodiment include:
if the second topographic information is a desert and the third topographic information is a road, taking the third topographic information as an output result;
and if the second topographic information is mud land, the weather information is rainy day, and the third topographic information is road, the second topographic information is used as an output result.
According to the method and the device, accurate identification and judgment of more special scenes are realized through the limitation of the preset constraint output condition, the identification error possibly occurring under the condition of special terrain is reduced, and the accuracy of terrain identification is improved. It should be noted that, the preset constraint output condition is not specifically limited, and in the specific implementation, the preset constraint output condition may be specifically set according to specific requirements.
As shown in fig. 2, an embodiment of the present invention further discloses a terrain recognition system for an automobile, which includes:
a point cloud data and image information obtaining module 21, configured to obtain point cloud data output by at least one vehicle-mounted laser radar and image information output by at least one vehicle-mounted camera;
the preprocessing module 22 is configured to preprocess the sampling points in the point cloud data to obtain second point cloud data; preprocessing the image information to obtain second image information;
a point cloud clustering module 23, configured to cluster the second point cloud data to obtain a point cluster corresponding to the second point cloud data;
a first topographic information output module 24, configured to classify the point clusters corresponding to the second point cloud data by using a first classifier, so as to obtain first topographic information;
an image feature extraction module 25, configured to perform feature extraction on the second image information based on a pre-trained deep neural network to obtain an image feature vector corresponding to the second image information;
a second topographic information output module 26, configured to classify the image feature vector corresponding to the second image information by using a second classifier, and obtain second topographic information;
a map information acquisition module 27 configured to acquire map information of a position of the vehicle, where the map information includes third topographic information;
and a terrain result output module 28, configured to fuse the first terrain information, the second terrain information, and the third terrain information to obtain a terrain recognition result.
It is understood that the vehicle terrain recognition system of the present invention may also include other existing functional modules that support the operation of the vehicle terrain recognition system. The terrain recognition system of the vehicle shown in fig. 2 is only an example, and should not impose any limitation on the functionality and scope of use of embodiments of the present invention.
The terrain identification system of the automobile in this embodiment is used for implementing the above method for identifying the terrain of the automobile, so for the specific implementation steps of the terrain identification system of the automobile, reference may be made to the above description of the method for identifying the terrain of the automobile, and details are not repeated here.
The embodiment of the invention also discloses terrain identification equipment of the automobile, which comprises a processor and a memory, wherein the memory stores the executable instruction of the processor; the processor is configured to perform the steps of the above described method of terrain recognition of a vehicle via execution of executable instructions. Fig. 3 is a schematic structural diagram of a terrain recognition apparatus for an automobile according to the present disclosure. An electronic device 600 according to this embodiment of the invention is described below with reference to fig. 3. The electronic device 600 shown in fig. 3 is only an example and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 3, the electronic device 600 is embodied in the form of a general purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one memory unit 620, a bus 630 connecting the different platform components (including the memory unit 620 and the processing unit 610), a display unit 640, etc.
Wherein the memory unit stores program code which can be executed by the processing unit 610 such that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention as described in the above-mentioned topographic identification method part of the present description of the vehicle. For example, processing unit 610 may perform the steps as shown in fig. 1.
The storage unit 620 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM)6201 and/or a cache memory unit 6202, and may further include a read-only memory unit (ROM) 6203.
The memory unit 620 may also include a program/utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 600, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 600 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 650. Also, the electronic device 600 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via the bus 630. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, to name a few.
The invention also discloses a computer readable storage medium for storing a program, which when executed implements the steps in the terrain recognition method for a vehicle. In some possible embodiments, the various aspects of the invention may also be implemented in the form of a program product comprising program code for causing a terminal device to perform the steps according to various exemplary embodiments of the invention described in the description above for the method for terrain recognition of a vehicle when the program product is run on the terminal device.
As shown above, when the program of the computer-readable storage medium of this embodiment is executed, the laser radar is used to obtain the first terrain information of the terrain where the vehicle is going to travel, the camera is used to obtain the second terrain information, the position where the vehicle is located is used to obtain the third terrain information, and the three terrain information are fused to obtain the final terrain recognition result, so that the accurate recognition of the terrain where the vehicle is going to travel is realized, various subsystems related to the vehicle can be prepared in advance, the vehicle can conveniently and safely pass through dangerous terrain, and the reliability of the all-terrain control system of the vehicle is improved.
Fig. 4 is a schematic structural diagram of a computer-readable storage medium of the present invention. Referring to fig. 4, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
According to the method, the system, the equipment and the storage medium for recognizing the terrain of the automobile, provided by the embodiment of the invention, point cloud data consisting of a plurality of terrain surface information sampling points are obtained by utilizing a vehicle-mounted laser radar, and first terrain information is obtained by processing the point cloud data; acquiring image information of a terrain surface by using a camera, and preprocessing, characteristic extraction and classification are carried out on the image information to obtain second terrain information; the position of the vehicle is used for acquiring third terrain information, and the final terrain recognition result is obtained by fusing the three terrain information, so that accurate recognition of the terrain where the vehicle is going to run is realized, each subsystem related to the vehicle can be prepared in advance, the vehicle can stably and safely pass through dangerous terrain, and the reliability of the all-terrain vehicle control system is improved; on the other hand, the user does not need to manually switch the terrain mode, and driving comfort is improved.
The foregoing is a more detailed description of the invention in connection with specific preferred embodiments and it is not intended that the invention be limited to these specific details. For those skilled in the art to which the invention pertains, several simple deductions or substitutions can be made without departing from the spirit of the invention, and all shall be considered as belonging to the protection scope of the invention.
Claims (12)
1. A terrain recognition method for an automobile, comprising the steps of:
s10, point cloud data output by at least one vehicle-mounted laser radar and image information output by at least one vehicle-mounted camera are obtained;
s20, the point cloud data is a set composed of a plurality of sampling points, the attribute information corresponding to each sampling point at least comprises laser reflection intensity, the sampling points with the laser reflection intensity lower than an intensity threshold value in the point cloud data are removed, and the rest sampling points in the point cloud data are used as second point cloud data; preprocessing the image information to obtain second image information;
s30, clustering the second point cloud data to obtain a point cluster corresponding to the second point cloud data;
s40, classifying the point clusters corresponding to the second point cloud data by using a first classifier to obtain first terrain information;
s50, extracting the features of the second image information based on a pre-trained deep neural network to obtain an image feature vector corresponding to the second image information;
s60, classifying the image feature vectors corresponding to the second image information by using a second classifier to obtain second topographic information;
s70, obtaining map information of the position of the vehicle, wherein the map information comprises third topographic information;
and S80, fusing the first topographic information, the second topographic information and the third topographic information to obtain a topographic identification result.
2. The method for recognizing the terrain of an automobile according to claim 1, wherein the removing of the sampling points in the point cloud data in which the laser reflection intensity is lower than the intensity threshold and the taking of the remaining sampling points in the point cloud data as the second point cloud data specifically includes:
removing the sampling points of which the laser reflection intensity is lower than an intensity threshold value in the point cloud data;
processing the rest sampling points in the point cloud data by using a median filter;
and taking the rest sampling points obtained after the processing as second point cloud data.
3. The method for recognizing a topography of an automobile according to claim 1, wherein in step S20, said image information is subjected to an adaptive wiener filter process to obtain second image information.
4. The terrain recognition method of claim 1, wherein in step S30, sampling points in the second point cloud data having a density greater than a preset density threshold within a preset search radius are classified into one class, and a point cluster corresponding to the second point cloud data is obtained.
5. The terrain recognition method of claim 1, wherein the pre-trained deep neural network in step S50 is trained by:
inputting a training sample into a pre-established initial image feature extraction network based on a deep neural network, wherein the training sample comprises a sample image and a mark for representing the terrain of the sample image;
and taking the sample image as input, taking the mark for representing the terrain of the sample image as expected output, training the initial image feature extraction network, and obtaining the trained image feature extraction network.
6. The terrain recognition method of claim 1, wherein in step S80, when the first terrain information and the second terrain information are the same, the first terrain information is used as a terrain recognition result;
when the first terrain information and the third terrain information are the same, taking the first terrain information as a terrain identification result;
when the second topographic information and the third topographic information are the same, taking the second topographic information as a topographic identification result;
and when the first topographic information, the second topographic information and the third topographic information are different from each other, sending prompt information for manually selecting a topographic mode to a display screen.
7. The terrain recognition method of an automobile of claim 1, wherein each of the first terrain information, the second terrain information, and the third terrain information includes at least a road, a snow land, a mud land, a sand land, or a desert.
8. The terrain recognition method of claim 1, wherein the first classifier and the second classifier both employ a random forest classification algorithm.
9. The terrain recognition method of an automobile of claim 4, wherein the predetermined search radius is 30 cm and the predetermined density threshold is 50.
10. A terrain recognition system for an automobile, comprising:
the point cloud data and image information acquisition module is used for acquiring point cloud data output by at least one vehicle-mounted laser radar and image information output by at least one vehicle-mounted camera;
the point cloud data is a set consisting of a plurality of sampling points, the attribute information corresponding to each sampling point at least comprises laser reflection intensity, and the preprocessing module is used for removing the sampling points with the laser reflection intensity lower than an intensity threshold value in the point cloud data and taking the rest sampling points in the point cloud data as second point cloud data; preprocessing the image information to obtain second image information;
the point cloud clustering module is used for clustering the second point cloud data to obtain a point cluster corresponding to the second point cloud data;
the first topographic information output module is used for classifying the point clusters corresponding to the second point cloud data by using a first classifier to obtain first topographic information;
the image feature extraction module is used for extracting features of the second image information based on a pre-trained deep neural network to obtain an image feature vector corresponding to the second image information;
the second topographic information output module is used for classifying the image feature vectors corresponding to the second image information by using a second classifier to obtain second topographic information;
the map information acquisition module is used for acquiring map information of the position of the vehicle, and the map information comprises third topographic information;
and the terrain result output module is used for fusing the first terrain information, the second terrain information and the third terrain information to obtain a terrain recognition result.
11. An apparatus for recognizing topography of an automobile, comprising:
a processor;
a memory having stored therein executable instructions of the processor;
wherein the processor is configured to perform the steps of the method of terrain identification of a vehicle according to any one of claims 1 to 9 via execution of the executable instructions.
12. A computer-readable storage medium storing a program, wherein the program is executed to implement the steps of the method for recognizing a topography of a vehicle according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910462467.0A CN110188687B (en) | 2019-05-30 | 2019-05-30 | Method, system, device and storage medium for identifying terrain of automobile |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910462467.0A CN110188687B (en) | 2019-05-30 | 2019-05-30 | Method, system, device and storage medium for identifying terrain of automobile |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110188687A CN110188687A (en) | 2019-08-30 |
| CN110188687B true CN110188687B (en) | 2021-08-20 |
Family
ID=67718948
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910462467.0A Active CN110188687B (en) | 2019-05-30 | 2019-05-30 | Method, system, device and storage medium for identifying terrain of automobile |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110188687B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110793512A (en) * | 2019-09-11 | 2020-02-14 | 上海宾通智能科技有限公司 | Pose recognition method and device, electronic equipment and storage medium |
| CN111291697B (en) * | 2020-02-19 | 2023-11-21 | 阿波罗智能技术(北京)有限公司 | Method and device for detecting obstacles |
| CN112698301B (en) * | 2020-12-11 | 2024-06-28 | 中国科学院微电子研究所 | Laser radar target identification method, device and equipment based on distance strength association |
| CN113557528B (en) * | 2021-03-30 | 2023-11-28 | 商汤国际私人有限公司 | Method, device and system for generating point cloud completion network and processing point cloud data |
| CN113190015A (en) * | 2021-05-20 | 2021-07-30 | 中国第一汽车股份有限公司 | Terrain recognition method and device, computer equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016170330A1 (en) * | 2015-04-24 | 2016-10-27 | Oxford University Innovation Limited | Processing a series of images to identify at least a portion of an object |
| CN106527427A (en) * | 2016-10-19 | 2017-03-22 | 东风汽车公司 | Automatic driving sensing system based on highway |
| CN107031600A (en) * | 2016-10-19 | 2017-08-11 | 东风汽车公司 | Automated driving system based on highway |
| CN207473410U (en) * | 2017-11-16 | 2018-06-08 | 尹新通 | A kind of automobile intelligent servomechanism |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9062983B2 (en) * | 2013-03-08 | 2015-06-23 | Oshkosh Defense, Llc | Terrain classification system for a vehicle |
-
2019
- 2019-05-30 CN CN201910462467.0A patent/CN110188687B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2016170330A1 (en) * | 2015-04-24 | 2016-10-27 | Oxford University Innovation Limited | Processing a series of images to identify at least a portion of an object |
| CN106527427A (en) * | 2016-10-19 | 2017-03-22 | 东风汽车公司 | Automatic driving sensing system based on highway |
| CN107031600A (en) * | 2016-10-19 | 2017-08-11 | 东风汽车公司 | Automated driving system based on highway |
| CN207473410U (en) * | 2017-11-16 | 2018-06-08 | 尹新通 | A kind of automobile intelligent servomechanism |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110188687A (en) | 2019-08-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110188687B (en) | Method, system, device and storage medium for identifying terrain of automobile | |
| US11287823B2 (en) | Mapping active and inactive construction zones for autonomous driving | |
| CN108345822B (en) | Point cloud data processing method and device | |
| JP5468108B2 (en) | Method and system for detecting road terrain for a driver assistance system | |
| US10223910B2 (en) | Method and apparatus for collecting traffic information from big data of outside image of vehicle | |
| EP2096602B1 (en) | Methods and apparatus for runway segmentation using sensor analysis | |
| CN110632610A (en) | Autonomous vehicle localization using gaussian mixture model | |
| CN111177288A (en) | System for deriving autonomous vehicle enabled drivable maps | |
| CN102208012A (en) | Scene matching reference data generation system and position measurement system | |
| CN111291697A (en) | Method and device for recognizing obstacle | |
| WO2020252971A1 (en) | Intelligent driving control method and apparatus, and electronic device | |
| CN110458050B (en) | Vehicle cut-in detection method and device based on vehicle-mounted video | |
| US10635917B1 (en) | Method and device for detecting vehicle occupancy using passenger's keypoint detected through image analysis for humans' status recognition | |
| DE112018004953T5 (en) | INFORMATION PROCESSING DEVICE, INFORMATION PROCESSING PROCESS, PROGRAM AND MOVING BODY | |
| CN112750323A (en) | Management method, apparatus and computer storage medium for vehicle safety | |
| CN113640014B (en) | Method and device for constructing test scene of automatic driving vehicle and readable storage medium | |
| CN116829919A (en) | Automatic driving system testing method, testing system and storage medium based on aerial survey data | |
| CN115279643A (en) | On-board active learning method and apparatus for training a perception network of an autonomous vehicle | |
| CN115923820A (en) | Scene data collection method and device for automatic driving system of vehicle | |
| CN114821517A (en) | Method and system for learning neural networks to determine vehicle poses in an environment | |
| CN115129886A (en) | Driving scene recognition method and device and vehicle | |
| CN113752940B (en) | Control method, equipment, storage medium and device for tunnel entrance and exit lamp | |
| CN115578703A (en) | Laser perception fusion optimization method, device and equipment and readable storage medium | |
| US11262201B2 (en) | Location-based vehicle operation | |
| CN114722931A (en) | Vehicle-mounted data processing method and device, data acquisition equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PP01 | Preservation of patent right |
Effective date of registration: 20230131 Granted publication date: 20210820 |
|
| PP01 | Preservation of patent right | ||
| PD01 | Discharge of preservation of patent |
Date of cancellation: 20240108 Granted publication date: 20210820 |
|
| PD01 | Discharge of preservation of patent | ||
| PP01 | Preservation of patent right |
Effective date of registration: 20240227 Granted publication date: 20210820 |
|
| PP01 | Preservation of patent right |