CN113610883B

CN113610883B - Point cloud processing system and method, computer device, and storage medium

Info

Publication number: CN113610883B
Application number: CN202110489032.2A
Authority: CN
Inventors: 不公告发明人
Original assignee: New Drive Chongqing Intelligent Automobile Co ltd
Current assignee: New Drive Chongqing Intelligent Automobile Co ltd
Priority date: 2021-04-30
Filing date: 2021-04-30
Publication date: 2022-04-08
Anticipated expiration: 2041-04-30
Also published as: CN113610883A

Abstract

The application relates to a point cloud processing system and method, a computer device and a storage medium. The system comprises: the device comprises a central processing unit, an internal memory and a digital signal processor; the central processing unit is used for acquiring the point cloud data, preprocessing the point cloud data to obtain a corresponding two-dimensional image, storing the two-dimensional image into the internal memory and sending a control signal to the digital signal processor; and the digital signal processor is used for responding to the control signal, transmitting the two-dimensional image in the internal memory to a random access memory of the digital signal processor for image processing based on a direct memory access mechanism, and transmitting the processed image back to the internal memory. By adopting the system, the point cloud processing speed can be increased, and the real-time requirement can be better met.

Description

Point cloud processing system and method, computer device, and storage medium

Technical Field

The present disclosure relates to the field of point cloud processing technologies, and in particular, to a point cloud processing system and method, a computer device, and a storage medium.

Background

Laser radar point cloud processing is a key technology used in the fields of unmanned driving perception, path planning and the like, and has the characteristics of large data volume, high processing delay requirement and the like. When a laser radar point cloud processing algorithm is operated on a vehicle-mounted computing platform, a Central Processing Unit (CPU) is usually used for completing the operation, however, due to the fact that the data volume of laser point cloud is large, the problems of long computing time consumption and large CPU resource consumption exist, and the real-time requirement of a system cannot be met.

Disclosure of Invention

In view of the above, it is necessary to provide a point cloud processing system and method, a computer device, and a storage medium capable of accelerating a point cloud processing speed, in view of the above technical problems.

A point cloud processing system, comprising: the device comprises a central processing unit, an internal memory and a digital signal processor;

the central processing unit is used for acquiring point cloud data, preprocessing the point cloud data to obtain a corresponding two-dimensional image, storing the two-dimensional image into the internal memory, and sending a control signal to the digital signal processor;

the digital signal processor is configured to respond to the control signal, transmit the two-dimensional image in the internal memory to a random access memory of the digital signal processor for image processing based on a memory direct access mechanism, and transmit a processed image back to the internal memory.

A method of point cloud processing, the method comprising:

acquiring point cloud data;

preprocessing the point cloud data to obtain a corresponding two-dimensional image, and storing the two-dimensional image into the internal memory;

and sending a control signal to a digital signal processor, transmitting the two-dimensional image in the internal memory to a random access memory of the digital signal processor for image processing by the digital signal processor based on a direct memory access mechanism, and returning the processed image to the internal memory.

A point cloud processing apparatus, the apparatus comprising:

the acquisition module is used for acquiring point cloud data;

the point cloud preprocessing module is used for preprocessing the point cloud data to obtain a corresponding two-dimensional image and storing the two-dimensional image into the internal memory;

and the image processing control module is used for sending a control signal to the digital signal processor, transmitting the two-dimensional image in the internal memory to a random access memory of the digital signal processor for image processing by the digital signal processor based on a direct memory access mechanism, and returning the processed image to the internal memory.

A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:

acquiring point cloud data;

A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:

acquiring point cloud data;

A method of point cloud processing, the method comprising:

receiving a control signal sent by a central processing unit;

responding to the control signal, and transmitting the two-dimensional image in the internal memory to a random access memory of the digital signal processor for image processing based on a direct memory access mechanism; the two-dimensional image is obtained by the central processing unit through point cloud data, is obtained by preprocessing the point cloud data and is stored in the internal memory;

and transmitting the processed image back to the internal memory.

A point cloud processing apparatus, the apparatus comprising:

the receiving module is used for receiving a control signal sent by the central processing unit;

the image processing module is used for responding to the control signal and transmitting the two-dimensional image in the internal memory to a random access memory of the digital signal processor for image processing based on a direct memory access mechanism; the two-dimensional image is obtained by the central processing unit through point cloud data, is obtained by preprocessing the point cloud data and is stored in the internal memory;

and the transmission module is used for transmitting the processed image back to the internal memory.

receiving a control signal sent by a central processing unit;

and transmitting the processed image back to the internal memory.

receiving a control signal sent by a central processing unit;

and transmitting the processed image back to the internal memory.

According to the point cloud processing system and method, the computer equipment and the storage medium, the central processing unit executes the point cloud preprocessing task to convert the point cloud data into the two-dimensional image, so that the subsequent data volume to be processed is greatly reduced; the image processing task is executed through the digital signal processor, and the image processing time can be greatly shortened by utilizing the speed advantage of the digital signal processor; based on a direct memory access mechanism, the data transmission process does not need the participation of a central processing unit, and the high-efficiency data transmission can be realized. Therefore, the central processing unit is only responsible for task scheduling and processing tasks with low time consumption, and image processing tasks with high time consumption are put on the digital signal processor to be executed, so that the resource occupancy of the central processing unit is reduced, the image processing speed is increased, and the real-time requirement of the system is better met. In addition, the digital signal processor supports flexible development of a user-defined operator according to application requirements, a plurality of digital signal processors can run in parallel, resource advantages of the embedded heterogeneous computing platform can be fully exerted, and the embedded heterogeneous computing platform has the advantages of low power consumption and low cost, and is more suitable for vehicle-level application requirements with strict requirements on power consumption and cost.

Drawings

FIG. 1 is a schematic diagram of a point cloud processing system in one embodiment;

FIG. 2 is a schematic flow chart illustrating the steps of preprocessing point cloud data by the CPU to obtain a corresponding two-dimensional image according to one embodiment;

FIG. 3 is a flowchart illustrating a tenth image obtaining step performed by the second DSP branch in one embodiment by performing threshold segmentation based on gray-level histogram optimization on the fifth image;

FIG. 4 is a flowchart illustrating the steps of the CPU determining a first gray level threshold based on the first histogram of gray level values and determining a second gray level threshold based on the second histogram of gray level values according to one embodiment;

fig. 5 is a schematic flowchart illustrating a step of the digital signal processor performing blocking processing on a two-dimensional image block in the internal memory to the random access memory of the digital signal processor according to the image size, the storage address, and the preset block size in one embodiment;

FIG. 6 is a diagram illustrating image segmentation in one embodiment;

FIG. 7 is a diagram illustrating the transmission and processing of block data based on the Ping Pong Buffer mechanism in one embodiment;

FIG. 8 is a schematic diagram of the interaction of a digital signal processor with a central processing unit in one embodiment;

FIG. 9 is a schematic flow chart diagram illustrating a method for point cloud processing according to one embodiment;

FIG. 10 is a schematic flow chart diagram illustrating a method for point cloud processing according to one embodiment;

FIG. 11 is a schematic flow chart diagram illustrating a method for point cloud processing according to one embodiment;

FIG. 12 is a block diagram of a point cloud processing apparatus according to an embodiment;

FIG. 13 is a block diagram of a point cloud processing apparatus according to an embodiment;

FIG. 14 is a diagram showing an internal structure of a computer device in one embodiment;

FIG. 15 is a diagram showing an internal structure of a computer device according to an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

The application provides a point cloud processing system can be applied to on-vehicle computing platform among the intelligent driving system, for example, carries out real-time processing to the laser radar point cloud data that detects among the intelligent driving vehicle operation process, carries out road obstacle detection and road boundary detection then.

In one embodiment, as shown in fig. 1, there is provided a point cloud processing system comprising: central processing unit, internal memory and digital signal processor. The internal memory represents a system memory, and specifically, a double data rate synchronous dynamic random access memory (DDR) can be adopted and can be accessed by the central processing unit. The digital signal processor is internally provided with a random access memory (DRAM) and a direct memory access mechanism (DMA) interface, and the DMA interface can be called to realize data exchange between the internal memory and the random access memory.

And the central processing unit is used for acquiring the point cloud data, preprocessing the point cloud data to obtain a corresponding two-dimensional image, storing the two-dimensional image into the internal memory and sending a control signal to the digital signal processor.

And the digital signal processor is used for responding to the control signal, transmitting the two-dimensional image in the internal memory to a random access memory of the digital signal processor for image processing based on a direct memory access mechanism, and transmitting the processed image back to the internal memory.

In the point cloud processing system, the central processing unit executes the point cloud preprocessing task to convert the point cloud data into the two-dimensional image, so that the subsequent data volume to be processed is greatly reduced; the image processing task is executed through the digital signal processor, and the image processing time can be greatly shortened by utilizing the speed advantage of the digital signal processor; based on a direct memory access mechanism, the data transmission process does not need the participation of a central processing unit, and the high-efficiency data transmission can be realized. Therefore, the central processing unit is only responsible for task scheduling and processing tasks with low time consumption, and image processing tasks with high time consumption are put on the digital signal processor to be executed, so that the resource occupancy of the central processing unit is reduced, the image processing speed is increased, and the real-time requirement of the system is better met. In addition, the digital signal processor supports flexible development of a user-defined operator according to application requirements, a plurality of digital signal processors can run in parallel, resource advantages of the embedded heterogeneous computing platform can be fully exerted, and the embedded heterogeneous computing platform has the advantages of low power consumption and low cost, and is more suitable for vehicle-level application requirements with strict requirements on power consumption and cost.

In one embodiment, the point cloud data includes road information, which may include, but is not limited to, road obstacle information and road boundary information. The pre-processing of the point cloud data may include, but is not limited to, point cloud filtering, point cloud segmentation, and point cloud mapping to convert the three-dimensional point cloud data into a two-dimensional image.

Specifically, a Central Processing Unit (CPU) acquires point cloud data, which can be based on ROS subscribing laser radar node information, and analyzes pcl from ROS MSG, wherein the point cloud data comprises road obstacle information and road boundary information in a PointXYZI format.

In an embodiment, as shown in fig. 2, the step of preprocessing the point cloud data by the central processor to obtain a corresponding two-dimensional image may specifically include the following steps S202 to S208.

S202, determining a coordinate value range according to the coordinates of each point cloud in the point cloud data, and filtering the point cloud data based on the coordinate value range to obtain a first point cloud.

The coordinates of the point cloud are three-dimensional coordinates (including x coordinates, y coordinates and z coordinates), and the coordinate value range comprises the value ranges of the x coordinates, the y coordinates and the z coordinates. And setting reasonable value ranges of the x coordinate, the y coordinate and the z coordinate according to the coordinates of each point cloud, and determining the reasonable value ranges as coordinate value ranges. Point clouds whose coordinates are not within the range are filtered to filter out unwanted noise. The point cloud with coordinates in this range is retained as the first point cloud.

And S204, carrying out point cloud segmentation on the first point cloud, and extracting a second point cloud containing road information.

A first point cloud may be understood as comprising two parts of point clouds, one part of the point cloud comprising road information (or ground information), i.e. a second point cloud, and the other part of the point cloud containing other information than road information (e.g. sky, cloud, etc.), i.e. outliers. The first point cloud may be point cloud segmented, the outer points may be removed, and the second point cloud including the road information may be extracted and retained by using any existing or future-appearing point cloud segmentation algorithm, for example, a random sample consensus (RANSAC) algorithm based on a planar model.

And S206, mapping the second point cloud into a first image according to the height threshold of the obstacle, wherein the first image is an image containing the road obstacle.

The gray value of each pixel in the first image corresponds to the height (i.e., z coordinate) of each point cloud in the second point cloud, and the obstacle height threshold refers to a preset maximum height of the obstacle, it can be understood that the point cloud with the z coordinate less than or equal to the obstacle height threshold may correspond to the obstacle, and the point cloud with the z coordinate greater than the obstacle height threshold corresponds to no obstacle. And mapping the z coordinate smaller than or equal to the height threshold of the obstacle to a gray value 255, and mapping the z coordinate larger than the height threshold of the obstacle to a gray value 0, thereby realizing the mapping of the second point cloud to a first image, wherein the first image is a binary image containing the road obstacle. It should be noted that the obstacle height threshold may be set in combination with actual situations, and is not limited herein.

S208, mapping the second point cloud into a second image according to the road edge height threshold, wherein the second image is an image containing a road boundary; the two-dimensional image includes a first image and a second image.

The gray value of each pixel in the second image corresponds to the height (i.e., z coordinate) of each point cloud in the second point cloud, and the road edge height threshold refers to a preset maximum road edge height. And dividing the z coordinate less than or equal to the road edge height threshold value by the conversion factor to map the z coordinate into a gray value, and mapping the z coordinate greater than the road edge height threshold value into a gray value 0, thereby realizing the mapping of the second point cloud into a second image, wherein the second image is a gray image containing a road boundary. It should be noted that the road edge height threshold may be set in combination with actual situations, and is not limited herein.

After the central processor preprocesses the point cloud data to obtain a first image (represented by I1) and a second image (represented by I2), the first image and the second image are stored in an internal memory (represented by DDR), and a control signal is sent to the digital signal processor so that the digital signal processor can execute subsequent image processing tasks. After receiving the control signal, the digital signal processor transfers the first image and the second image in the internal memory to its own random access memory (denoted by DRAM) based on DMA, and performs image processing in the DRAM.

In one embodiment, a digital signal processor includes: a first digital signal processor branch and a second digital signal processor branch processed in parallel; the first digital signal processor branch is used for carrying out median filtering on the first image to obtain a third image and carrying out expansion operation on the third image to obtain a fourth image when carrying out image processing; the second digital signal processor branch is used for carrying out corrosion operation on the second image to obtain a fifth image when carrying out image processing, and carrying out threshold segmentation based on gray value histogram optimization on the fifth image to obtain a tenth image; the processed image includes a fourth image and a tenth image.

The first digital signal processor branch may include a first digital signal processor (represented by DSP 1) and a second digital signal processor (represented by DSP 2). The DSP1 performs median filtering on I1 to obtain a filtered image, i.e., a third image (denoted by I3). The DSP2 performs a dilation operation on I3 to obtain a dilated image, i.e., a fourth image (denoted by I4). The kernel size used for the median filtering and the dilation operation can be set according to actual requirements, for example, the kernel size for the median filtering is set to [5,5], and the kernel size for the dilation operation is set to [10,30 ].

The second digital signal processor branch may include a third digital signal processor (represented by DSP 3) and a fourth digital signal processor (represented by DSP 4). The DSP3 performs an etching operation on the I1 to obtain an etched image, namely a fifth image (indicated by I5). The DSP4 performs threshold segmentation based on gray-value histogram optimization on I5, and obtains a segmented image, i.e., a tenth image (denoted by I10). The size of the kernel used in the etching operation may be set according to actual requirements, for example, the size of the kernel of the etching operation is set to [10,30 ].

In the above embodiment, the first digital signal processor branch and the second digital signal processor branch are executed in parallel, so that the image containing the road obstacle and the image containing the road boundary can be processed in parallel, and the image processing task can be executed by using the optimization operator embedded in the DSP, thereby improving the image processing speed. The contour characteristics of the road barrier and the road boundary are different, median filtering and expansion operation are carried out on the image containing the road barrier, the contour of the road barrier is favorably highlighted, the accuracy of subsequent road barrier contour detection is improved, the image containing the road boundary is subjected to corrosion operation and threshold segmentation based on gray value histogram optimization, the contour of the road boundary is favorably highlighted, and the accuracy of subsequent road boundary contour detection is improved.

In an embodiment, as shown in fig. 3, the second digital signal processor branches to perform threshold segmentation based on gray-scale value histogram optimization on the fifth image to obtain a tenth image, which may specifically include the following steps S302 to S308.

And S302, dividing the fifth image into a sixth image and a seventh image according to the image height ratio.

The DSP4 performs region of interest (ROI) segmentation on the I5 according to the image height proportion to obtain a sixth image (represented by I6) and a seventh image (represented by I7), the widths of the I6 and the I7 are the same as the width of the I5, and the heights of the I6 and the I7 can be obtained through the following formulas: i6_ height ═ I5_ height a, I7_ height ═ I5_ height-I6_ height. Wherein I5_ height, I6_ height, and I7_ height represent the heights of I5, I6, and I7, respectively; and a represents the image height proportion, the value range is (0,1), and the specific value can be set by combining the actual situation, which is not limited here. For example, if the I5 size is 1000 × 1000, i.e. both width and height are 1000 pixels, and a is 0.4, the I6 size obtained after segmentation is 1000 × 400, and the I7 size is 1000 × 600.

And S304, respectively calculating a first gray value histogram of the sixth image and a second gray value histogram of the seventh image.

The DSP4 obtains the gray value of each pixel in the I6, and counts the gray values of all the pixels to obtain a first gray value Histogram (represented by Histogram 1), wherein the abscissa of the Histogram1 is the gray value, the value range is [0,255], and the ordinate of the Histogram1 is the number of pixels corresponding to each gray value in the I6.

The DSP4 obtains the gray value of each pixel in the I7, and counts the gray values of all the pixels to obtain a second gray value Histogram (represented by Histogram 2), wherein the abscissa of the Histogram2 is the gray value, the value range is [0,255], and the ordinate of the Histogram2 is the number of pixels corresponding to each gray value in the I7.

S306, segmenting the sixth image according to a first gray threshold determined based on the first gray value histogram to obtain an eighth image, and segmenting the seventh image according to a second gray threshold determined by the central processing unit based on the second gray value histogram to obtain a ninth image.

The first gray threshold and the second gray threshold may be determined by the DSP4 or may be determined by the central processing unit. In one embodiment, after DSP4 obtains Histogram1 and Histogram2, it transmits Histogram1 and Histogram2 back to DDR, and a Central Processing Unit (CPU) obtains Histogram1 and Histogram2 from DDR, calculates a first gray threshold (denoted by thresh1) for threshold splitting I6 based on Histogram1, calculates a second gray threshold (denoted by thresh2) for threshold splitting I7 based on Histogram2, and transmits thresh1 and thresh2 to DSP4 through a message passing mechanism using thresh1 and thresh2 as parameters.

The DSP4 segments I6 according to thresh1, and specifically, may convert all the gray-scale values of the pixels with gray-scale values smaller than thresh1 in I6 into 0, and convert all the gray-scale values of the pixels with gray-scale values greater than or equal to thresh1 in I6 into 255, thereby obtaining an eighth image (denoted by I8), where I8 is a binary image.

The DSP4 segments I7 according to thresh2, and specifically, the grayscale values of all pixels in I7 with grayscale values smaller than thresh2 are all converted to 0, and the grayscale values of all pixels in I7 with grayscale values greater than or equal to thresh2 are converted to 255, so as to obtain a ninth image (denoted by I9), where I9 is a binary image.

S308, the eighth image and the ninth image are merged into a tenth image.

The DSP4 performs ROI merging of I8 and I9 to obtain a tenth image (denoted by I10). For example, the size of I8 is 1000 × 400, the size of I9 is 1000 × 600, and the combined I10 size is 1000 × 1000.

In the above embodiment, the processing task of threshold segmentation based on gray value histogram optimization is divided into four subtasks, namely ROI segmentation, gray value histogram calculation, threshold segmentation and ROI merging, and each subtask can be executed by using an optimization operator embedded in the DSP, so that the image processing speed is increased.

In an embodiment, as shown in fig. 4, the determining, by the central processor, a first gray threshold based on the first histogram of gray values and a second gray threshold based on the second histogram of gray values may specifically include the following steps S402 to S408.

S402, acquiring a first gray value histogram and a second gray value histogram from the internal memory.

After DSP4 processes the results to obtain Histogram1 and Histogram2, Histogram1 and Histogram2 are transmitted back to DDR from which the CPU obtains Histogram1 and Histogram 2.

S404, dividing each pixel in the first gray value histogram into a first background pixel and a first target pixel according to a first initial threshold, respectively calculating a first background pixel mean value and a first target pixel mean value, and determining a first final threshold based on the first background pixel mean value and the first target pixel mean value.

The first initial threshold represents a set gray value threshold (denoted by thresh _ last 1) that is traversed between 0 and 255 gray values, considered to belong to the background if the current gray value is less than thresh _ last1, and considered to belong to the target if the current gray value is greater than or equal to thresh _ last 1. Accordingly, each pixel in Histogram1 is divided into a background pixel and a target pixel, i.e., a first background pixel and a first target pixel, the gray value of the first background pixel is less than thresh _ last1, and the gray value of the first target pixel is greater than or equal to thresh _ last 1. the value of thresh _ last1 can be set according to the actual situation, for example, 105.

The first background pixel mean value is the mean value of the gray values of all the first background pixels, and can be calculated by the following formula:

where mean _ b1 represents the first background pixel mean, i represents the gray value, P represents the gray value₁[i]Representing a pixel with a gray value of i in Histogram1The number, num _ b1, represents the total pixel number of the first background pixels.

The first target pixel mean value is the mean value of the gray values of all the first target pixels, and can be calculated by the following formula:

where mean _ o1 represents the first target pixel mean value, i represents the gray value, P represents the gray value₁[i]Indicating the number of pixels having a gray value of i in Histogram1, and num _ o1 indicating the total number of pixels of the first target pixel.

The first gray threshold (thresh1) is determined based on mean _ b1 and mean _ o1, thresh1 may be calculated by the following formula: thresh1 ═ m _ b1+ m _ o 1)/2.

S406, dividing each pixel in the second gray value histogram into a second background pixel and a second target pixel according to a second initial threshold, calculating a second background pixel mean value and a second target pixel mean value respectively, and determining a second gray threshold based on the second background pixel mean value and the second target pixel mean value.

The second initial threshold represents a set gray value threshold (denoted by thresh _ last 2) that is traversed between 0 and 255 gray values, considered to be background if the current gray value is less than thresh _ last2, and considered to be target if the current gray value is greater than or equal to thresh _ last 2. Accordingly, each pixel in Histogram2 is divided into a background pixel and a target pixel, i.e., a second background pixel and a second target pixel, the gray value of the second background pixel is less than thresh _ last2, and the gray value of the second target pixel is greater than or equal to thresh _ last 2. the value of thresh _ last2 can be set according to the actual situation, for example, 105.

The second background pixel mean value is the mean value of the gray values of all the second background pixels, and can be calculated by the following formula:

where mean _ b2 represents the second background pixel mean, i represents the gray value, P represents the gray value₂[i]Represents Histhe number of pixels having a gray value of i in the togram2, and num _ b2 represents the total number of pixels of the second background pixel.

The second target pixel mean value is the mean value of the gray values of all the second target pixels, and can be calculated by the following formula:

where mean _ o2 represents the second target pixel mean value, i represents the gray value, P represents the gray value₂[i]Indicating the number of pixels having a gray value of i in Histogram2, and num _ o2 indicating the total number of pixels of the second target pixel.

Determining a second gray threshold (thresh2) based on mean _ b2 and mean _ o2, thresh2 may be calculated by the following equation: thresh2 ═ m _ b2+ m _ o 2)/2.

S408, the first grayscale threshold and the second grayscale threshold are transmitted to a second digital signal processor branch.

After the CPU calculates thresh1 and thresh2, thresh1 and thresh2 are used as parameters and are transmitted to DSP4 through a message passing mechanism, so that DSP4 performs a threshold segmentation task based on thresh1 and thresh 2.

In the embodiment, the set initial gray value threshold is optimized based on the gray value histogram to obtain the final gray value threshold, so that the background pixel and the target pixel can be more accurately distinguished, and the accuracy of subsequent threshold segmentation is improved.

In one embodiment, the central processing unit is further configured to obtain a fourth image and a tenth image from the internal memory, and perform contour detection on the fourth image and the tenth image respectively to obtain road obstacle contour information and road boundary contour information.

After the DSP has performed all image processing tasks, I4 and I10 are obtained, and I4 and I10 are transmitted back to the DDR. It is understood that I4 represents a processed image containing road obstacles and I4 represents a processed image containing road boundaries. The CPU obtains I4 and I10 from the DDR, then profile detection can be carried out on I4 and I10 respectively by adopting any existing or future-likely profile detection algorithm, such as OpenCV function findContours, so as to obtain road obstacle profile information and road boundary profile information, and finally road obstacle detection and road boundary detection are achieved.

In one embodiment, the digital signal processor obtains the image size and the storage address of the two-dimensional image before obtaining the two-dimensional image to be processed from the internal memory. The digital signal processor transmits the two-dimensional image in the internal memory to a random access memory of the digital signal processor for image processing, and specifically, the two-dimensional image in the internal memory is transmitted to the random access memory of the digital signal processor for block processing according to the image size, the storage address and the preset block size.

The image size includes an image width and an image height. The Tile size represents the size of the smallest data unit (denoted by Tile) of the image processed by the DSP, Tile may be a small rectangular Tile in the image, and the Tile size includes a Tile width and a Tile height. The storage address indicates a storage address of the image in the internal memory, and may specifically be a storage head address, and the storage address of each Tile may be obtained according to the storage head address and the offset corresponding to each Tile, so that each Tile is sequentially transmitted to the DRAM of the DSP for processing.

In the above embodiment, based on the Tile partitioning mechanism, data transmission and data processing can be performed simultaneously, that is, when processing the previous data, the next data can be transmitted simultaneously, so that the image processing speed is further increased.

In one embodiment, a first buffer and a second buffer are provided in a random access memory of the digital signal processor. Specifically, two data buffers are allocated in the DRAM space of the DSP, named Ping Buffer and Pong Buffer, respectively.

As shown in fig. 5, the method for blocking the two-dimensional image in the internal memory by the digital signal processor according to the image size, the storage address, and the preset blocking size may specifically include the following steps S502 to S512.

And S502, determining the serial number and the position information of each block data according to the image size, the storage address and the preset block size.

The image size comprises an image width and an image height, the block size comprises a block width and a block height, and the storage address is specifically a storage first address. Specifically, the image width is subjected to residue taking on the block width to obtain a width remainder, and the image height is subjected to residue taking on the block height to obtain a height remainder; and determining the serial number and the position information of each block data according to the storage initial address, the block width, the block height, the width remainder and the height remainder.

As shown in fig. 6, a schematic diagram of image segmentation in one embodiment is provided. The image size is 1000 × 1000, i.e., the image Width (expressed in Width) and the image Height (expressed in Height) are both 1000. The Tile size is 256 × 128, i.e., the Tile Width (indicated by Tile _ Width) and the Tile Height (indicated by Tile _ Height) are 256 and 128, respectively. The image is segmented to obtain 4 image sub-blocks, which are respectively represented by Part1, Part2, Part3 and Part 4.

Part1 is located at the upper left side of the image, has a Width (denoted by Width _ 1) that can be evenly divided by Tile _ Width, Width _ 1-Width% Tile _ Width 768, and a Height (denoted by Height _ 1) that can be evenly divided by Tile _ Height, Height _ 1-Height% Tile _ Height 896, where% is the remainder sign. Setting the position of the upper left corner of the image as [0,0], the position range of Part1 in the width direction is [0,768], and the position range in the height direction is [0,896 ].

Part2 is located below the left side of the image, has a Width (denoted by Width _ 2) that is evenly divisible by Tile _ Width, Width _ 2-Width% Tile _ Width 768, a Height (denoted by Height _ 2) that is not evenly divisible by Tile _ Height, Height _ 2-Height% Tile _ Height 104, where% is a remainder symbol. The position of Part2 in the width direction is in the value range of [0,768], and the position in the height direction is in the value range of [896,1000 ].

Part3 is located above the right side of the image, has a Width (denoted by Width _ 3) that is not divisible by Tile _ Width, Width _ 3-Width% Tile _ Width-232, a Height (denoted by Height _ 3) that is divisible by Tile _ Height, Height _ 3-Height% Tile _ Height-896, where% is the remainder sign. The position of Part3 in the width direction is in the value range of [768,1000], and the position in the height direction is in the value range of [0,896 ].

Part4 is located below the left side of the image, has a Width (denoted by Width _ 4) that is not evenly divisible by Tile _ Width, a Width _4 ═ Width% Tile _ Width ═ 232, a Height (denoted by Height _ 4) that is not evenly divisible by Tile _ Height, and Height _4 ═ Height% Tile _ Height ═ 104, where% is the remainder symbol. The position of Part4 in the width direction is in the value range of [768,1000], and the position in the height direction is in the value range of [896,1000 ].

After obtaining 4 image sub-blocks, each image sub-block is blocked, Part1 is divided into 21 block data with the size of 256 × 128, Part2 is divided into 3 block data with the size of 256 × 104, Part3 is divided into 7 block data with the size of 232 × 128, and Part4 is 1 block data with the size of 232 × 104, so that the whole image is divided into 32 block data, the block data of Part1, Part2, Part3 and Part4 are numbered in sequence, and the serial number of each block data is determined. For example, for Part1, numbering may be performed in the order of left to right and top to bottom, where Tile1, Tile2 and Tile3 in fig. 6 respectively represent Tile data with the first 3-bit sequence number.

And (3) taking the position [0,0] of the upper left corner of the image as a storage head address, and obtaining the position information of each block data according to the position offset of each block data in the width and height directions. For example, for Tile1, its positional offsets in the width and height directions are 256 and 128, respectively; for Tile2, its positional offsets in the width and height directions are 512 and 128, respectively; for Tile3, the positional offsets in the width and height directions are 768 and 128, respectively.

S504, the first sequence number is used as a first current sequence number, and the block data corresponding to the first current sequence number is transmitted to the first cache according to the position information of the block data corresponding to the first current sequence number.

The first ordinal number is a number sequentially ranked first. As shown in fig. 7, a schematic diagram for performing block data transmission and processing based on the Ping Pong Buffer mechanism in one embodiment is provided. Taking Part1 processing as an example, the chunk data corresponding to the first sequence number is Tile1, Tile1 is transmitted to Ping buffer, and the first current sequence number at this time is the first sequence number.

S506, the second sequence number is used as a second current sequence number, and the block data corresponding to the second current sequence number is transmitted to a second cache according to the position information of the block data corresponding to the second current sequence number.

The second ordinal number is a number sequentially ranked second. Referring to fig. 7, the block data corresponding to the second sequence number is Tile2, Tile2 is transmitted to Pong buffer, and the second current sequence number is the second sequence number.

And S508, processing the block data corresponding to the first current sequence number, and simultaneously transmitting the block data corresponding to the first subsequent sequence number to a first cache according to the position information of the block data corresponding to the first subsequent sequence number, wherein the first subsequent sequence number represents the next sequential sequence number of the second current sequence number.

Referring to fig. 7, the block data corresponding to the first subsequent sequence number is Tile3, the DSP processes Tile1 and transmits Tile3 to Ping buffer, where the first subsequent sequence number is the third sequential sequence number, i.e., the sequence number sequentially arranged in the third order.

And S510, after the processing of the block data corresponding to the first current sequence number is finished, processing the block data corresponding to the second current sequence number, and simultaneously transmitting the block data corresponding to the second subsequent sequence number to a second cache according to the position information of the block data corresponding to the second subsequent sequence number, wherein the second subsequent sequence number represents the next sequential sequence number of the first subsequent sequence number.

Referring to fig. 7, after the DSP has processed Tile1, Tile2 is processed and Tile4 is transmitted to the Pong buffer, where the second subsequent sequence number is the fourth sequential sequence number, i.e., the sequence number sequentially arranged in the fourth place.

And S512, after the processing of the block data corresponding to the second current sequence number is finished, taking the first subsequent sequence number and the second subsequent sequence number as a new first current sequence number and a new second current sequence number respectively, and returning to the step of processing the block data corresponding to the first current sequence number until the processing of the block data corresponding to all the sequence numbers is finished.

After the Tile2 is processed by the DSP, the third order number and the fourth order number are used as the new First current order number and the Second current order number, and the process returns to step S508 to perform the First iteration (First iteration) and step S510 to perform the Second iteration (Second iteration) until all the Tile data corresponding to all the order numbers are processed.

In the above embodiment, based on the Ping Pong Buffer mechanism, the data transmission and data processing tasks can be run in parallel, so that the data transmission and processing speed is further increased, the overall image processing speed is increased, and the time consumption is reduced.

It should be noted that, in the foregoing embodiment, the DSP1 performs median filtering on the I1, the DSP2 performs dilation operation on the I3, the DSP3 performs erosion operation on the I1, and the DSP4 performs ROI segmentation, gray value histogram calculation, threshold segmentation, and ROI merging on the I5, where these image processing tasks may all implement data transmission and processing based on a Tile partitioning mechanism and a Ping Pong Buffer mechanism, thereby greatly shortening the processing time of the point cloud image. The Tile size can be set in conjunction with the actual situation, for example, Tile sizes for dilation and erosion operations are set to 64 × 64, and Tile sizes for median filtering, ROI segmentation, gray value histogram calculation, threshold segmentation, and ROI merging are set to 256 × 128.

In one embodiment, the two-dimensional image of the laser point cloud is processed based on a heterogeneous acceleration platform (SVP platform) for processing intelligent Vision in a Hi3559AV100 chip and a DSP hardware module, wherein the DSP is a programmable hardware acceleration module under the SVP platform, and 4 Cadence Vision P6 DSPs with the main frequency of 700MHz are integrated in the Hi3559AV100 chip.

As shown in fig. 8, a schematic diagram of the interaction between the DSP side and the CPU side in one embodiment is provided. At a DSP end, realizing a user-defined DSP operator based on Tile partitioning; compiling the DSP operator to generate a libdsp _ algo.a library file; and linking the DSP library files to generate four bin files, namely hi _ iram.bin, hi _ sram.bin, hi _ dram0.bin and hi _ dram1.bin, for the CPU end to load and execute tasks.

At the CPU end, executing DSP program initialization, calling HI _ MPI _ SVP _ DSP _ PowerOn to electrify DSP 1-DSP 4 equipment, and calling HI _ MPI _ SVP _ DSP _ LoadBin interface to load the four bin files respectively; calling an HI _ MPI _ SYS _ MmzAlloc interface to allocate MMZ image memories required by the DSP for task processing; calling an HI _ MPI _ SVP _ DSP _ EnableCore interface to enable DSP 1-DSP 4 cores to start working respectively, calling the DSP to perform task processing based on the HI _ MPI _ SVP _ DSP _ RPC and HI _ MPI _ SVP _ DSP _ Query interfaces, namely executing custom operators on DSP 1-DSP 4 respectively, namely executing median filtering on the DSP, executing expansion operation on DSP1, executing corrosion operation on DSP2, and executing threshold segmentation (comprising gray value ROI segmentation, gray value histogram calculation, threshold segmentation and ROI combination) based on gray value ROI optimization on DSP 3; waiting for DSP processing to complete and returning the results back and performing subsequent processing, such as contour detection.

In the embodiment, based on the SVP intelligent computing platform on the Hi3559AV100 chip, the CPU and the DSP are mixed for programming, the CPU is only responsible for task scheduling and processing tasks which are low in time consumption, the image operation part which is high in time consumption is placed on the DSP to be executed, the tasks are executed in parallel among a plurality of DSP cores, and the CPU resource occupancy rate is reduced. In addition, based on the relevant optimization operators in the Cadence Xtensia Imaging (XI) library: the image expansion operation of the custom template size is realized by using an xiDiate _ NxM _ U8 interface, the image erosion operation of the custom template size is realized by using an xiErode _ NxM _ U8 interface, the histogram calculation is realized by using an xiCalcHist _ U8U32 interface, ROI segmentation and combination are realized by using an xiCopyTile interface, threshold segmentation is realized by using an xiThresholdInv _ U8 interface, and the task processing speed is accelerated. Taking two point cloud images with the size of 1000 × 1000 as an example, 497ms is consumed for completing all processing on a CPU, and only 50ms is consumed after DSP is adopted, so that the performance of the point cloud image processing algorithm is greatly improved, the real-time requirement of the system can be met, and the deployment and application of the laser point cloud processing algorithm on an embedded computing platform are realized.

In one embodiment, as shown in fig. 9, a point cloud processing method is provided, which is exemplified by applying the method to the central processing unit of fig. 1, and includes the following steps S902 to S906.

And S902, acquiring point cloud data.

And S904, preprocessing the point cloud data to obtain a corresponding two-dimensional image, and storing the two-dimensional image into an internal memory.

And S906, sending a control signal to the digital signal processor, transmitting the two-dimensional image in the internal memory to a random access memory of the digital signal processor for image processing by the digital signal processor based on a direct memory access mechanism, and returning the processed image to the internal memory.

In an embodiment, the step of preprocessing the point cloud data to obtain a corresponding two-dimensional image may specifically include: determining a coordinate value range according to the coordinates of each point cloud in the point cloud data, and filtering the point cloud data based on the coordinate value range to obtain a first point cloud; performing point cloud segmentation on the first point cloud, and extracting a second point cloud containing road information; mapping the second point cloud into a first image according to the height threshold of the obstacle, wherein the first image is an image containing the road obstacle; according to the road edge height threshold, mapping the second point cloud into a second image, wherein the second image is an image containing a road boundary; the two-dimensional image includes a first image and a second image.

In one embodiment, a digital signal processor includes: a first digital signal processor branch and a second digital signal processor branch processed in parallel; the step of sending the control signal to the digital signal processor may specifically include: sending a control signal to a first digital signal processor branch and a second digital signal processor branch, enabling the first digital signal processor branch to carry out median filtering on the first image to obtain a third image, and carrying out expansion operation on the third image to obtain a fourth image; the second digital signal processor branch carries out corrosion operation on the second image to obtain a fifth image, and threshold segmentation based on gray value histogram optimization is carried out on the fifth image to obtain a tenth image; the processed image includes a fourth image and a tenth image.

In one embodiment, the method further comprises: and acquiring a fourth image and a tenth image from the internal memory, and respectively carrying out contour detection on the fourth image and the tenth image to acquire road obstacle contour information and road boundary contour information.

For the specific description of the above-mentioned embodiment of the point cloud processing method, reference may be made to the related description in the foregoing embodiment of the point cloud processing system, and details are not repeated here. In the point cloud processing method, the central processing unit executes the point cloud preprocessing task to convert the point cloud data into the two-dimensional image, so that the subsequent data volume to be processed is greatly reduced; the image processing task is executed through the digital signal processor, and the image processing time can be greatly shortened by utilizing the speed advantage of the digital signal processor; based on a direct memory access mechanism, the data transmission process does not need the participation of a central processing unit, and the high-efficiency data transmission can be realized. Therefore, the central processing unit is only responsible for task scheduling and processing tasks with low time consumption, and image processing tasks with high time consumption are put on the digital signal processor to be executed, so that the resource occupancy of the central processing unit is reduced, the image processing speed is increased, and the real-time requirement of the system is better met. In addition, the digital signal processor supports flexible development of a user-defined operator according to application requirements, a plurality of digital signal processors can run in parallel, resource advantages of the embedded heterogeneous computing platform can be fully exerted, and the embedded heterogeneous computing platform has the advantages of low power consumption and low cost, and is more suitable for vehicle-level application requirements with strict requirements on power consumption and cost.

In one embodiment, as shown in fig. 10, a point cloud processing method is provided, which is exemplified by the application of the method to the digital signal processor of fig. 1, and includes the following steps S1002 to S1006.

And S1002, receiving a control signal sent by the central processing unit.

S1004, responding to the control signal, and transmitting the two-dimensional image in the internal memory to a random access memory of the digital signal processor for image processing based on a direct memory access mechanism; the two-dimensional image is obtained by acquiring point cloud data by a central processing unit, preprocessing the point cloud data and storing the point cloud data in an internal memory.

And S1006, returning the processed image to the internal memory.

In one embodiment, the method further comprises: acquiring the image size and the storage address of the two-dimensional image; the step of transmitting the two-dimensional image in the internal memory to a random access memory of a digital signal processor for image processing may specifically include: and according to the image size, the storage address and the preset block size, transmitting the two-dimensional image blocks in the internal memory to the random access memory for block processing.

In one embodiment, a first buffer memory and a second buffer memory are arranged in the random access memory; according to the image size, the storage address and the preset block size, the step of transferring the two-dimensional image block in the internal memory to the random access memory for block processing may specifically include: determining the serial number and the position information of each block data according to the image size, the storage address and the preset block size; taking the first sequence number as a first current sequence number, and transmitting the block data corresponding to the first current sequence number to a first cache according to the position information of the block data corresponding to the first current sequence number; taking the second sequence number as a second current sequence number, and transmitting the block data corresponding to the second current sequence number to a second cache according to the position information of the block data corresponding to the second current sequence number; processing the block data corresponding to the first current sequence number, and simultaneously transmitting the block data corresponding to the first subsequent sequence number to a first cache according to the position information of the block data corresponding to the first subsequent sequence number, wherein the first subsequent sequence number represents a next sequential sequence number of the second current sequence number; after the processing of the block data corresponding to the first current sequence number is finished, processing the block data corresponding to the second current sequence number, and simultaneously transmitting the block data corresponding to the second subsequent sequence number to a second cache according to the position information of the block data corresponding to the second subsequent sequence number, wherein the second subsequent sequence number represents the next sequential sequence number of the first subsequent sequence number; and after the processing of the block data corresponding to the second current sequence number is finished, taking the first subsequent sequence number and the second subsequent sequence number as a new first current sequence number and a new second current sequence number respectively, and returning to the step of processing the block data corresponding to the first current sequence number until the processing of the block data corresponding to all the sequence numbers is finished.

In one embodiment, the image size includes an image width and an image height, the tile size includes a tile width and a tile height, and the storage address includes a storage head address; the step of determining the sequence number and the position information of each block data according to the image size, the storage address and the preset block size may specifically include: obtaining a width remainder by carrying out remainder on the width of the image to the width of the block, and obtaining a height remainder by carrying out remainder on the height of the image to the height of the block; and determining the serial number and the position information of each block data according to the storage initial address, the block width, the block height, the width remainder and the height remainder.

In one embodiment, the two-dimensional image includes a first image and a second image, the first image is an image containing a road obstacle, and the second image is a grayscale image containing a road boundary; the image processing comprises the following two branches of parallel processing: the first item: performing median filtering on the first image to obtain a third image, and performing expansion operation on the third image to obtain a fourth image; the second term is: carrying out corrosion operation on the second image to obtain a fifth image, and carrying out threshold segmentation based on gray value histogram optimization on the fifth image to obtain a tenth image; the processed image includes a fourth image and a tenth image.

In an embodiment, the step of performing threshold segmentation based on gray-value histogram optimization on the fifth image to obtain a tenth image may specifically include: dividing the fifth image into a sixth image and a seventh image according to the image height proportion; respectively calculating a first gray value histogram of a sixth image and a second gray value histogram of a seventh image; segmenting the sixth image according to a first gray threshold determined by the central processing unit based on the first gray value histogram to obtain an eighth image, and segmenting the seventh image according to a second gray threshold determined by the central processing unit based on the second gray value histogram to obtain a ninth image; the eighth image and the ninth image are merged into a tenth image.

In one embodiment, the method for determining the first gray threshold includes: dividing each pixel in the first gray value histogram into a first background pixel and a first target pixel according to a first initial threshold, respectively calculating a first background pixel mean value and a first target pixel mean value, and determining a first gray threshold based on the first background pixel mean value and the first target pixel mean value; the method for determining the second gray level threshold value comprises the following steps: dividing each pixel in the second gray value histogram into a second background pixel and a second target pixel according to a second initial threshold, respectively calculating a second background pixel mean value and a second target pixel mean value, and determining a second gray threshold based on the second background pixel mean value and the second target pixel mean value.

In one embodiment, as shown in fig. 11, there is provided a point cloud processing method including the steps of: the CPU reads the point cloud data and carries out point cloud preprocessing to obtain an image I1 containing road obstacles and an image I2 containing road boundaries; the DSP1 performs median filtering on the I1 to obtain I3, and the DSP2 performs expansion operation on the I2 to obtain I4; the DSP3 carries out corrosion operation on the I2 to obtain I5, and the DSP4 carries out threshold segmentation on the I5 based on gray value histogram optimization to obtain I10; and the CPU respectively carries out contour detection on the I4 and the I10 to obtain road obstacle contour information and road boundary contour information.

It should be understood that, although the steps in the flowcharts related to the above embodiments are shown in sequence as indicated by the arrows, the steps are not necessarily executed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in each flowchart related to the above embodiments may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a part of the steps or stages in other steps.

In one embodiment, as shown in fig. 12, there is provided a point cloud processing apparatus 1200 applied to a central processor, including: an acquisition module 1210, a point cloud preprocessing module 1220, and an image processing control module 1230, wherein:

an obtaining module 1210 is configured to obtain point cloud data.

The point cloud preprocessing module 1220 is configured to preprocess the point cloud data to obtain a corresponding two-dimensional image, and store the two-dimensional image in the internal memory.

The image processing control module 1230 is configured to send a control signal to the digital signal processor, and the digital signal processor transmits the two-dimensional image in the internal memory to the random access memory of the digital signal processor for image processing based on a direct memory access mechanism, and returns the processed image to the internal memory.

In one embodiment, as shown in fig. 13, there is provided a point cloud processing apparatus 1300 applied to a digital signal processor, including: a receiving module 1310, an image processing module 1320, and a transmitting module 1330, wherein:

the receiving module 1310 is configured to receive a control signal sent by the central processing unit.

An image processing module 1320, configured to respond to the control signal, and transmit the two-dimensional image in the internal memory to a random access memory of the digital signal processor for image processing based on a direct memory access mechanism; the two-dimensional image is obtained by acquiring point cloud data by a central processing unit, preprocessing the point cloud data and storing the point cloud data in an internal memory.

The transmission module 1330 is configured to transmit the processed image back to the internal memory.

For specific limitations of the point cloud processing apparatus, reference may be made to the above limitations of the point cloud processing method, which is not described herein again. The modules in the point cloud processing device can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 14. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a point cloud processing method.

In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 15. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a point cloud processing method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

It will be appreciated by those skilled in the art that the configurations shown in fig. 14 or 15 are only block diagrams of some of the configurations relevant to the present application, and do not constitute a limitation on the computer apparatus to which the present application is applied, and a particular computer apparatus may include more or less components than those shown in the drawings, or may combine some components, or have a different arrangement of components.

In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the above-described method embodiments when executing the computer program.

In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the respective method embodiment as described above.

In one embodiment, a computer program product or computer program is provided that includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the steps in the various method embodiments described above.

It should be understood that the terms "first", "second", etc. in the above-described embodiments are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated. Further, in the description of the present application, the meaning of "a plurality" means at least two unless otherwise specified.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile Memory can include Random Access Memory (RAM) or external cache Memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A point cloud processing system, the system comprising: the device comprises a central processing unit, an internal memory and a digital signal processor;

the digital signal processor is configured to respond to the control signal, transmit the two-dimensional image in the internal memory to a random access memory of the digital signal processor for image processing based on a memory direct access mechanism, and transmit a processed image back to the internal memory;

the central processing unit is specifically configured to, when preprocessing the point cloud data to obtain a corresponding two-dimensional image:

determining a coordinate value range according to the coordinates of each point cloud in the point cloud data, and filtering the point cloud data based on the coordinate value range to obtain a first point cloud;

performing point cloud segmentation on the first point cloud, and extracting a second point cloud containing road information;

mapping the second point cloud into a first image according to an obstacle height threshold, wherein the first image is an image containing a road obstacle;

mapping the second point cloud into a second image according to a road edge height threshold, wherein the second image is an image containing a road boundary; the two-dimensional image includes the first image and the second image.

2. The system of claim 1, wherein the digital signal processor comprises: a first digital signal processor branch and a second digital signal processor branch processed in parallel;

the first digital signal processor branch is used for carrying out median filtering on the first image to obtain a third image and carrying out expansion operation on the third image to obtain a fourth image when carrying out image processing;

the second digital signal processor branch is used for carrying out corrosion operation on the second image to obtain a fifth image when carrying out image processing, and carrying out threshold segmentation based on gray value histogram optimization on the fifth image to obtain a tenth image; the processed image includes the fourth image and the tenth image.

3. The system according to claim 2, wherein the second digital signal processor branch, when performing threshold segmentation based on gray-value histogram optimization on the fifth image to obtain a tenth image, is specifically configured to:

dividing the fifth image into a sixth image and a seventh image according to the image height proportion;

respectively calculating a first gray value histogram of the sixth image and a second gray value histogram of the seventh image;

segmenting the sixth image according to a first gray threshold determined based on the first gray value histogram to obtain an eighth image, and segmenting the seventh image according to a second gray threshold determined based on the second gray value histogram to obtain a ninth image;

merging the eighth image and the ninth image into a tenth image.

4. The system according to claim 3, wherein said second digital signal processor branch is further configured to transmit said first histogram of gray scale values and said second histogram of gray scale values back to said internal memory;

the central processing unit is further configured to:

acquiring the first gray value histogram and the second gray value histogram from the internal memory;

dividing each pixel in the first gray value histogram into a first background pixel and a first target pixel according to a first initial threshold, respectively calculating a first background pixel mean value and a first target pixel mean value, and determining a first gray threshold based on the first background pixel mean value and the first target pixel mean value;

dividing each pixel in the second gray value histogram into a second background pixel and a second target pixel according to a second initial threshold, respectively calculating a second background pixel mean value and a second target pixel mean value, and determining a second gray threshold based on the second background pixel mean value and the second target pixel mean value;

communicating the first and second grayscale thresholds to the second digital signal processor branch.

5. The system according to claim 4, wherein the central processing unit is further configured to obtain the fourth image and the tenth image from the internal memory, perform contour detection on the fourth image and the tenth image, respectively, and obtain road obstacle contour information and road boundary contour information.

6. A method of point cloud processing, the method comprising:

acquiring point cloud data;

preprocessing the point cloud data to obtain a corresponding two-dimensional image, and storing the two-dimensional image into an internal memory;

sending a control signal to a digital signal processor, transmitting the two-dimensional image in the internal memory to a random access memory of the digital signal processor for image processing by the digital signal processor based on a direct memory access mechanism, and returning the processed image to the internal memory;

preprocessing the point cloud data to obtain a corresponding two-dimensional image, wherein the preprocessing comprises the following steps:

7. The method of claim 6, wherein the digital signal processor comprises: a first digital signal processor branch and a second digital signal processor branch processed in parallel;

sending a control signal to a digital signal processor, comprising: sending control signals to the first digital signal processor branch and the second digital signal processor branch such that

The first digital signal processor branch performs median filtering on the first image to obtain a third image, and performs expansion operation on the third image to obtain a fourth image;

the second digital signal processor branch carries out corrosion operation on the second image to obtain a fifth image, and threshold segmentation based on gray value histogram optimization is carried out on the fifth image to obtain a tenth image; the processed image includes the fourth image and the tenth image.

8. The method of claim 7, further comprising:

and acquiring the fourth image and the tenth image from the internal memory, and respectively performing contour detection on the fourth image and the tenth image to acquire road obstacle contour information and road boundary contour information.

9. A method of point cloud processing, the method comprising:

receiving a control signal sent by a central processing unit;

returning the processed image to the internal memory;

the central processing unit acquires point cloud data, and pre-processes the point cloud data to obtain the two-dimensional image, and the method comprises the following steps: the central processing unit determines a coordinate value range according to the coordinates of each point cloud in the point cloud data, and filters the point cloud data based on the coordinate value range to obtain a first point cloud; performing point cloud segmentation on the first point cloud, and extracting a second point cloud containing road information; mapping the second point cloud into a first image according to an obstacle height threshold, wherein the first image is an image containing a road obstacle; mapping the second point cloud into a second image according to a road edge height threshold, wherein the second image is an image containing a road boundary; the two-dimensional image includes the first image and the second image.

10. The method of claim 9, wherein the two-dimensional image comprises a first image and a second image, the first image being an image containing a road obstacle, the second image being an image containing a road boundary; the image processing comprises the following two branches of parallel processing:

the first item: performing median filtering on the first image to obtain a third image, and performing expansion operation on the third image to obtain a fourth image;

the second term is: carrying out corrosion operation on the second image to obtain a fifth image, and carrying out threshold segmentation based on gray value histogram optimization on the fifth image to obtain a tenth image; the processed image includes the fourth image and the tenth image.

11. The method of claim 10, wherein performing threshold segmentation based on gray-value histogram optimization on the fifth image to obtain a tenth image comprises:

merging the eighth image and the ninth image into a tenth image.

12. The method of claim 11, wherein the determining of the first gray threshold value comprises: dividing each pixel in the first gray value histogram into a first background pixel and a first target pixel according to a first initial threshold, respectively calculating a first background pixel mean value and a first target pixel mean value, and determining a first gray threshold based on the first background pixel mean value and the first target pixel mean value;

the method for determining the second gray level threshold value comprises the following steps: dividing each pixel in the second gray value histogram into a second background pixel and a second target pixel according to a second initial threshold, respectively calculating a second background pixel mean value and a second target pixel mean value, and determining a second gray threshold based on the second background pixel mean value and the second target pixel mean value.

13. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor realizes the steps of the method of any one of claims 6 to 12 when executing the computer program.

14. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 6 to 12.