JP7107015B2 - Point cloud processing device, point cloud processing method and program - Google Patents

Description

The present invention relates to a point cloud processing device, a point cloud processing method, and a program.

In recent years, there has been an increasing demand for three-dimensional measurement of real space and three-dimensional modeling of measurement data. Methods for realizing three-dimensional measurement of real space and three-dimensional modeling of measurement data include three-dimensional lasers that detect reactions to active light (near-infrared light), such as LIDAR (Laser Imaging Detection and Ranging). A method using a sensor is mentioned.

Regarding three-dimensional measurement, Patent Document 1 adds a new point cloud generated by processing information obtained from an image sensor for areas where the point density is sparse in the point cloud obtained using LIDAR. A technique for increasing the resolution of a point group by doing so is disclosed. Also, in the method of obtaining a depth map using infrared dots, a region that does not reflect infrared rays sufficiently becomes a missing region in the depth map. Regarding this point, Patent Document 2 discloses a method of interpolating a missing area using an image having RGB color information. Note that the depth map may be used to generate a point group.

Patent No. 5778237 Japanese Patent Publication No. 2016-522923

According to the techniques described in Patent Documents 1 and 2, it is possible to obtain a point cloud or depth map that expresses the real space at a certain point in time. However, Patent Literature 1 and Patent Literature 2 do not disclose a technique for extracting a background point cloud belonging to a background area such as a floor or a wall in real space from the entire point cloud. Even if there is a technique to extract the background point cloud from the entire point cloud obtained by the measurement at a certain point, if part of the background region is hidden by the object, part of the background point cloud will be lost. put away.

Accordingly, the present invention has been made in view of the above problems. It is an object of the present invention to provide a point cloud processing device, a point cloud processing method, and a program.

In order to solve the above problems, according to an aspect of the present invention, from a three-dimensional point group corresponding to each timing obtained by measuring the real space from the same point at each of a plurality of timings with LIDAR , a background extraction unit for extracting a background point group, which is a point group belonging to a background region, based on the distribution of the three-dimensional point group corresponding to each timing; a background integration unit that generates an integrated background point cloud by integrating at least a part of each background point cloud extracted by the background extraction unit; By repeatedly extracting a new partial point group and integrating the new partial point group into the integrated background point group, the integrated background point group is repeatedly updated, and points constituting the integrated background point group are determined. A point cloud processing device is provided that deletes some of the points that make up the integrated background point cloud when the number exceeds an upper limit . Further, in order to solve the above problems, according to an aspect of the present invention, a three-dimensional point corresponding to each timing obtained by measuring the real space from the same point at each of a plurality of timings with LIDAR a background extracting unit for extracting a background point group, which is a point group belonging to a background region, from the group based on the distribution of the three-dimensional point group corresponding to each timing; a background integration unit that generates an integrated background point cloud by integrating at least part of each extracted background point cloud, the background integration unit adding new background points extracted by the background extraction unit. A point cloud processing device is provided, which repeatedly updates the integrated background point cloud by repeatedly extracting a new partial point cloud from the group at random and integrating the new partial point cloud into the integrated background point cloud. be done.

Some of the points forming the integrated background point group may be random points forming the integrated background point group.

The background integration unit may randomly extract the new partial point cloud from the new background point cloud.

The background integration unit may randomly extract the new partial point clouds from different ranges for each of the plurality of new background point clouds.

The point cloud processing device may further include an object detection unit that detects an area in which an object exists by calculating a difference between a certain point cloud and the integrated background point cloud.

The object detection unit includes an integrated background point obtained by integrating a point cloud corresponding to a first timing and a partial point cloud extracted from a point cloud corresponding to a second timing before the first timing. A region in which an object is present may be detected by calculating the difference with the group.

The background area may include at least one of a floor area and a wall area.

Further, in order to solve the above problems, according to another aspect of the present invention, a three-dimensional image corresponding to each timing obtained by measuring the real space from the same point at each of a plurality of timings with LIDAR extracting a background point group, which is a point group belonging to a background region, from the point group based on the distribution of the three-dimensional point group corresponding to each timing; and extracting from the point group corresponding to each of the plurality of timings. and generating an integrated background point cloud by integrating at least a portion of each extracted background point cloud, wherein the generating the integrated background point cloud comprises a new extracted background point cloud; by repeatedly extracting a partial point cloud and integrating the new partial point cloud with the integrated background point cloud, the integrated background point cloud is repeatedly updated, and the number of points constituting the integrated background point cloud A point cloud processing method is provided , including deleting a part of the points constituting the integrated background point cloud when is above an upper limit value . Further, in order to solve the above problems, according to another aspect of the present invention, a three-dimensional image corresponding to each timing obtained by measuring the real space from the same point at each of a plurality of timings with LIDAR extracting a background point group, which is a point group belonging to a background region, from the point group based on the distribution of the three-dimensional point group corresponding to each timing; and extracting from the point group corresponding to each of the plurality of timings. and generating a merged background point cloud by integrating at least a portion of each extracted background point cloud, said generating a merged background point cloud randomly selected from the extracted new background point cloud. and repeatedly updating the integrated background point cloud by extracting a new partial point cloud and integrating the new partial point cloud into the integrated background point cloud. be done.

Further, in order to solve the above problems, according to another aspect of the present invention, the computer measures the real space from the same point at each of a plurality of timings with LIDAR, and corresponds to each timing. a background extraction unit for extracting a background point group, which is a point group belonging to a background region, from the three-dimensional point group based on the distribution of the three-dimensional point group corresponding to each timing; and corresponding to each of the plurality of timings. a background integration unit configured to generate an integrated background point cloud by integrating at least part of each background point cloud extracted from the point cloud extracted from the background extraction unit; By repeatedly extracting a new partial point cloud from a new background point cloud and integrating the new partial point cloud into the integrated background point cloud, the integrated background point cloud is repeatedly updated, and the integrated background point A program is provided for functioning as a point cloud processing device that deletes some of the points that make up the integrated background point cloud when the number of points that make up the group exceeds an upper limit . Further, in order to solve the above problems, according to another aspect of the present invention, the computer measures the real space from the same point at each of a plurality of timings with LIDAR, and corresponds to each timing. a background extraction unit for extracting a background point group, which is a point group belonging to a background region, from the three-dimensional point group based on the distribution of the three-dimensional point group corresponding to each timing; and corresponding to each of the plurality of timings. a background integration unit configured to generate an integrated background point cloud by integrating at least part of each background point cloud extracted from the point cloud extracted from the background extraction unit; randomly extracting a new partial point cloud from a new background point cloud, and repeatedly integrating the new partial point cloud into the integrated background point cloud, thereby repeatedly updating the integrated background point cloud. A program is provided for functioning as a processing device.

According to the present invention described above, it is possible to obtain the background point group belonging to the background area of the real space with higher accuracy.

1 is an explanatory diagram showing a point cloud processing system according to an embodiment of the present invention; FIG. FIG. 4 is an explanatory diagram showing a specific example of real space; FIG. 3 is an explanatory diagram showing a background point group extracted from the point group in the real space shown in FIG. 2; It is an explanatory view showing composition of point group processing device 200 by an embodiment of the present invention. FIG. 4 is an explanatory diagram showing a specific example of extraction of partial point group Pi; FIG. 4 is an explanatory diagram showing a specific example of an integrated background point cloud; It is a flow chart which shows operation of point group processing device 200 by an embodiment of the present invention. It is an explanatory view showing the 1st modification. It is an explanatory view showing a second modification. 2 is a block diagram showing the hardware configuration of the point cloud processing device 200; FIG.

Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In the present specification and drawings, constituent elements having substantially the same functional configuration are denoted by the same reference numerals, thereby omitting redundant description.

In addition, in this specification and drawings, a plurality of components having substantially the same functional configuration may be distinguished by attaching different alphabets after the same reference numerals. However, when there is no particular need to distinguish between a plurality of constituent elements having substantially the same functional configuration, only the same reference numerals are given to each of the plurality of constituent elements.

<Overview of point cloud processing system>
An embodiment of the present invention relates to a point cloud processing system that detects an object existing in a real space by processing a point cloud obtained by measuring the real space using LIDAR (Laser Imaging Detection and Ranging). Prior to detailed description of embodiments of the present invention, an overview of a point cloud processing system according to embodiments of the present invention will be described.

FIG. 1 is an explanatory diagram showing a point cloud processing system according to an embodiment of the present invention. As shown in FIG. 1, a point cloud processing system according to an embodiment of the present invention has a LIDAR 100 and a point cloud processing device 200 . The LIDAR 100 and the point cloud processing device 200 may be connected by wire, may be connected wirelessly, or may be connected via a network.

(LIDAR)
The LIDAR 100 is a measuring device that measures a three-dimensional environment in real space. The LIDAR 100 is installed at a point, for example being supported by a tripod 110 as shown in FIG. The LIDAR 100 repeats measurements using a laser over a plurality of frames from the same installed point to obtain a three-dimensional point cloud for each frame. The LIDAR 100 outputs the point cloud obtained by measurement to the point cloud processing device 200 .

The number of laser transmission/reception sensors provided in the LIDAR 100, the laser scanning method, and the measurable range are not particularly limited. For example, the number of laser transmitting and receiving sensors may be 8, 16, 32, 64, 128, and so on. Further, the laser scanning method may be a method of changing the direction of the laser along the horizontal direction, or a method of changing the direction of the laser along the vertical direction. Also, the measurable range may be 120° or omnidirectional in the horizontal field of view, and may be 20°, 30° or 40° in the vertical field of view.

(Point cloud processing device)
The point cloud processing device 200 detects an object area where an object exists in real space based on the point cloud input from the LIDAR 100 . Object region detection can be achieved by segmenting point clouds belonging to background regions (fixed objects such as floors and walls) from the point cloud input from the LIDAR 100 . Therefore, the point cloud processing device 200 also has a function of extracting a background point cloud, which is a point cloud belonging to the background region, from the point cloud input from the LIDAR 100 . The object to be detected by the point group processing device 200 is not particularly limited. For example, the point group processing device 200 may detect a moving object such as a person or a vehicle. A specific example of background point cloud extraction by the point cloud processing device 200 will now be described with reference to FIGS. 2 and 3. FIG.

FIG. 2 is an explanatory diagram showing a specific example of real space. The real space shown in FIG. 2 is a space surrounded by a floor surface F and a wall surface W, and an object O exists on the floor surface F. As shown in FIG. In this specification, it is assumed that the LIDAR 100 is installed such that the XY plane is horizontal to the floor surface F, and the XZ plane or YZ plane is horizontal to the wall surface W.

FIG. 3 is an explanatory diagram showing a background point group extracted from the point group in the real space shown in FIG. As shown in FIG. 3, the background point group is scattered in the areas of the floor surface F and the wall surface W other than the portions shielded by the object O. As shown in FIG. Note that FIG. 3 merely shows the background point group in a simplified manner, and in reality, the background point group may be obtained with a larger number of points, or the points may be distributed in an arc shape. .

(Organization of issues)
The above object detection can be performed accurately by using a sufficient background point group. On the other hand, as shown in FIG. 3, in the background point cloud in which the point cloud of the portion occluded by the object O is missing, there is a possibility that part of the point cloud that should be treated as the background area will be detected as the object. .

Therefore, the inventor of the present invention has created an embodiment of the present invention with the above circumstances as a point of view. The point cloud processing device 200 according to the embodiment of the present invention can obtain a background point cloud belonging to the background area of the real space with higher accuracy. Hereinafter, the configuration and operation of the point cloud processing apparatus 200 according to the embodiment of the present invention will be sequentially described in detail.

<Configuration of point cloud processing device>
FIG. 4 is an explanatory diagram showing the configuration of the point cloud processing device 200 according to the embodiment of the present invention. As shown in FIG. 4, the point cloud processing device 200 according to the embodiment of the present invention includes a point cloud storage unit 220, a background extraction unit 230, a background point cloud storage unit 240, a background integration unit 250, an object detection unit 260, a display It has a control unit 270 and a display unit 280 .

(Point cloud memory)
The point cloud storage unit 220 stores the point cloud of each frame input from the LIDAR 100 . It is assumed that the background point group belonging to the background area and the object point group belonging to the object area are mixed in the point group without distinction. The point group input from the LIDAR 100 may be hereinafter referred to as point group Po.

(Background extraction part)
The background extraction unit 230 extracts a background point group belonging to the background region from the point cloud Po stored in the point cloud storage unit 220 . The background area corresponds to, for example, a floor surface and a wall surface, and the background point group can include a floor surface point group Pof belonging to the floor surface and a wall surface point group Pow belonging to the wall surface, as shown in FIG. It is estimated that the floor surface is the largest plane in the point group Po horizontal to the XY plane, and the wall surface is the largest plane in the point group Po horizontal to the XZ plane or the YZ plane. Therefore, the background extraction unit 230 obtains the floor point group Pof and the wall point group Pow by executing RANSAC (Random Sampling Consensus). Hereinafter, after explaining the procedure for estimating the point group by RANSAC of the point group forming the planar shape, a specific method for obtaining the floor point group Pof and the wall surface point group Pow by RANSAC will be explained.

By executing RANSAC on the point group Po using three parameters, a point group P_plane that forms a planar shape is output. The three parameters include the normal direction vpl of the plane shape to be estimated, the error threshold dist_pl of the distance between the point group forming the plane and the plane, and the error threshold of the angle between the point group forming the plane and the plane. angle_pl is applicable. These parameters can be set manually. Summarizing the above, the point cloud P_plane is expressed as follows.
P_plane:=RANSAC(Po, vpl, dist_pl, angle_pl)

Then, the background extraction unit 230 can obtain the floor point group Pof by applying the Z-axis direction to the vpl described above. The processing for obtaining the floor point group Pof is expressed as follows.
Pof := RANSAC(Po, Z, dist_pl, angle_pl)

Furthermore, the background extraction unit 230 can obtain the wall surface point group Pow_Y belonging to the wall surface parallel to the XZ plane by applying the Y-axis direction to the vpl described above. In addition, the background extraction unit 230 can obtain the wall point group Pow_X belonging to the wall surface horizontal to the YZ plane by applying the X-axis direction to vpl described above. A point group in which the wall surface point group Pow_Y and the wall surface point group Pow_X are combined is the wall surface point group Pow. The process of obtaining the wall surface point group Pow is expressed as follows.
Pow_Y:=RANSAC(Po, Y, dist_pl, angle_pl)
Pow_X:=RANSAC(Po, X, dist_pl, angle_pl)
Pow:= Pow_Y + Pow_X

(Background point cloud memory)
The background point cloud storage unit 240 stores the background point cloud. Specifically, when the floor point group Pof and the wall surface point group Pow extracted by the background extraction unit 230 by the above-described processing are point groups extracted from the point group Po of the first frame, the background point group storage unit 240 stores these floor point group Pof and wall point group Pow as an initial background point group. The background point cloud storage unit 240 also stores an integrated background point cloud obtained by the background integration unit 250, which will be described later.

(Background Integration Department)
The background integration unit 250 combines the initial background point cloud stored in the background point cloud storage unit 240 and the partial point cloud Pi of the background point cloud (floor point cloud Pof and wall surface point cloud Pow) extracted from the new frame. By integrating, an integrated background point group Pb is generated. Further, when the integrated background point group Pb is already stored in the background point cloud storage unit 240, the background integration unit 250 combines the partial points of the integrated background point group Pb and the background point cloud extracted from the new frame. By integrating the group Pi, the integrated background point cloud Pb is updated.

Here, the partial point group Pi of the floor point group Pof and the wall point group Pow extracted from the new frame may be a point group randomly extracted from the floor point group Pof and the wall point group Pow. Alternatively, it may be a point cloud extracted according to a rule specified by the user. The process by which the background integration unit 250 extracts such a partial point group Pi is expressed as follows. Pi_size is the number of points contained in the sub-point cloud Pi and is a parameter given by the user. The number of Pi_size may be, for example, 0.3% to 1.0% of the number of points of the floor point cloud Pof and the wall point cloud Pow extracted from the new frame.
Pi:=Extract_pi(Pow+Pof, random or user rule, Pi_size)

Then, the background integration unit 250 generates or updates the integrated background point group Pb by adding the partial point group Pi described above to the initial background point group or the integrated background point group Pb. The processing is expressed as follows.
Pb:=Pb+Pi

The extraction of the partial point group Pi and the generation of the integrated background point group Pb will be described in more detail with reference to FIGS. 5 and 6. FIG.

FIG. 5 is an explanatory diagram showing a specific example of extraction of the partial point group Pi. FIG. 5 shows the floor point group Pof and the wall point group Pow extracted by the background extraction unit 230 in the frame next to the frame shown in FIG. Comparing FIG. 3 and FIG. 5, the object O is moving, and the positions where the point cloud is missing due to the object O being blocked are different.

The background integration unit 250 extracts, for example, six partial point groups Pi indicated by thick lines in FIG. 5 from the floor point group Pof and the wall point group Pow shown in FIG. Then, the background integration unit 250 generates the integrated background point group Pb shown in FIG. 6 by adding the partial point group Pi to the initial background point group shown in FIG. In the integrated background point cloud Pb shown in FIG. 6, a point cloud is added to the position where the point cloud was missing due to being blocked by the object O in FIG. It is possible to improve the accuracy of the integrated background point group Pb by repeating the addition of such a partial point group Pi over a plurality of frames to update the integrated background point group Pb.

However, if the addition of the partial point cloud Pi is repeated, the number of points of the integrated background point cloud Pb will increase excessively, and the storage area and computational resources will be squeezed. Therefore, the maximum score Npbmax is set as the upper limit of the score of the integrated background point group Pb, and the background integration unit 250 performs processing on the integrated background point group Pb according to the relationship between the score of the integrated background point group Pb and the maximum score Npbmax. may be executed.

Specifically, when the score of the integrated background point group Pb is less than the maximum score Npbmax, the background integration unit 250 updates the integrated background point group Pb by adding the partial point group Pi to the integrated background point group Pb. .

On the other hand, when the score of the integrated background point group Pb is equal to or greater than the maximum score Npbmax, the background integration unit 250 randomly deletes the same number of points P_d as the partial point group Pi from the integrated background point group Pb. Then, the background integration unit 250 updates the integrated background point group Pb by adding the partial point group Pi to the integrated background point group Pb after the point Pb_d is erased.

The above processing is organized as follows.
Pb:=Pb+Pi (when maximum score Npbmax>Pb score)
Pb:=Pb-Pb_d+Pi (when the maximum score Npbmax≤Pb)

(Object detection unit)
The object detection unit 260 uses the point group Po and the integrated background point group Pb to detect an area in which an object exists (object area). Therefore, the object detection unit 260 calculates the difference between the point group Po and the integrated background point group Pb. In order to calculate the difference at high speed, the object detection unit 260 converts the point group Po and integrated background point group Pb into an Octree structure to obtain Oct_Po and Oct_Pb. Then, the object detection unit 260 obtains point group information Diff_Po_Pb of the difference between Oct_Po and Oct_Pb. Further, the object detection unit 260 applies Diff_Po_Pb to the point group Po to obtain a difference point group Pd, which is the difference between the point group Po and the integrated background point group Pb.

The processing for obtaining the difference point group Pd described above is expressed as follows.
Oct_Po := Octree(Po)
Oct_Pb:=Octree(Pb)
Diff_Po_Pb:=Octdiff(Oct_Po, Oct_Pb)
Pd:=Octindice(Po, Diff_Po_Pb)

Then, the object detection unit 260 obtains an object region Region_d by clustering the difference point group Pd. Specifically, the object detection unit 260 first converts the difference point group Pd into a Kdtree structure for clustering, and obtains Kd_Pd. Then, the object detection unit 260 performs clustering using Kd_Pd, the permissible distance dist_c between each cluster, the minimum number of cluster point cloud points Ncmin, and the maximum number of cluster point cloud points Ncmax. get 0 to a plurality of Pc are obtained by the clustering. Note that dist_c, Ncmin and Ncmax may be parameters specified by the user.

Furthermore, the object detection unit 260 calculates an object region Region_d in which the object exists from the Pc group. Region_d has a list structure, and each element stores the number of points of each Pc, the maximum and minimum coordinates of the cluster, the cluster size, the center of gravity of the cluster, the distance from the cluster to the LIDAR 100, the type of object, and the like. The object detection unit 260 calculates the number of point groups and the coordinates of the cluster by searching the point group of Pc, and from the number of point groups and the coordinates of the cluster, the cluster size, the center of gravity of the cluster, the distance from the cluster to the LIDAR 100, and It is possible to calculate the type of object and so on.

The above-described processing from clustering to detection of the object region Region_d is expressed as follows. Note that Kdtree is a function that converts an argument to a Kdtree structure, Clustering is a function that outputs a cluster group that constitutes an object in a point group, Region_f is the number of point groups in each cluster, the maximum and minimum coordinates of the cluster, A function that outputs a list of structures containing cluster size, cluster centroid, distance from cluster to LIDAR 100, and object type.
Kd_Pd:=Kdtree(Pd)
Pc:=Clustering(Kd_Pd, dist_c, Ncmin, Ncmax)
Region_d := Region_f(Pc)

(Display control unit)
The display control unit 270 uses the point cloud Po read from the point cloud storage unit 220 and the object region Region_d to generate a detection result screen showing the detection result of the object in the point cloud Po. The detection result screen is generated by arranging the point group Po, arranging a cluster region (cube) using the maximum and minimum coordinates of the cluster obtained from Region_d, and using the maximum and minimum coordinates. For each object present in Region_d, compute each cube vertex and draw a line between each cube vertex to be an edge of the cube.

(Display part)
The display unit 280 displays the detection result screen generated by the display control unit 270. FIG. This makes all object regions present in the point cloud Po visible.

<Operation of point cloud processing device>
The configuration of the point group processing device 200 according to the embodiment of the present invention has been described above. Next, the operation of the point cloud processing device 200 according to the embodiment of the present invention will be described with reference to FIG.

FIG. 7 is a flow chart showing the operation of the point cloud processing device 200 according to the embodiment of the present invention. As shown in FIG. 7, first, the point cloud storage unit 220 stores the point cloud Po of the first frame input from the LIDAR 100 (S304).

Then, the background extraction unit 230 extracts a background point group (floor point group Pof and wall point group Pow) belonging to the background region from the point group Po stored in the point cloud storage unit 220 (S308). The background point cloud storage unit 240 stores the background point cloud extracted by the background extraction unit 230 as an initial background point cloud (S312).

After that, when the point cloud Po of a new frame is input from the LIDAR 100, the point cloud storage unit 220 stores the point cloud Po of the new frame (S316). Subsequently, the background extraction unit 230 extracts a background point group from the point group Po of the new frame (S320), and the background integration unit 250 extracts a partial point group Pi from the background point group (S324).

Here, when the integrated background point cloud Pb is stored in the background point cloud storage unit 240, the background integration unit 250 determines whether or not the score of the integrated background point cloud Pb is equal to or greater than the maximum score Npbmax (S328 ). If the score of the integrated background point cloud Pb is equal to or greater than the maximum score Npbmax, or if the integrated background point cloud Pb is not stored in the background point cloud storage unit 240 (S328/No), the process proceeds to S336. On the other hand, when the score of the integrated background point cloud Pb is equal to or greater than the maximum score Npbmax (S328/Yes), the background integration unit 250 deletes the same number of point clouds as the partial point cloud Pi from the integrated background point cloud Pb (S332). .

Subsequently, the background integration unit 250 updates (generates) the integrated background point group Pb by integrating the partial point group Pi into the integrated background point group Pb (initial background point group in the first round) (S336). Then, the object detection unit 260 calculates the difference between the point group Po of the new frame and the integrated background point group Pb to detect the object area (S340), and the display unit 280 detects the object area in the point group Po of the new frame. It is superimposed and displayed (S344).

After that, the processing of S316 to S344 is repeated until the measurement by the LIDAR 100 is completed (S348).

<Effect>
According to the embodiments of the present invention described above, various effects can be obtained.

For example, the background integration unit 250 extracts a partial point cloud Pi from the background point cloud of the new frame and adds the partial point cloud Pi to update the integrated background point cloud Pb. In contrast, all the background point clouds of the new frame are added, and after the number of points of the integrated background point cloud Pb reaches the maximum number of points Npbmax, a part of the point cloud of the integrated background point cloud Pb is added to the new frame. A method of replacing all the background point groups in . However, in this method, when the object moves, although the loss of the background point cloud at the position of the object before movement can be compensated, the background point cloud that was originally present at the position of the object after movement rapidly disappears. put away. In contrast, according to the embodiment of the present invention, it is possible to improve the robustness of the integrated background region Pb against object movement.

The background integration unit 250 also randomly (probabilistically) extracts a partial point cloud Pi from the background point cloud of the new frame. According to such a configuration, it is possible to interpolate missing points in the background point cloud without specifying a place where the point group is missing.

Further, after the number of points of the integrated background point group Pb reaches the maximum score Npbmax, the background integration unit 250 randomly deletes the points included in the integrated background point group Pb. According to this configuration, even if the integrated background point group Pb includes a point group that does not belong to the background area, it is possible to gradually eliminate the point group, and as a result, the integrated background point group Pb It is possible to improve the accuracy.

<Modification>
The embodiments of the present invention have been described above. Several variations of the embodiments of the present invention are described below. In addition, each modification described below may be applied to the embodiment of the present invention independently, or may be applied to the embodiment of the present invention in combination. Further, each modification may be applied instead of the configuration described in the embodiment of the invention, or may be additionally applied to the configuration described in the embodiment of the invention.

(First modification)
An example has been described above in which the background integration unit 250 randomly extracts the partial point group Pi from the entire background point group. A first modified example is a modified example relating to the extraction of this partial point group Pi. The background integration unit 250 may randomly extract the partial point cloud Pi from within a different range for each frame. The first modified example will be described more specifically with reference to FIG.

FIG. 8 is an explanatory diagram showing a first modified example. As shown in FIG. 8, the background integration unit 250 divides the background point cloud into a plurality of regions D1 to D3. , the partial point group Pi may be extracted. For example, in the n-th frame, the background integration unit 250 may randomly extract a partial point cloud Pi from the point cloud included in the region D1, such as the five point clouds indicated by the thick lines. Then, the background integration unit 250 randomly extracts the partial point cloud Pi from the point cloud included in the region D2 in the n+1th frame, and randomly extracts the partial point cloud Pi from the point cloud included in the region D3 in the n+2th frame. may be extracted.

(Second modification)
An example has been described above in which the integrated background point group Pb is updated using the point group Po of a new frame, and object detection is performed using the integrated background point group Pb after updating the integrated background point group Pb. A second modification relates to the relationship between the update of the integrated background point group Pb and object detection. In the second modification, the object detection unit 260 extracts the point cloud Po(n) of the frame n and the partial point cloud Pi The object area is detected by calculating the difference from the integrated background point group Pb(n-1) obtained by integrating (n-1). Hereinafter, the second modified example will be described more specifically with reference to FIG.

FIG. 9 is an explanatory diagram showing a second modified example. The processing of S304 to S316 is as described with reference to FIG. As shown in FIG. 9, the background integration unit 250 updates the integrated background point cloud Pb using the new frame point cloud Po (S320, S324, S328, S332, S336). The object detection unit 260 detects an object region in parallel with the update of the integrated background point group Pb by the background integration unit 250 .

That is, the object detection unit 260 calculates the difference between the point group Po of the new frame and the integrated background point group Pb updated in the previous frame, and detects the object area (S322). Then, the display unit 280 superimposes and displays the object region on the point group Po of the new frame (S326).

With such a configuration, object detection can be performed without waiting for the integrated background point group Pb to be updated using the point group Po of a new frame. can be improved.

<Hardware configuration>
The embodiments of the present invention have been described above. Information processing such as extraction of the background point cloud and updating of the integrated background point cloud described above is realized by cooperation between software and hardware of the point cloud processing device 200 described below.

FIG. 10 is a block diagram showing the hardware configuration of the point cloud processing device 200. As shown in FIG. The point group processing device 200 includes a CPU (Central Processing Unit) 201 , a ROM (Read Only Memory) 202 , a RAM (Random Access Memory) 203 and a host bus 204 . Further, the point cloud processing device 200 includes a bridge 205, an external bus 206, an interface 207, an input device 208, a display device 209, an audio output device 210, a storage device (HDD) 211, a drive 212, and a network interface 215 .

The CPU 201 functions as an arithmetic processing device and a control device, and controls general operations within the point cloud processing device 200 according to various programs. Alternatively, the CPU 201 may be a microprocessor. The ROM 202 stores programs, calculation parameters, and the like used by the CPU 201 . The RAM 203 temporarily stores programs used in the execution of the CPU 201, parameters that change as appropriate during the execution, and the like. These are interconnected by a host bus 204 comprising a CPU bus or the like. Functions of the above-described background extraction unit 230, background integration unit 250, object detection unit 260, display control unit 270, and the like can be realized by cooperation of the CPU 201, ROM 202, RAM 203, and software.

The host bus 204 is connected via a bridge 205 to an external bus 206 such as a PCI (Peripheral Component Interconnect/Interface) bus. It should be noted that the host bus 204, bridge 205 and external bus 206 do not necessarily have to be configured separately, and these functions may be implemented in one bus.

The input device 208 includes input means for the user to input information, such as a mouse, keyboard, touch panel, button, microphone, sensor, switch, and lever. It consists of a control circuit and the like. A user of the point cloud processing device 200 can input various data to the point cloud processing device 200 and instruct processing operations by operating the input device 208 .

The display device 209 includes, for example, a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device, a projector device, an OLED (Organic Light Emitting Diode) device and a lamp. Also, the audio output device 210 includes audio output devices such as speakers and headphones.

The storage device 211 is a data storage device configured as an example of a storage unit of the point cloud processing device 200 according to this embodiment. The storage device 211 may include a storage medium, a recording device that records data on the storage medium, a reading device that reads data from the storage medium, a deletion device that deletes data recorded on the storage medium, and the like. The storage device 211 is configured by, for example, a HDD (Hard Disk Drive), an SSD (Solid Storage Drive), or a memory having equivalent functions. The storage device 211 drives storage and stores programs executed by the CPU 201 and various data.

The drive 212 is a reader/writer for storage media, and is built in or externally attached to the point cloud processing apparatus 200 . The drive 212 reads out information recorded in the attached removable storage medium 24 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and outputs it to the RAM 203 or the storage device 211 . Drive 212 can also write information to removable storage medium 24 .

The network interface 215 is, for example, a communication interface configured with a communication device or the like for connecting to a network. The network interface 215 may be a wireless LAN (Local Area Network) compatible communication device or a wired communication device that performs wired communication.

<Supplement>
Although the preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention belongs can conceive of various modifications or modifications within the scope of the technical idea described in the claims. It is understood that these also naturally belong to the technical scope of the present invention.

For example, each step in the processing of the point cloud processing device 200 of this specification does not necessarily have to be processed in chronological order according to the order described as the flowchart. For example, each step in the processing of the point cloud processing device 200 may be processed in an order different from the order described as the flowchart, or may be processed in parallel.

In the above description, an example of generating the integrated background point cloud Pb by adding partial point clouds to the initial background point cloud has been described. An integrated background point group Pb may be generated by integration.

Also, the object measured by the LIDAR 100 according to the embodiment of the present invention is not particularly limited. For example, the object measured by the LIDAR 100 may be indoors, outdoors, or a construction site.

It is also possible to create a computer program for causing hardware such as the CPU, ROM, and RAM built into the point cloud processing device 200 to exhibit functions equivalent to the components of the point cloud processing device 200 described above. A storage medium storing the computer program is also provided.

100 LIDAR
110 Tripod 200 Point cloud processing device 220 Point cloud storage unit 230 Background extraction unit 240 Background point cloud storage unit 250 Background integration unit 260 Object detection unit 270 Display control unit 280 Display unit

Claims (12)

Based on the distribution of the three-dimensional point cloud corresponding to each timing, from the three-dimensional point cloud corresponding to each timing obtained by measuring the real space with LIDAR from the same point at each of a plurality of timings, a background extracting unit that extracts a background point cloud that is a point cloud belonging to a background region;
a background integration unit that generates an integrated background point cloud by integrating at least part of each background point cloud extracted from the point cloud corresponding to each of the plurality of timings;
with
The background integration unit extracts a new partial point cloud from the new background point cloud extracted by the background extraction unit, and repeats integrating the new partial point cloud into the integrated background point cloud, repeatedly updating the integrated background point cloud, and deleting some points that make up the integrated background point cloud when the number of points that make up the integrated background point cloud exceeds an upper limit; Point cloud processor. From the three-dimensional point cloud corresponding to each timing obtained by measuring the real space with LIDAR from the same point at each of a plurality of timings, based on the distribution of the three-dimensional point cloud corresponding to each timing, a background extracting unit that extracts a background point cloud that is a point cloud belonging to a background region;
a background integration unit that generates an integrated background point cloud by integrating at least part of each background point cloud extracted from the point cloud corresponding to each of the plurality of timings;
with
The background integration unit randomly extracts a new partial point cloud from the new background point cloud extracted by the background extraction unit, and repeats integrating the new partial point cloud into the integrated background point cloud. A point cloud processing device that repeatedly updates the integrated background point cloud by: 2. The point cloud processing device according to claim 1 , wherein some of the points forming said integrated background point group are random points forming said integrated background point group. 4. The point cloud processing device according to claim 1 , wherein said background integration unit randomly extracts said new partial point cloud from said new background point cloud. 3. The point cloud processing device according to claim 2 , wherein the background integration unit randomly extracts the new partial point clouds from different ranges for each of the plurality of new background point clouds. The point cloud processing device is
The point cloud processing device according to any one of claims 1 to 5 , further comprising an object detection unit that detects an area in which an object exists by calculating a difference between a certain point cloud and the integrated background point cloud. The object detection unit includes an integrated background point obtained by integrating a point cloud corresponding to a first timing and a partial point cloud extracted from a point cloud corresponding to a second timing before the first timing. 7. The point cloud processing device according to claim 6 , wherein an area in which an object exists is detected by calculating a difference with the group. The point cloud processing apparatus according to any one of claims 1 to 7 , wherein the background area includes at least one of a floor area and a wall area. From the three-dimensional point cloud corresponding to each timing obtained by measuring the real space with LIDAR from the same point at each of a plurality of timings, based on the distribution of the three-dimensional point cloud corresponding to each timing, extracting a background point cloud, which is a point cloud belonging to the background region;
generating an integrated background point cloud by integrating at least a portion of each background point cloud extracted from the point cloud corresponding to each of the plurality of timings;
including
Generating the integrated background point cloud includes extracting a new partial point cloud from the extracted new background point cloud and integrating the new partial point cloud into the integrated background point cloud, repeatedly updating the integrated background point cloud, and deleting some points constituting the integrated background point cloud when the number of points constituting the integrated background point cloud exceeds an upper limit value. Processing method. From the three-dimensional point cloud corresponding to each timing obtained by measuring the real space with LIDAR from the same point at each of a plurality of timings, based on the distribution of the three-dimensional point cloud corresponding to each timing, extracting a background point cloud, which is a point cloud belonging to the background region;
generating an integrated background point cloud by integrating at least a portion of each background point cloud extracted from the point cloud corresponding to each of the plurality of timings;
including
Generating the integrated background point cloud includes randomly extracting a new partial point cloud from the extracted new background point cloud, and repeating the process of integrating the new partial point cloud into the integrated background point cloud. A point cloud processing method, comprising iteratively updating the integrated background point cloud by: the computer,
From the three-dimensional point cloud corresponding to each timing obtained by measuring the real space with LIDAR from the same point at each of a plurality of timings, based on the distribution of the three-dimensional point cloud corresponding to each timing, a background extracting unit that extracts a background point cloud that is a point cloud belonging to a background region;
a background integration unit that generates an integrated background point cloud by integrating at least part of each background point cloud extracted from the point cloud corresponding to each of the plurality of timings;
with
The background integration unit extracts a new partial point cloud from the new background point cloud extracted by the background extraction unit, and repeats integrating the new partial point cloud into the integrated background point cloud, A point cloud processing device that repeatedly updates the integrated background point cloud, and deletes some points that make up the integrated background point cloud when the number of points that make up the integrated background point cloud exceeds an upper limit value. A program to function as the computer,
From the three-dimensional point cloud corresponding to each timing obtained by measuring the real space with LIDAR from the same point at each of a plurality of timings, based on the distribution of the three-dimensional point cloud corresponding to each timing, a background extracting unit that extracts a background point cloud that is a point cloud belonging to a background region;
a background integration unit that generates an integrated background point cloud by integrating at least part of each background point cloud extracted from the point cloud corresponding to each of the plurality of timings;
with
The background integration unit randomly extracts a new partial point cloud from the new background point cloud extracted by the background extraction unit, and repeats integrating the new partial point cloud into the integrated background point cloud. A program for functioning as a point cloud processing device that repeatedly updates the integrated background point cloud by.

Images