WO2024063078A1 - Method and system for generating 3-dimensional map data - Google Patents

Abstract

Provided is a method for generating 3-dimensional map data representing the 3-dimensional structure of a drive environment of a vehicle, wherein: a first 2-dimensional measurement area sensor (11) that acquires 3-dimensional point cloud data by measuring distance to a target point on a plane diagonally intersecting a central axis CL in the front-rear direction of a dolly (10), and an IMU that acquires positioning information enabling identification of the position and orientation of the dolly (10) are utilized to perform a coordinate conversion process based on the positioning information acquired by the IMU, on the 3-dimensional point cloud data obtained by the 2-dimensional measurement area sensor (11), to thereby convert the data into 3-dimensional point cloud data in which criterion position and orientation are identified; and the 3-dimensional point cloud data after the coordinate conversion is mapped onto a 3-dimensional space to generate 3-dimensional map data.

Description

3D map data generation method and generation system

The present invention relates to the generation of three-dimensional map data representing the three-dimensional structure of a vehicle driving environment.

In recent years, various driving support technologies have been proposed to reduce the burden of driving a vehicle. Examples of driving support technologies include technologies that support driving by having the vehicle side take charge of a portion of vehicle control, such as brake control in an automatic braking function and steering control in a lane keeping function. Furthermore, there are proposals for advanced driving support technology that realizes automatic driving by executing almost all vehicle controls such as steering control and speed control on the vehicle side, reducing the operational burden on the driver to near zero (see the following patent document) (See 1).

Additionally, automatic guided vehicles are widely used in factories, distribution warehouses, etc. As a system for automatically traveling an automatic guided vehicle, there has been proposed a system for an automatic guided vehicle that automatically travels while detecting structures and obstacles in the surrounding environment (see Patent Document 2 below).

In order to realize advanced vehicle control, such as advanced driving support technology for vehicle operation and autonomous driving of automated guided vehicles that do not rely on floor markings such as magnetic tape, it is necessary for the vehicle to be able to accurately grasp the driving environment. It is necessary that the For example, in order to realize autonomous vehicle driving, a highly accurate three-dimensional map containing three-dimensional information representing the three-dimensional structure of the surrounding environment is required.

For example, to generate three-dimensional map data representing the road environment, three-dimensional point cloud data that includes information on the distance and direction to each point of features such as curbs, guardrails, and signs is required. For example, a mobile mapping system (MMS) has been proposed that acquires three-dimensional point cloud data based on distance images acquired by a stereo camera, laser scanner, etc. (see, for example, Patent Document 3). Vehicles equipped with a mobile mapping system can acquire three-dimensional point cloud data while driving.

Japanese Patent Application Publication No. 2018-128364 JP2021-96731A Japanese Patent Application Publication No. 2010-191066

Distance images tend to have a large amount of information because they include distance information for each position within a two-dimensional area, and when generating three-dimensional map data based on these distance images, the amount of calculation processing is enormous. Therefore, there is a high possibility that three-dimensional map data cannot be efficiently generated.

The present invention has been made in view of the above-mentioned conventional problems, and aims to provide a method or system for efficiently generating three-dimensional map data.

One aspect of the present invention is a method for generating three-dimensional map data representing a three-dimensional structure of a vehicle's driving environment using a measurement vehicle that is movable on a moving plane that is a floor or ground, the method comprising the steps of:
a first two-dimensional range sensor attached to the measurement vehicle so as to acquire three-dimensional point cloud data by measuring distances to one or more target points on a first plane that is perpendicular or oblique to a central axis of the measurement vehicle in a longitudinal direction;
a positioning information acquisition circuit for acquiring positioning information capable of identifying the position and direction of the measurement vehicle;
acquiring the three-dimensional point cloud data by the first two-dimensional range sensor while acquiring the positioning information by the positioning information acquisition circuit;
converting the three-dimensional point cloud data obtained by the first two-dimensional range sensor into three-dimensional point cloud data in which a reference position and orientation are specified by performing a coordinate transformation process based on the positioning information acquired by the positioning information acquisition circuit;
The present invention relates to a method for generating three-dimensional map data by mapping three-dimensional point cloud data after coordinate transformation into a three-dimensional space.

One aspect of the present invention is a system that generates three-dimensional map data representing a three-dimensional structure of a vehicle driving environment using a measurement vehicle that can move on a moving plane such as a floor or ground surface, the system comprising:
The measurement vehicle is configured such that three-dimensional point cloud data can be obtained by measuring the distance to one or more target points on a first plane perpendicular or oblique to the central axis in the longitudinal direction of the measurement vehicle. a first two-dimensional range sensor attached to the
a positioning information acquisition circuit that acquires positioning information that can identify the position and direction of the measurement vehicle;
A coordinate conversion process based on the positioning information acquired by the positioning information acquisition circuit is applied to the three-dimensional point cloud data obtained by the first two-dimensional range sensor, and a three-dimensional point cloud whose reference position and orientation are specified. A coordinate conversion circuit that converts into data,
A 3D map data generation system includes a mapping circuit that generates a 3D map by mapping 2D point cloud data after coordinate conversion onto a 3D space.

The three-dimensional map generation method and generation system of the present invention is a method for generating a three-dimensional map based on three-dimensional point cloud data acquired by a first two-dimensional range sensor attached to a measuring vehicle; It is a system.

The first two-dimensional range sensor in the present invention is attached to the measuring vehicle so as to be able to acquire three-dimensional point cloud data on a first plane that is perpendicular or oblique to the longitudinal central axis of the measuring vehicle. ing. Acquiring three-dimensional point cloud data by the first two-dimensional range sensor while the measurement vehicle is moving corresponds to scanning the driving environment with the first two-dimensional range sensor.

In the present invention, coordinate transformation is performed on the three-dimensional point group data obtained by the first two-dimensional range sensor based on the positioning information acquired by the positioning information acquisition circuit. Then, a three-dimensional map is generated by mapping the three-dimensional point group data after coordinate conversion onto a three-dimensional space.

Incidentally, a three-dimensional range sensor is a sensor that acquires three-dimensional information of a three-dimensional space in which the depth of each point in a two-dimensional area is taken into consideration as a distance image. On the other hand, a two-dimensional range sensor is a sensor that acquires three-dimensional information on a plane in which the depth of each point on a one-dimensional line is considered. Compared to 3D point cloud data acquired by a 3D range sensor, 3D point cloud data acquired by a 2D range sensor has a much smaller amount of information. Therefore, by using the three-dimensional point cloud data acquired by the two-dimensional range sensor, it is possible to suppress the amount of calculation processing required to generate the three-dimensional map. According to the present invention, a three-dimensional map can be efficiently generated.

FIG. 2 is a perspective view of a truck that is an example of a measurement vehicle in Example 1. FIG. 1 is a system diagram showing the configuration of a three-dimensional map generation system in Example 1. FIG. FIG. 3 is a diagram illustrating the relationship between the central axis CL of the cart and the plane DS on which the two-dimensional range sensor detects an object in Example 1. FIG. 3 is a flow diagram showing the flow of three-dimensional map generation processing in Example 1. FIG. FIG. 3 is an explanatory diagram showing how the cart acquires three-dimensional point group data in Example 1. FIG. FIG. 3 is a perspective view of another truck in Example 1. FIG. 7 is an explanatory diagram of a magnetic marker in Example 2. FIG. 3 is an explanatory diagram of a truck in Example 2. FIG. 3 is a system diagram showing the configuration of a three-dimensional map generation system in Example 2. FIG. 3 is a flow diagram showing the flow of three-dimensional map generation processing in Example 2. FIG. FIG. 7 is a diagram illustrating the relationship between the central axis CL of the cart and the plane DS on which the first and second two-dimensional range sensors detect objects in Example 3; FIG. 3 is a system diagram showing the configuration of a three-dimensional map generation system in Example 3. FIG. FIG. 7 is a flowchart showing the flow of three-dimensional map generation processing in Example 3. FIG. 7 is an explanatory diagram of a calibration post in Example 4. 7 is a flowchart showing the flow of calibration processing in Example 4. FIG. FIG. 7 is an explanatory diagram showing how the trolley observes the calibration post in Example 4. FIG. 7 is an explanatory diagram showing a truck in Example 5. FIG. 7 is an explanatory diagram of the truck seen from the rear side in the fifth embodiment.

Embodiments of the present invention will be specifically described using the following examples.
(Example 1)
This example relates to a method and system for generating three-dimensional map data using the two-dimensional range sensor 11. The contents will be explained with reference to FIGS. 1 to 6.

In this example, a generation system 1 and a generation method in which a trolley 10 (FIG. 1), which is an example of a measurement vehicle, moves on a floor surface (moving plane) and generates three-dimensional map data of a driving environment will be described. The driving environment in this example is a driving environment inside a facility such as a factory. The generated three-dimensional map data is used by an automated transport vehicle (not shown) to move within the facility.

The generation system 1 (FIG. 2) of this example includes a trolley 10, a two-dimensional range sensor 11 that acquires three-dimensional point cloud data, an IMU (Inertial Measurement Unit) 22 that enables inertial navigation, a coordinate conversion circuit 131, and a mapping It is configured to include a circuit 133 and a storage section 135 that stores three-dimensional map data. In the configuration of this example, a two-dimensional range sensor 11, an IMU 22, a coordinate conversion circuit 131, a mapping circuit 133, and a storage unit 135 for storing three-dimensional map data are provided in a trolley 10, which is an example of a measurement vehicle.

The trolley 10 is a four-wheeled vehicle configured to automatically travel along a pre-programmed predetermined route. The size of the trolley 10 is approximately 2 m in total length, 1 m in width, and 1 m in height. The trolley 10 is equipped with a steering unit for steering wheels, a drive motor for rotationally driving the drive wheels, a control unit 13, and the like to enable automatic travel.

The control unit 13 is a unit that controls the traveling of the cart 10 so that it can move along a predetermined route. The control unit 13 reads route data from a storage device that constitutes the memory section 135, and controls the cart 10. In the configuration of this example, the control unit 13 mounted on the cart 10 realizes the functions of the coordinate conversion circuit 131 and mapping circuit 133 described above.

Note that while the coordinate conversion circuit 131, the mapping circuit 133, and the storage unit 135 are necessary components of the generation system 1, it is not essential that they be incorporated into the trolley 10. For example, the information acquired by the two-dimensional range sensor 11 and IMU 22 may be wirelessly transmitted to a server device or the like. In this case, the server device or the like can execute processing for generating a three-dimensional map.

The two-dimensional range sensor 11 (an example of a first two-dimensional range sensor) detects an object located on a plane DS (see FIG. 3) obliquely intersecting the central axis CL of the truck 10 in the front-rear direction. This sensor acquires three-dimensional point cloud data by measuring the distance to. Note that the plane DS on which the two-dimensional range sensor 11 acquires the three-dimensional point group data may be a plane orthogonal to the central axis CL.

The IMU 22 (Figure 2) is an inertial navigation unit that estimates the displacement and orientation change of the trolley 10 by inertial navigation, and is an example of a positioning information acquisition circuit. The IMU 22 is equipped with a magnetic sensor 221, an acceleration sensor 222, and a gyro sensor 223. The displacement estimated by the IMU 22 is the positional fluctuation amount of the trolley 10. Similarly, the orientation change amount is the orientation change amount that indicates the direction of the trolley 10. The two-dimensional range sensor 11 is fixed to the trolley 10. The displacement and orientation change amount estimated by the IMU 22 can be treated as the displacement or orientation change amount of the two-dimensional range sensor 11.

In the configuration of this example, the control unit 13 estimates the latest vehicle position or vehicle orientation based on the displacement amount and orientation change amount estimated by the IMU 22. The control unit 13 then controls the traveling of the trolley 10 so that it moves along a predetermined route. Route data representing a predetermined route is stored in a storage device forming the storage section 135 described above so that the control unit 13 can read it. In the configuration of this example, initial values whose absolute values are known are set for the vehicle position and vehicle direction. The control unit 13 estimates the latest vehicle position and vehicle orientation by adding the displacement amount and orientation change amount estimated by the IMU 22 to the initial position and initial orientation.

The coordinate conversion circuit 131 is a circuit that performs coordinate conversion processing on the three-dimensional point group data acquired by the two-dimensional range sensor 11. The three-dimensional point cloud data acquired by the two-dimensional range sensor 11 is point cloud data in a local coordinate system based on the direction of the two-dimensional range sensor 11. The coordinate conversion circuit 131 converts three-dimensional point group data belonging to the local coordinate system to three-dimensional point group data belonging to the global coordinate system. This conversion is realized by coordinate conversion processing based on the vehicle position and vehicle orientation estimated by inertial navigation.

The mapping circuit 133 is a circuit that generates three-dimensional map data by mapping three-dimensional point cloud data onto a three-dimensional space. The three-dimensional point group data mapped by the mapping circuit 133 is three-dimensional point group data belonging to the global coordinate system after coordinate transformation. The mapping circuit 133 executes a process of additionally writing three-dimensional point cloud data to the three-dimensional map data stored in the storage unit 135 every time new three-dimensional point cloud data is acquired.

Next, the flow of processing executed by the generation system 1 of this example will be explained with reference to the flow diagram of FIG. 4. The flowchart in the figure shows the flow of three-dimensional map generation processing. Note that a description of the process for automatically driving the trolley 10 will be omitted.

While the trolley 10 is moving, the IMU 22 estimates the amount of displacement and the amount of change in direction (S101), and the two-dimensional range sensor 11 acquires three-dimensional point cloud data (S102). As shown in FIG. 5, the cart 10 acquires three-dimensional point group data while changing its position as it moves. The IMU 22 estimates the amount of displacement or the amount of change in orientation based on the initial position whose absolute position is known and the orientation whose absolute orientation is known. The control unit 13 estimates the latest vehicle position and vehicle orientation based on the displacement amount and orientation change amount estimated by the IMU 22 (S103).

The coordinate conversion circuit 131 (control unit 13) performs coordinate conversion processing on three-dimensional point cloud data belonging to a local coordinate system based on the two-dimensional range sensor 11, and converts it into three-dimensional point cloud data belonging to a global coordinate system. (S104). Specifically, the coordinate conversion circuit 131 executes the above coordinate conversion process based on the vehicle position and vehicle orientation estimated by the control unit 13.

The mapping circuit 133 (control unit 13) maps the three-dimensional point group data of the global coordinate system after the coordinate transformation onto a three-dimensional space (S105), thereby generating a three-dimensional map. This three-dimensional map is not complete and covers every part of the driving environment. The three-dimensional map includes three-dimensional information of an area of the driving environment that corresponds to the route traveled by the trolley 10. As shown in FIG. 5, the degree of completion of the three-dimensional map improves as the distance traveled by the cart 10 increases.

Note that, as shown in FIG. 6, in addition to the two-dimensional range sensor 11, an additional two-dimensional range sensor (an example of a third two-dimensional range sensor) 113 may be provided on the trolley 10. In this case, the amount of information in the three-dimensional map data can be increased. For example, as shown in FIG. 6, in the case of a two-dimensional range sensor 11 whose measurement surface is a plane DS that slopes downward, is the object located at a lower position or higher than the mounting position of the two-dimensional range sensor 11? , the timing and aspects in which three-dimensional information can be acquired differ depending on the situation. In the case of an object located at a lower position than the mounting position of the two-dimensional range sensor 11, when the trolley 10 approaches, three-dimensional information of the front surface facing the trolley 10 can be acquired. On the other hand, in the case of an object located at a higher position than the mounting height of the two-dimensional range sensor 11, three-dimensional information on the back surface can only be acquired after the trolley 10 has passed. In this way, in the case of only the two-dimensional range sensor 11 whose measurement surface is the forward-sloping plane DS, for objects lower than the two-dimensional range sensor 11, the front surface facing the trolley 10 as it approaches It is difficult to obtain three-dimensional information about the surface on the back side. Similarly, for objects located higher than the two-dimensional range sensor 11, while it is possible to obtain three-dimensional information on the back surface, it is difficult to obtain three-dimensional information on the front surface.

As such, it is advisable to combine a two-dimensional range sensor 113 whose measurement surface is a forward-rising plane DS (an example of a third plane), as shown in Figure 6. The two-dimensional range sensor 113 can acquire three-dimensional information on a surface that cannot be acquired by the two-dimensional range sensor 11 whose measurement surface is a forward-sloping plane DS. The three-dimensional point cloud data acquired by the two two- dimensional range sensors 11 and 113 in Figure 6 can be handled in the same way, and it is advisable to map the three-dimensional point cloud data after coordinate transformation for each.

In addition to the two-dimensional range sensor 11, a two-dimensional range sensor is provided whose measurement surface is a plane parallel to the central axis CL of the cart 10 (see FIG. 3) and along the vertical direction (vertical plane). That's good too. By using the three-dimensional point group data of this two-dimensional range sensor, it is possible to easily detect the angular change in the pitch direction of the trolley 10. For example, by focusing on the displacement of one target point in two three-dimensional point cloud data acquired at different times, the angular change in the pitch direction of the trolley 10 can be estimated with high accuracy.

In this example, a route inside a facility such as a factory is exemplified as the driving environment. The driving environment may be a road on which ordinary vehicles travel or a road exclusively for buses. When the driving environment is a road, the measurement vehicle generates three-dimensional map data while moving on the road surface (moving plane).

(Example 2)
This example is an example in which the generation system of Example 1 is used as a reference, and the accuracy of estimating the vehicle position and vehicle orientation is improved by using the magnetic marker 100 disposed on the floor. The contents will be explained with reference to FIGS. 7 to 10.

The magnetic marker 100 is a permanent magnetic sheet that can be attached to the floor of the track, as shown in FIG. The magnetic marker 100 has a circular sheet shape with a diameter of 100 mm and a thickness of about 2 mm. The permanent magnet sheet is a ferrite rubber magnet sheet made of a polymer material in which iron oxide magnetic powder is dispersed and formed into a sheet shape.

An RFID (Radio Frequency IDentification) tag 15, which is a wireless tag that outputs information wirelessly, is attached to the surface 100S of the magnetic marker 100. The RFID tag 15 is a sheet-shaped electronic component in which an IC chip is mounted on the surface of a tag sheet cut out from, for example, a PET (Polyethylene terephthalate) film.

The RFID tag 15, which constitutes the information providing unit, operates using power supplied wirelessly from the trolley (measurement vehicle) 10, and transmits a tag ID (identification information), which is an example of unique information of the magnetic marker 100. On the trolley 10 side, the memory unit 135 (Figure 9) stores the position information of the magnetic marker 100 linked to the tag ID. If the trolley 10 side can obtain the tag ID of the magnetic marker 100, it is possible to read the position information of the corresponding magnetic marker 100.

Note that the ferrite rubber magnet forming the magnetic marker 100 has magnetic particles dispersed in a polymeric material, and therefore has an electrical characteristic of low conductivity. Therefore, the magnetic marker 100 can suppress the generation of eddy currents and the like in response to wireless power supply to the RFID tag 15. The RFID tag 15 attached to the magnetic marker 100 can efficiently receive wirelessly transmitted power.

As shown in FIGS. 8 and 9, the trolley 10 includes a sensor array 21 including a magnetic sensor Cn and a tag reader 24 in addition to the configuration of the first embodiment. As shown in FIG. 9, the sensor array 21 includes 15 magnetic sensors Cn (n is an integer from 1 to 15) arranged in a straight line, and a detection processing circuit 212 incorporating a CPU (not shown). . Note that in the sensor array 21, 15 magnetic sensors Cn are arranged at equal intervals of 5 cm.

The magnetic sensor Cn is a sensor that detects magnetism using the known MI effect (Magnet Impedance Effect). The MI effect is a magnetic effect in which the impedance of a magnetically sensitive material such as an amorphous wire changes sensitively in response to an external magnetic field. The magnetic sensor Cn is capable of detecting magnetism acting in the axial direction of the linearly incorporated amorphous wire. Each magnetic sensor Cn is incorporated into the sensor array 21 so that the magnetic detection direction is the same. The sensor array 21 is attached to the trolley 10 so that each magnetic sensor Cn can detect magnetism in the vertical direction. In the configuration of this example, this sensor array 21 is attached to the truck 10 along the vehicle width direction.

The detection processing circuit 212 of the sensor array 21 is an arithmetic circuit that executes marker detection processing for detecting the magnetic marker 100 and the like. The detection processing circuit 212 is configured using a CPU (central processing unit) that executes various calculations, memory elements such as ROM (read only memory) and RAM (random access memory), and the like.

The detection processing circuit 212 acquires sensor signals output from each magnetic sensor Cn at a cycle of, for example, 3 kHz, and executes marker detection processing. In the marker detection process, in addition to detecting the magnetic marker 100, the amount of lateral deviation of the cart 10 with respect to the magnetic marker 100 is measured. The detection processing circuit 212 measures the amount of lateral deviation, for example, by identifying the position in the vehicle width direction of the peak value of the distribution of magnetic measurement values of the magnetic sensors Cn arranged in the vehicle width direction. When the detection processing circuit 212 detects the magnetic marker 100, it inputs the detection result including the amount of lateral shift to the control unit 13 along with the fact that the magnetic marker 100 has been detected.

The tag reader 24 (FIGS. 8 and 9) is a communication unit that wirelessly communicates with the RFID tag 15 (FIG. 7) attached to the magnetic marker 100. The tag reader 24 wirelessly transmits the power necessary for the operation of the RFID tag 15 and receives the tag ID transmitted by the RFID tag 15. The tag reader 24 may also be configured to start operating in response to the detection of the magnetic marker 100.

Next, the operation of the generation system 1 of this example will be explained with reference to the flow diagram of FIG. The processing flow in the figure is based on the processing flow in FIG. 4 referred to in the first embodiment, with steps S111 to S115 added. When the determination in step S111 is YES, that is, when the magnetic marker 100 is detected, the processes in steps S112 to S115 are executed. On the other hand, if the determination in step S111 is NO, that is, if the magnetic marker 100 is not detected, the processes in steps S112 to S115 are not executed and are bypassed. The processing contents in this case are exactly the same as the processing according to the flowchart of FIG. 4 of the first embodiment.

Step S111 of determining whether a magnetic marker has been detected is executed after three-dimensional point group data is acquired (S102) and the vehicle position and vehicle orientation are estimated by inertial navigation (S103). In step S111, the presence or absence of detection of the magnetic marker 100 is determined using the detection result of the marker detection process input from the detection processing circuit 212. Note that, as described above, the detection result when the magnetic marker 100 is detected includes the fact that the magnetic marker 100 has been detected, the amount of lateral deviation of the trolley 10 with respect to the detected magnetic marker 100, and the like.

If the magnetic marker 100 is detected (S111: YES), the tag ID is read from the RFID tag 15 held by the magnetic marker 100 (S112). The control unit 13 uses the read tag ID to refer to the stored data in the storage unit 135 and acquires the position information of the magnetic marker 100 to which the tag ID is attached (S113). Then, based on the placement position of the magnetic marker 100 represented by the acquired position information, a position shifted by the amount of lateral shift included in the detection result of the marker detection process is specified by calculation (S114, positioning information acquisition circuit). Control unit 13 replaces the vehicle position estimated in step S103 with the vehicle position specified in step S114 (S115).

As described above, in the generation system 1 of this example, the vehicle position is estimated with high accuracy using the magnetic markers 100 arranged on the road. In estimating a vehicle position using inertial navigation, accumulation of errors is inevitable in principle. While estimating the vehicle position etc. by inertial navigation, when the magnetic marker 100 is detected, if the vehicle position is estimated using the magnetic marker 100 as a reference, the error accumulated by inertial navigation can be reset. In inertial navigation after the vehicle position has been estimated according to the detection of the magnetic marker 100, it is preferable to estimate the amount of displacement based on the vehicle position.

It should be noted that the acceleration sensor, gyro sensor, etc. included in the IMU are prone to offset errors due to temperature changes, changes over time, and the like. The offset error is essentially a so-called zero point shift in which the sensor output does not become zero in a situation where the sensor output is zero. A zero point shift of an acceleration sensor or a gyro sensor causes an estimation error in inertial navigation. When the magnetic marker 100 is detected, it is also possible to specify the positional deviation between the vehicle position determined by inertial navigation and the vehicle position based on the magnetic marker 100. Estimated errors due to inertial navigation can be identified based on the positional deviation of the vehicle position due to inertial navigation, thereby making it possible to calibrate sensors such as acceleration sensors that acquire inertial information.

It is also good to arrange a sub-magnetic marker for estimating the vehicle heading near the magnetic marker 100 (main magnetic marker). It is also good to link information on the absolute orientation connecting the main magnetic marker and the sub magnetic marker to the tag ID together with the position information. The distance between the main magnetic marker and the sub magnetic marker is preferably set to a short distance that is less than the entire length of the cart 10. With such a short distance, changes in the orientation of the truck 10 can be ignored, and the orientation of the truck 10 can be estimated with high accuracy (positioning information acquisition circuit). When the absolute heading (vehicle heading) can be estimated using the magnetic marker 100, it is preferable to replace the vehicle heading estimated by inertial navigation with the newly estimated absolute heading.

It is also good to attach a sub magnetic marker to all the magnetic markers 100. In this case, the vehicle position and vehicle orientation can be estimated every time the (main) magnetic marker 100 is detected, so a system configuration that does not include the IMU 22 is possible. If it is combined with the IMU 22, it is sufficient to simply attach sub magnetic markers to some of the magnetic markers 100.

In the case of a system configuration that does not include the IMU 22, three-dimensional point cloud data may be acquired in response to detection of the magnetic marker 100. The three-dimensional point cloud data may be subjected to coordinate conversion in response to the vehicle position and vehicle orientation estimated using the magnetic marker 100, and may be mapped into a three-dimensional space. In this case, a three-dimensional map including discrete three-dimensional data for each position of the magnetic marker 100 may be generated. For example, an automated guided vehicle that uses the three-dimensional map may refer to the three-dimensional map every time it detects a magnetic marker 100.
Other configurations and effects are the same as those of the first embodiment.

(Example 3)
This example is based on the generation system of Example 1, and employs a two-dimensional range sensor 112 (an example of a second two-dimensional range sensor) instead of the IMU in order to estimate the vehicle position and vehicle direction. This is an example. The contents will be explained with reference to FIGS. 11 to 13.

The trolley 10 (FIG. 11) that constitutes the generation system 1 of this example is equipped with a two-dimensional range sensor 11 that acquires three-dimensional point cloud data that is the source data of three-dimensional map data, as well as a vehicle position and vehicle orientation. It is equipped with a two-dimensional range sensor 112 that acquires three-dimensional point cloud data for estimation. The plane DS (an example of a second plane) on which the two-dimensional range sensor 112 measures distance is a horizontal plane parallel to the center axis CL of the cart 10 in the front-rear direction.

The control unit 13 (FIG. 12) is configured to estimate the vehicle position and vehicle orientation using time-series three-dimensional point group data acquired by the two-dimensional range sensor 112. The control unit 13 estimates the amount of displacement and the amount of change in direction of the truck 10 by processing this three-dimensional point group data. For example, if the distance to any one object and the change in direction over time are known, the amount of displacement and the amount of change in direction of the trolley 10 can be estimated. The control unit 13 calculates the latest vehicle position by adding the estimated displacement amount or azimuth change amount to the reference vehicle position whose absolute position is known and the reference vehicle direction whose absolute direction is known. , estimate vehicle heading.

The flow of the operation by the generation system 1 of this example will be explained with reference to the flow diagram of FIG. 13. While the trolley 10, which is an example of a measurement vehicle, is moving, three-dimensional point cloud data (A) by the two-dimensional range sensor 11 and three-dimensional point cloud data (B) by the two-dimensional range sensor 112 are acquired (S121 ). The control unit 13 estimates the amount of displacement and the amount of change in direction of the bogie 10 by processing the three-dimensional point group data (B) (S122), and further estimates the vehicle position and vehicle direction (S123, positioning information acquisition circuit).

The control unit 13 performs coordinate transformation processing on the three-dimensional point group data (A) acquired by the two-dimensional range sensor 11 based on the estimated vehicle position and vehicle orientation (S124). Then, the control unit 13 generates three-dimensional map data by mapping the three-dimensional point group data after the coordinate transformation (S125).

In addition to mapping the 3D point cloud data from the 2D range sensor 11, 3D point cloud data obtained by performing coordinate transformation processing on the 3D point cloud data from the 2D range sensor 112 is mapped. That's good too.

In addition to the two- dimensional range sensors 11 and 112, it is also possible to provide a two-dimensional range sensor whose measurement surface is a plane parallel to the central axis CL of the trolley 10 and along the vertical direction. In this case, the angular change in the pitch direction of the truck 10 can be detected using the three-dimensional point group data of the two-dimensional range sensor.

Note that it is also possible to combine the system of this example with an IMU. In this case, the vehicle position and vehicle direction are estimated by inertial navigation based on the positioning information acquired by the IMU, while the estimation error of inertial navigation is identified by the positioning information based on the 3D point cloud data from the 2D range sensor 112. It's also good to do. If the estimation error of inertial navigation can be identified, then it is possible to make corrections to improve the estimation accuracy of inertial navigation.
Note that the other configurations and effects are the same as in Example 1.

(Example 4)
This example is based on the generation system of Example 1, and adds a mechanism that enables calibration of the sensor included in the IMU. The contents will be explained with reference to FIGS. 14 to 16.

The generation system 1 of this example uses a calibration post (an example of a rod-shaped member) 101 installed in the driving environment to calibrate a sensor (IMU) that measures inertial information, such as an acceleration sensor or a gyro sensor. This is a system that makes it possible.

The calibration post 101 (FIG. 14) is a cylinder with a diameter of 10 cm that is erected on the floor along the vertical direction (an example of a known orientation). The height of the upper end of the calibration post 101 is 1 m. The calibration post 101 is installed, for example, at a side of the route along which the trolley 10 travels, or at the back of a corner, that is, at a location where it hits when viewed from the approach direction. The calibration post 101 is installed at a location where the flatness of the road surface is sufficiently high. Therefore, the azimuth change in the pitch direction when the trolley 10 observes the calibration post 101 can be ignored.

The calibration post 101 is provided at a location where there are no similar structures around it. Therefore, the calibration post 101 can be easily identified on the trolley 10 side. Note that it is also possible to provide a characteristic shape, such as providing a groove of a predetermined width extending in the vertical direction, on the surface of the side facing the cart 10 when it approaches. By measuring the cross-sectional shape of the above-mentioned surface of the calibration post 101, the calibration post 101 can be identified more easily.

Next, the operation of the generation system 1 of this example will be explained using the flow diagram of FIG. 15 and FIG. 16. Note that in the generation system 1 of this example, a calibration flag indicating a calibration period is set and managed by the control unit. When the calibration post 101 is observed for the first time, the calibration flag is set to 1 and the calibration period begins.

The trolley 10 acquires three-dimensional point group data while moving (S201). When the calibration flag is zero (S202: YES), the control unit processes the acquired three-dimensional point group data to determine whether the three-dimensional data of the calibration post 101 is included, that is, the calibration post 101. is observed (S203). If the calibration post 101 has not been observed, the calibration flag is zero, so acquisition of three-dimensional point group data is repeatedly executed in accordance with the flow of S201 → S202: YES → S203: NO.

If the calibration post 101 is observed when the calibration flag is zero (S202: YES → S203: YES), the control unit executes the process of step S204. In this step S204, the following three processes are executed. The first process is a process of extracting the 3D data of the observation point P1 of the calibration post 101 (see FIG. 16, an example of one target point) from the 3D point cloud data acquired in step S201 above. be. In the second process, the three-dimensional data of the observation point P2 of the calibration post 101 (see FIG. 16, an example of a prediction target point) is calculated when the trolley 10 travels and is displaced by a predetermined amount dx. , is a process calculated by simulation calculation. The third process is a process of setting the calibration flag to 1. When the calibration flag is set to 1, the calibration period begins.

The control unit repeatedly acquires the three-dimensional point group data until the estimated displacement amount by the IMU reaches the predetermined amount dx (S205: NO → S201). At this time, since the calibration period is in progress and the flag value of the calibration flag is not zero (S202: NO), the processes of steps S203 and S204 described above are bypassed. Here, the amount of displacement estimated by the IMU includes an estimation error caused by measurement errors of the acceleration sensor and the gyro sensor. Therefore, the actual displacement amount dxs when it is determined in step S205 that the estimated displacement amount has reached the predetermined amount dx is highly likely to be different from the predetermined amount dx.

When it is determined that the estimated displacement amount of the trolley 10 has reached the predetermined amount dx (S205: YES), the control unit calculates the observation point of the calibration post 101 from the actually measured three-dimensional point group data by the two-dimensional range sensor 11. P2s (see FIG. 16; an example of another target point) is extracted, and three-dimensional data of observation point P2s is specified (S206). Note that the estimated displacement amount of the truck 10 includes an estimation error due to inertial navigation. Therefore, as described above, when it is determined that the estimated displacement amount of the truck 10 has reached the predetermined amount dx, the actual displacement amount is dxs. The above three-dimensional data at the observation point P2s is not the data at the time when the displacement amount of the trolley 10 reaches dx, but the data at the time when the displacement amount reaches dxs.

The control unit compares the three-dimensional data of the observation point P2 of the calibration post 101 based on the simulation calculation with the three-dimensional data of the observation point P2s of the calibration post 101 based on the actual measurement, and identifies an error (estimated error) (S207 ). The three-dimensional data obtained by the simulation calculation is the three-dimensional data of the observation point P2 obtained in step S204 above. The three-dimensional data of the actual observation point P2s is the three-dimensional data of the observation point P2s specified in step S206 above.

The control unit uses the error identified in step S207 to calibrate the acceleration sensor and gyro sensor (S208). Calibration is, for example, so-called bias adjustment that adjusts the zero point of each sensor. Calibration of the acceleration sensor can be performed mainly based on the vertical positional error of the observation point P2 and the observation point P2s. Calibration of the gyro sensor can be performed mainly based on the horizontal positional error of observation point P2 and observation point P2s. After such calibration is performed, the calibration flag is reset to zero (S208).

The above explanation is based on the assumption that the floor surface is horizontal and the orientation of the truck 10 (vehicle direction) does not change in the pitching direction. However, the pitching direction may change depending on the acceleration/deceleration of the truck 10, the air pressure of the wheels, etc. Such a situation in which the pitching direction changes is not suitable for sensor calibration.

Therefore, it is also good to mount two two-dimensional range sensors on the trolley 10 that can simultaneously observe the calibration post 101. Attach a marking such as a horizontal bar or reflective tape to a position on the calibration post that is simultaneously observed by the two two-dimensional range sensors when there is no change in the pitching direction of the cart 10. That's good too. Alternatively, two two-dimensional range sensors may be attached so that the upper ends of the calibration posts can be observed simultaneously. When a change occurs in the pitching direction of the truck 10, the markings and the upper end cannot be observed simultaneously by the two two-dimensional range sensors. In such a case, there is a high possibility that high calibration accuracy cannot be ensured, so it is preferable not to perform sensor calibration.

For example, two markings that can be detected by a two-dimensional range sensor may be attached to the calibration post 101 at a predetermined interval. Since the distance between the two markings is known on the calibration post 101, the relative positional relationship in the three-dimensional space is known.

When one of the two markings (an example of the first target point) is observed by the two-dimensional range sensor 11, the other marking (the second It is preferable to predict the amount of change (predicted amount of change) in the position and orientation of the trolley 10 when the object point (an example of the target point) is observed. Further, when the trolley 10 actually moves after one marking is observed and the other marking is actually observed, the position of the trolley 10 relative to the reference position and orientation is determined based on the positioning information from the IMU. It is better to estimate the amount of change in direction (estimated amount of change).

By comparing the above predicted change amount and the above estimated change amount, it is possible to identify the error in the positioning information acquired by the IMU. If errors in positioning information can be identified, it is possible to correct the positioning information acquired by the IMU. Here, when comparing the three-dimensional position corresponding to the first three-dimensional data and the three-dimensional position corresponding to the second three-dimensional data, two types of three-dimensional positions are specified in a common coordinate system. There is a need to.

In this example, the calibration post 101 is illustrated as extending in the vertical direction, but the direction in which the calibration post 101 extends does not have to be the vertical direction, and may be any direction as long as it is in a known direction. The above two markings do not need to be at two locations on the calibration post 101. The two markings may be provided on separate structures. It is only necessary that the relative positional relationship of each marking is known in three-dimensional space and that the two-dimensional range sensor can detect each marking.
Note that the other configurations and effects are the same as in Example 1.

(Example 5)
This example is based on the configuration of Example 1, and is configured such that the roll of the truck 10 around the center axis CL can be detected. The contents will be explained with reference to FIGS. 17 and 18. FIG. 17 is a side view of the truck 10, and FIG. 18 is an explanatory view looking forward from the rear side of the truck 10.

The trolley 10 of this example includes a two-dimensional range sensor 11 disposed at the upper front side of the trolley 10, and a two-dimensional range sensor 114 (an example of a third two-dimensional range sensor) disposed at the lower front side. ). As described in the first embodiment, the measurement surface of the two-dimensional range sensor 11 is a plane DS that is parallel to the vehicle width direction and slopes downward toward the front. On the other hand, the measurement surface of the two-dimensional range sensor 114 is a plane DS (an example of a third plane) parallel to the vehicle width direction and rising forward.

As shown in FIG. 17, the front downward plane DS and the front upward plane DS intersect in front of the truck 10, and appear as an intersection in the figure. As shown in FIG. 18, the intersection of these two planes DS forms an intersection line Xc along the vehicle width direction. Since the two planes DS are both planes parallel to the vehicle width direction, the intersection line Xc between the two planes DS is also parallel to the vehicle width direction.

The intersection line Xc (FIG. 18) is a line that passes through two or more common target points among the target points observed by the two-dimensional range sensor 11 and the target points observed by the two-dimensional range sensor 114. It can be identified as As shown in FIG. 18, if the roll, which is the inclination of the cart 10 in the rotational direction around the central axis CL, is zero, the intersection line Xc is parallel to the floor surface 1S. The roll angle of the truck 10 can be estimated by examining the slope of the intersection line Xc with respect to the floor surface 1S.
Note that the other configurations and effects are the same as in Example 1.

Although specific examples of the present invention have been described above in detail as in the embodiments, these specific examples merely disclose examples of techniques included in the scope of the claims. Needless to say, the scope of the claims should not be interpreted to be limited by the configurations, numerical values, etc. of the specific examples. The scope of the claims includes techniques in which the specific examples described above are variously modified, changed, or appropriately combined using known techniques and the knowledge of those skilled in the art.

1 3D map data generation system (generation system)
10 Dolly (measurement vehicle)
11 2D range sensor (first 2D range sensor)
112 Two-dimensional range sensor (second two-dimensional range sensor)
113, 114 Two-dimensional range sensor (third two-dimensional range sensor)
100 Magnetic marker 101 Calibration post (rod-shaped member)
13 Control unit (positioning information acquisition circuit)
131 Coordinate conversion circuit 133 Mapping circuit 135 Storage unit 15 RFID tag 21 Sensor array 212 Detection processing circuit (marker detection circuit)
22 IMU (positioning information acquisition circuit)
221 Magnetic sensor 222 Acceleration sensor 223 Gyro sensor 24 Tag reader CL Central axis Cn (n is an integer from 1 to 15) Magnetic sensor DS Plane

Claims (15)

1. A method of generating three-dimensional map data representing a three-dimensional structure of a vehicle driving environment using a measurement vehicle moving on a moving plane that is a floor or ground surface, the method comprising:
   The measurement vehicle is configured such that three-dimensional point cloud data can be obtained by measuring the distance to one or more target points on a first plane perpendicular or oblique to the central axis in the longitudinal direction of the measurement vehicle. a first two-dimensional range sensor attached to the
   A positioning information acquisition circuit that acquires positioning information that can identify the position and direction of the measurement vehicle,
   A coordinate conversion process based on the positioning information acquired by the positioning information acquisition circuit is applied to the three-dimensional point cloud data from the first two-dimensional range sensor, thereby creating a three-dimensional image whose reference position and orientation are specified. Convert to point cloud data,
   A 3D map data generation method that generates 3D map data by mapping 3D point cloud data after coordinate transformation into a 3D space.
2. In claim 1, the positioning information acquired by the positioning information acquisition circuit includes an amount of displacement with respect to a position whose absolute position is known, and an amount of change in direction, which is an amount of change in direction with respect to a direction whose absolute direction is known. It contains
   The three-dimensional point group data obtained by the first two-dimensional range sensor is a method for generating three-dimensional map data, in which absolute three-dimensional coordinates can be specified based on the positioning information.
3. 2. The method of generating three-dimensional map data according to claim 1, wherein the positioning information acquisition circuit acquires the positioning information by estimating the position and orientation of the measurement vehicle using inertial navigation.
4. In claim 1, the measurement vehicle includes a marker detection circuit for detecting markers arranged on a road surface on which the vehicle travels,
   The positioning information acquisition circuit acquires the positioning information by specifying the position of the measurement vehicle based on the marker installation position detected by the marker detection circuit.
5. 5. The method for generating three-dimensional map data according to claim 4, wherein the first two-dimensional range sensor acquires three-dimensional point cloud data in response to marker detection by the marker detection circuit.
6. In claim 4, the positioning information acquisition circuit estimates the position and orientation of the measurement vehicle by inertial navigation from when any marker is detected by the marker detection circuit until a new marker is detected. A method for generating three-dimensional map data by acquiring the positioning information.
7. The method for generating 3D map data according to claim 4, wherein the marker is a magnetic marker that exerts a magnetic effect on the surroundings and can be magnetically detected.
8. In claim 1, the measurement vehicle is provided with three-dimensional point group data by measuring a distance to at least one target point on a second plane that is a horizontal plane including the axial direction of the central axis of the measurement vehicle. A second two-dimensional range sensor for acquisition is attached,
   The positioning information acquisition circuit uses the three-dimensional point group data acquired by the second two-dimensional range sensor before the measurement vehicle moves, and the second two-dimensional range sensor after the measurement vehicle moves. A method for generating three-dimensional map data, wherein the positioning information is obtained by comparing the acquired three-dimensional point cloud data with the acquired three-dimensional point cloud data.
9. In claim 1, the measurement vehicle is provided with three-dimensional point group data by measuring a distance to at least one target point on a second plane that is a horizontal plane including the axial direction of the central axis of the measurement vehicle. A second two-dimensional range sensor for acquisition is attached,
   The positioning information acquisition circuit uses the three-dimensional point group data acquired by the second two-dimensional range sensor before the measurement vehicle moves, and the second two-dimensional range sensor after the measurement vehicle moves. It is possible to obtain positioning information by comparing the acquired three-dimensional point cloud data, and
   The positioning information can be obtained by estimating the position and direction of the measurement vehicle by inertial navigation,
   The positioning information acquisition circuit is configured to correct an error in the positioning information obtained by the inertial navigation using positioning information obtained by comparing three-dimensional point cloud data before and after the movement of the measurement vehicle. .
10. In any one of claims 1 to 9, the measurement vehicle includes one or more planes on a third plane that is perpendicular or oblique to the central axis of the measurement vehicle and intersects the first plane. A third two-dimensional range sensor is installed, which acquires three-dimensional point cloud data by measuring the distance to the target point.
    Coordinate conversion processing based on the positioning information acquired by the positioning information acquisition circuit is performed on three-dimensional point cloud data obtained by the first two-dimensional range sensor and three-dimensional point cloud data obtained by the third two-dimensional range sensor. By applying this method, it is converted into three-dimensional point cloud data in which the reference position and orientation are specified.
    A method for generating three-dimensional map data, which generates three-dimensional map data by mapping three-dimensional point cloud data in which the reference position and orientation are specified in a three-dimensional space.
11. In any one of claims 1 to 9, the measurement vehicle includes one or more planes on a third plane that is perpendicular or oblique to the central axis of the measurement vehicle and intersects the first plane. A third two-dimensional range sensor is installed, which acquires three-dimensional point cloud data by measuring the distance to the target point.
    The intersection of the first plane on which the first two-dimensional range sensor measures distance and the third plane on which the third two-dimensional range sensor measures distance with respect to the moving plane detecting the inclination of the measurement vehicle around the central axis based on the inclination;
    A method for generating three-dimensional map data, which corrects a position at which the coordinate-transformed three-dimensional point group data is mapped in a three-dimensional space based on the inclination of the measurement vehicle around the central axis.
12. 2. The method according to claim 1, wherein, when a position and orientation of the measurement vehicle change by a predetermined amount after a certain target point is observed by the first two-dimensional range sensor, a predicted target point that can be observed by the first two-dimensional range sensor in a known orientation based on the certain target point is obtained,
    When the amount of change in the position and orientation of the measurement vehicle based on the positioning information acquired by the positioning information acquisition circuit reaches the predetermined amount, another target point located in a known orientation with respect to the one target point is observed by the first two-dimensional range sensor;
    determining an error in the positioning information acquired by the positioning information acquisition circuit based on a positional deviation between the predicted target point and the other target point;
    A method for generating three-dimensional map data, which corrects the positioning information acquired by the positioning information acquisition circuit according to an error in the positioning information.
13. In claim 12, the one target point, the other target point, and the prediction target point are points on the outer surface of a rod-shaped member extending in a three-dimensional space along the known direction. How to generate 3D map data.
14. In claim 1, in a three-dimensional space, a first target point and a second target point whose relative positional relationship is known are set,
    When the first target point is observed by the first two-dimensional range sensor, the second target point is determined by the first two-dimensional range sensor based on the position and direction of the measurement vehicle at the time of the observation. predicting the amount of change in the position and orientation of the measurement vehicle when the target point is observed;
    After the first target point is observed by the first two-dimensional range sensor, the measurement vehicle moves and the second target point is actually observed by the first two-dimensional range sensor. estimating the amount of change in the position and orientation of the measurement vehicle with respect to the reference position and orientation based on the positioning information acquired by the positioning information acquisition circuit;
    Identifying an error in the positioning information acquired by the positioning information acquisition circuit by comparing the predicted amount of change and the estimated amount of change,
    A method for generating three-dimensional map data in which positioning information acquired by the positioning information acquisition circuit is corrected according to an error in the positioning information.
15. A system that generates three-dimensional map data representing a three-dimensional structure of a vehicle driving environment using a measurement vehicle that can move on a moving plane that is a floor surface or the ground,
    The measurement vehicle is configured such that three-dimensional point cloud data can be obtained by measuring the distance to one or more target points on a first plane perpendicular or oblique to the central axis in the longitudinal direction of the measurement vehicle. a first two-dimensional range sensor attached to the
    a positioning information acquisition circuit that acquires positioning information that can identify the position and direction of the measurement vehicle;
    A coordinate conversion process based on the positioning information acquired by the positioning information acquisition circuit is applied to the three-dimensional point cloud data obtained by the first two-dimensional range sensor, and a three-dimensional point cloud whose reference position and orientation are specified. A coordinate conversion circuit that converts into data,
    A system for generating three-dimensional map data, comprising: a mapping circuit that generates a three-dimensional map by mapping two-dimensional point group data after coordinate conversion onto a three-dimensional space.

Images