WO2024201821A1 - Control device, control method, and program - Google Patents

Abstract

A control device for at least temporarily controlling a moving body that, while leading a person being led, autonomously moves in a region in which another pedestrian walks. The control device comprises a recognition unit that recognizes an object including the person being led and the other pedestrian, a prediction unit that predicts the future position of the recognized other pedestrian, a route generation unit that generates a route on which the moving body is to proceed in the future, and a drive control unit that controls a drive device attached to the moving body so that the moving body moves along the path. On the basis of the distance between the person being led and the other pedestrian and the relative speed, an index value representing the degree to which the other pedestrian affects the person being led is calculated, and the route is generated so that the moving body is positioned at an intermediate location between the person being led and the other pedestrian when the index value is maximized.

Description

Control device, control method, and program

The present invention relates to a control device, a control method, and a program.

In recent years, moving objects (called robots, micromobility, etc.) that move autonomously following a user for purposes such as transporting the user's luggage have been put to practical use. An invention for a driving control device related to micromobility has been disclosed (Patent Document 1). Research is also being conducted on moving objects that not only follow a user but also move autonomously leading the user.

Special Publication No. 2020-529050

“Social Force Model for Pedestrian Dynamics” D. Helbing and P. Molnar, , 20 May 1998.

　In conventional technology, it was sometimes impossible to generate an appropriate route depending on the situation the moving object was in. For example, no consideration was given to how the moving object should behave in relation to the person being led.

The present invention was made in consideration of these circumstances, and one of its objectives is to provide a control device, control method, and program that can generate a route for a moving body so that it behaves appropriately in relation to a leading subject.

The control device, control method, and program according to the present invention employ the following configuration.
(1): A control device according to one embodiment of the present invention is a control device that controls a moving body that moves autonomously in an area where other pedestrians walk while at least temporarily leading a leading subject, and includes a recognition unit that recognizes objects including the leading subject and the other pedestrians, a prediction unit that predicts the future positions of the recognized other pedestrians, a path generation unit that generates a path that the moving body should take in the future, and a drive control unit that controls a drive unit attached to the moving body so that the moving body moves along the path, wherein the path generation unit calculates an index value that represents the degree of influence that the other pedestrians have on the leading subject based on the distance and relative speed between the leading subject and the other pedestrians, and generates the path so that the moving body is located midway between the leading subject and the other pedestrians at the time when the index value is maximum.

(2): In the above aspect (1), the index value is calculated to be larger when the distance is shorter and the component of the relative speed in the direction in which the other pedestrian is moving toward the leading subject is larger.

(3): In the aspect of (1) above, when a plurality of waypoints corresponding to a plurality of other pedestrians respectively have a predetermined relationship with each other, the route generation unit generates the route so that the moving body passes through an integrated waypoint that combines the plurality of waypoints.

(4): In the aspect of (3) above, the predetermined relationship is one in which the multiple intermediate points are located within a predetermined distance from each other, and when the multiple intermediate points are passed, a sharp turn of less than a predetermined angle occurs in the path of the moving body.

(5): A control method according to another aspect of the present invention is a control method executed by a processor of a control device that controls a moving object that moves autonomously in an area where other pedestrians walk while at least temporarily leading a leading object, and includes the steps of: recognizing objects including the leading object and the other pedestrians; predicting the future positions of the recognized other pedestrians; generating a path that the moving object should follow in the future; and controlling a drive unit attached to the moving object so that the moving object moves along the path, wherein generating the path includes calculating an index value that represents the degree of influence that the other pedestrians have on the leading object based on the distance and relative speed between the leading object and the other pedestrians, and generating the path so that the moving object is located at the midpoint between the leading object and the other pedestrians at the time when the index value is maximum.

(11): A program according to another aspect of the present invention causes a processor of a control device that controls a moving object that moves autonomously in an area where other pedestrians walk while at least temporarily leading a leading object, to execute the following operations: recognizing objects including the leading object and the other pedestrians; predicting the future positions of the recognized other pedestrians; generating a path for the moving object to follow in the future; and controlling a drive unit attached to the moving object so that the moving object moves along the path, wherein generating the path includes calculating an index value representing the degree of influence that the other pedestrians have on the leading object based on the distance and relative speed between the leading object and the other pedestrians, and generating the path so that the moving object is located midway between the leading object and the other pedestrians at the point where the index value is maximum.

According to aspects (1) to (11), a route for a moving object can be generated so that the moving object behaves appropriately with respect to a leading subject.

FIG. 1 is a configuration diagram of a moving object. FIG. 2 is a configuration diagram of a control device. 13 is a flowchart showing an example of a process performed by a prediction unit. FIG. 11 is a diagram for explaining an example of a method for generating an ideal route. FIG. 13 is a diagram illustrating an example of how a waypoint is set. FIG. 13 is a diagram illustrating a situation in which a route cannot be conveniently generated. 13 is a diagram showing an example of a route in a case where an integrated waypoint that combines a plurality of waypoints is set by an integration processing unit; FIG.

[overview]
Hereinafter, with reference to the drawings, an embodiment of the control device, control method, and program of the present invention will be described. The control device of the present invention controls the drive device of the moving body to move the moving body. The moving body in the present invention is an autonomous moving body leading a leading subject in an area where pedestrians walk. The area where pedestrians walk is a sidewalk, a public open space, a floor in a building, etc., and may include a roadway. In the following description, it is assumed that a person does not ride on the moving body, but a person may ride on the moving body. The leading subject is, for example, one of the pedestrians, but it may also be a robot or an animal (hereinafter, this is referred to as user U). For example, the moving body moves a little ahead of an elderly user U while heading toward a predetermined destination point, thereby operating to prevent other pedestrians who hinder the movement of the user U from getting too close to the user U (i.e., it operates to make a way for the user U). The user U is not limited to an elderly person, but may be a person who tends to have difficulty walking, a child, a person shopping at a supermarket, a patient moving in a hospital, a pet taking a walk, etc. It should be noted that such an operation may not be performed constantly, but may be performed temporarily. For example, when a moving body is traveling alongside or chasing a user and detects a specified situation in the user's direction of travel (e.g., the presence of an obstacle or traffic congestion), the moving body may temporarily lead the user by executing the algorithm of the present invention.

[Basic configuration]
1 is a configuration diagram of a moving body 1. The moving body 1 is equipped with, for example, an HMI (Human Machine Interface) 10, an object detection device 20, a driving device 30, a sensor 40, and a control device 100. These components are supported or housed by a base 5.

The HMI 10 presents various information to the user U and accepts input operations by the user U. The HMI 10 includes various display devices, speakers, buzzers, touch panels, switches, keys, short-range wireless communication devices, etc. For example, the HMI 10 accepts the setting of a destination point.

The object detection device 20 is a device that generates data for recognizing other pedestrians and the user U present around the moving body 1. The object detection device 20 includes, for example, a camera whose shooting range is the area around the moving body 1. The object detection device 20 may also include sensors such as a radar device, a LIDAR (Light Detection and Ranging) sensor, and an ultrasonic sensor, as well as an object recognition device that identifies objects by performing sensor fusion processing based on the output of these sensors.

The driving device 30 is a mechanism for moving the moving body 1 including the base body 5 in any direction. The driving device 30 includes, for example, a plurality of wheels, a driving motor attached to one or more of the wheels, and a steering device attached to one or more of the wheels. There are no particular restrictions on the configuration of the driving device 30, and it may have any configuration. In principle, the driving device 30 moves the moving body 1 while keeping the front surface of the base body 5 facing the traveling direction of the moving body 1.

Sensor 40 is a sensor for detecting the behavior of moving body 1. Sensor 40 includes, for example, a wheel speed sensor for detecting the wheel speed, an acceleration sensor for detecting the acceleration acting on moving body 1, a yaw rate sensor attached near the center of gravity in the horizontal direction of base body 5, a steering angle sensor for detecting the steering angle of the steered wheels (steered wheels), and an orientation sensor for detecting the horizontal orientation of moving body 1.

2 is a configuration diagram of the control device 100. The control device 100 includes, for example, a recognition unit 110, a prediction unit 120, a route generation unit 130, and a drive control unit 140. The route generation unit 130 includes, for example, an index value calculation unit 132, an intermediate point setting unit 134, an integration processing unit 136, and a future position determination unit 138. These components are realized, for example, by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by collaboration between software and hardware. The program may be stored in advance in a storage device (a storage device with a non-transient storage medium) such as a hard disk drive (HDD) or flash memory, or may be stored in a removable storage medium (non-transient storage medium) such as a DVD or CD-ROM, and installed in the storage device by inserting the storage medium into a drive device.

The recognition unit 110 recognizes objects including other pedestrians (hereinafter simply referred to as pedestrian P) and user U based on the data output by the object detection device 20. When the object detection device 20 is a camera, the recognition unit 110 recognizes pedestrians by inputting camera images into a trained model for identifying pedestrians. The same applies to objects other than pedestrians. In order to distinguish between pedestrians P and user U, the recognition unit 110 may store multiple images of user U captured in advance by a camera as templates in a storage unit (not shown), and identify user U by comparing the templates with the camera images. In addition, the recognition unit 110 may recognize the position of user U by using communication directivity to perform short-range wireless communication between the terminal device held by user U and the HMI 10.

The prediction unit 120 predicts the future position of the pedestrian P. Furthermore, the prediction unit 120 may also predict the future position of the user U based on the previous route Rm(z) generated one cycle ago while the route generation unit 130 is performing the repetitive processing.

Figure 3 is a flowchart showing an example of the processing contents of the prediction unit 120. First, the prediction unit 120 inputs the previous route Rm(z) (S1) and generates an ideal route Ru#(i) for the user U to follow the moving body 1 (S2). The ideal route Ru#(i) is the route along which the user U is predicted to travel, assuming that no pedestrian P is present in the vicinity of the moving body 1 and the user U. Figure 4 is a diagram for explaining an example of a method for generating the ideal route Ru#(i). The route Rm is expressed as a series of multiple route points Om for each step. A step is a future point in time that arrives at a predetermined time interval. For example, the prediction unit 120 generates the ideal route Ru# of the user U so that the movement vector Vu-1 of the user U moving from step 0 (calculation time) to step 1 faces the position Om-1 of the moving body 1 at step 1 of the previous trajectory Rm(z), the movement vector Vu-2 of the user U moving from step 1 to step 2 faces the position Om-2 of the moving body 1 at step 2 of the previous trajectory Rm(z), and the movement vector Vu-3 of the user U moving from step 2 to step 3 faces the position Om-3 of the moving body 1 at step 3 of the previous trajectory Rm(z), that is, so as to follow the moving body 1 with a delay of one cycle. This generation method is merely an example, and other generation methods may be adopted. For example, the prediction unit 120 may generate the ideal route Ru# of the user U using a method similar to that for predicting the future position of the pedestrian P described later.

Returning to FIG. 3, the prediction unit 120 predicts the future position of pedestrian P for each pedestrian P (k = 1 to m) and outputs a predicted route Rp(k) (S3). The prediction unit 120, for example, fits the past movement trajectory of pedestrian P to a line or curve represented by a function, and predicts the future position of pedestrian P by assuming that pedestrian P will continue to move along that line or curve in the future. The prediction unit 120 predicts the future position of pedestrian P up to q steps ahead, assuming, for example, that pedestrian P moves at a constant speed without accelerating or decelerating (if there is a tendency for pedestrian P to accelerate or decelerate by the time of prediction, that tendency may be continued). The prediction unit 120 outputs the ideal route Ru# of user U and the expected route Rp(k) of pedestrian P to the route generation unit 130.

The index value calculation unit 132 of the path generation unit 130 calculates an index value F1 representing the degree of influence that the pedestrian P has on the user U in each future step for each pedestrian P. The index value F1 is calculated to be a larger value as the distance between the pedestrian P and the user U is shorter and the relative speed between the pedestrian P and the user U is higher. The index value F1 is expressed by, for example, Equation (1). In the equation, d _up is the distance between the pedestrian P and the user U. ω _up is calculated by Equation (2). In the equation, v _up is the relative speed between the pedestrian P and the user. Note that λ and η are weighting coefficients. u _up is a vector obtained by normalizing ω _up as expressed by Equation (3). _{e ui} is a normalized orientation vector (the direction of the pedestrian P as seen from the user U) expressed by Equation (4).

F1=-{exp ^{-η・dup・||ωup||} }・u _up …(1)
ω _up =λ・v _up +e _up …(2)
u _up = ω _up / | | ω _up | | …(3)
e _up = (Pp-Pu)/||Pp-Pu||...(4)

The midpoint setting unit 134 extracts the step where the index value F1 calculated for each future step for each pedestrian P is maximum for each pedestrian P. It then obtains the position on the predicted route Rp(k) corresponding to the step where the index value F1 is maximum, and sets the midpoint between the pedestrian P and the user U at that step.

Figure 5 is a diagram illustrating how waypoints are set. In the diagram, there are four pedestrians P, k = 1 to 4. As for pedestrian P(1), since he is moving away from user U, his index value F1 is cut off by the threshold value and he is not taken into consideration for route Rm. The index value F1 is maximum for pedestrian P(2) in step 7, for pedestrian P(3) in step 9, and for pedestrian P(4) in step 13. In the diagram, P(2,7), P(3,9), and P(4,13) are positions () corresponding to steps (7,9,13) where the index values F1 of pedestrians P(2), P(3), and P(4) are maximum, respectively. U(7), U(9), and U(13) are positions predicted to be passed by user U in steps 7, 9, and 13, respectively. The midpoint setting unit 134 sets the midpoint MID (7) between P (2, 7) and U (7), the midpoint MID (9) between P (3, 9) and U (9), and the midpoint MID (13) between P (4, 13) and U (13) as the midpoints corresponding to steps 7, 9, and 13, respectively.

The processing details of the integrated processing unit 136 will be described later.

The future position determination unit 138 generates the route Rm for the moving body 1 so that the set waypoints are passed in step order. Furthermore, the future position determination unit 138 generates the route Rm for the moving body 1 by adding a bias toward the destination point using some method. The future position determination unit 138 determines the route Rm for the moving body 1 to be a smooth curve that can pass the waypoints in step order and generally leads to the destination point. For example, the future position determination unit 138 generates the route Rm for the moving body 1 by appropriately combining spline curves, circular arcs, straight lines, etc.

As a result, the moving body 1 is controlled so that its presence does not cause other pedestrians P to get too close to the user U. For example, the index value F1 should be at its maximum when the pedestrian P is closest to the user U, and so by moving to the intermediate point in that situation, the moving body 1 can restrain the pedestrian P from getting too close to the user U.

The example in Figure 5 shows how the waypoints are conveniently arranged in step order, but in reality, if the waypoints are set according to the principle and route Rm is generated, the route Rm may end up with the moving body 1 going back and forth. Figure 6 shows an example of a situation in which route Rm cannot be conveniently generated. As shown, MID (9) is closer to the moving body 1 than MID (7), which creates a situation in which route Rm cannot be generated smoothly. As a result, the behavior of the moving body 1 becomes unstable, and the user U may confuse the pedestrian P.

To deal with this, if multiple set waypoints have a predetermined relationship with each other, the integration processing unit 136 sets an integrated waypoint that combines the multiple waypoints.The future position determination unit 138 then generates a route Rm so that the mobile unit 1 passes through the integrated waypoint, without taking into account the individual waypoints combined into the integrated waypoint.

The specified relationship means that there are multiple intermediate points within a specified distance, and when the multiple intermediate points are passed, a sharp turn of less than a specified angle (e.g., about 90 degrees) occurs on the route Rm of the moving body 1. A(7) in FIG. 6 indicates the range of the specified distance as seen from MID(7). In this example, MID(9) is within A(7), and a sharp turn occurs on the route Rm of the moving body 1 when MID(7) and MID(9) are passed, so MID(7) and MID(9) are determined to have a specified relationship.

FIG. 7 is a diagram showing an example of route Rm when an integrated waypoint that combines multiple waypoints is set by the integration processing unit 136. In the figure, MID*(7,9) is the integrated waypoint that combines MID(7) and MID(9). The integration processing unit 136 sets the integrated waypoint by, for example, calculating a weighted sum for both position and time using the ratio of the magnitude of F1 (equations (2) and (3)). In the equations, F1(7) is the magnitude of index value F1 of pedestrian P(3) in step 7, and F1(9) is the magnitude of index value F1 of pedestrian P(2) in step 7. In addition, t* indicates the timing (step) at which the integrated waypoint should be passed.

MID*(7,9)={MID(9)×F1(2)+MID(7)×F1(3)}/{F1(7)+F1(9)}…(2)
t*={9×F1(2)+7×F1(3)}/{F1(7)+F1(9)}…(3)

By performing processing in this manner, the behavior of the moving body 1 can be stabilized.

According to the embodiment described above, a route for the mobile body 1 can be generated so that the mobile body 1 behaves appropriately with respect to the lead subject (user U).

In the above description, the control device 100 is described as being mounted on the moving body 1, but this is not limited thereto. The control device 100 may be installed at a location away from the moving body 1, acquire output data from the object detection device 20 via communication, and transmit a drive instruction signal to the drive device 30, i.e., remotely control the moving body 1.

The above-described embodiment can be expressed as follows.
A control device for controlling a moving body that autonomously moves in an area where other pedestrians are walking while leading a leading subject,
one or more storage media storing computer-readable instructions;
a processor coupled to the one or more storage media;
The processor executes the computer-readable instructions to:
Recognizing objects including the leading subject and the other pedestrians;
predicting a future location of the recognized other pedestrians; and
generating a route to be taken by the moving body in the future;
controlling a drive unit attached to the moving body to move the moving body along the path;
generating the route includes calculating an index value representing a degree of influence that the other pedestrian has on the target leading person based on a distance and a relative speed between the target leading person and the other pedestrian, and generating the route such that the moving object is located at a midpoint between the target leading person and the other pedestrian at a time when the index value is maximum.
Control device.

　Although the above describes the form for carrying out the present invention using an embodiment, the present invention is in no way limited to such an embodiment, and various modifications and substitutions can be made without departing from the spirit of the present invention.

REFERENCE SIGNS LIST 1 Mobile object 20 Object detection device 30 Driving device 100 Control device 110 Recognition unit 120 Prediction unit 130 Path generation unit 132 Index value calculation unit 134 Waypoint setting unit 136 Integrated processing unit 138 Future position determination unit 140 Driving control unit

Claims (6)

A control device for controlling a moving body that autonomously moves in an area where other pedestrians are walking while at least temporarily leading a leading subject,
A recognition unit that recognizes objects including the leading person and the other pedestrians;
a prediction unit for predicting a future position of the recognized other pedestrian;
A route generation unit that generates a route that the moving body should take in the future;
a drive control unit that controls a drive device attached to the moving body so that the moving body moves along the path;
Equipped with
the route generation unit calculates an index value representing a degree of influence that the other pedestrian has on the target leading person based on a distance and a relative speed between the target leading person and the other pedestrian, and generates the route such that the moving body is located at a midpoint between the target leading person and the other pedestrian at a time when the index value is maximum.
Control device. The index value is calculated to be a larger value as the distance is shorter and the component of the relative speed in the direction in which the other pedestrian moves toward the leading target pedestrian is larger.
The control device according to claim 1. the route generation unit generates the route such that the moving object passes through an integrated route that combines a plurality of waypoints corresponding to the plurality of other pedestrians when the plurality of waypoints have a predetermined relationship with each other.
The control device according to claim 1. the predetermined relationship is a relationship in which the plurality of intermediate points are located within a predetermined distance from each other, and a sharp turning portion of less than a predetermined angle occurs on the path of the moving object when the moving object passes through the plurality of intermediate points;
The control device according to claim 3. A control method executed by a processor of a control device that controls a moving object that moves autonomously in an area where other pedestrians are walking while at least temporarily leading a leading subject, comprising:
Recognizing objects including the leading subject and the other pedestrians;
predicting a future location of the recognized other pedestrians; and
generating a route to be taken by the moving body in the future;
and controlling a drive device attached to the moving body so as to move the moving body along the path;
generating the route includes calculating an index value representing a degree of influence that the other pedestrian has on the target leading person based on a distance and a relative speed between the target leading person and the other pedestrian, and generating the route such that the moving object is located at a midpoint between the target leading person and the other pedestrian at a time when the index value is maximum.
Control methods. A processor of a control device that controls a moving object that moves autonomously in an area where other pedestrians are walking while at least temporarily leading a leading target person,
Recognizing objects including the leading subject and the other pedestrians;
predicting a future location of the recognized other pedestrians; and
generating a route to be taken by the moving body in the future;
and controlling a drive device attached to the moving body so as to move the moving body along the path,
generating the route includes calculating an index value representing a degree of influence that the other pedestrian has on the target leading person based on a distance and a relative speed between the target leading person and the other pedestrian, and generating the route such that the moving object is located at a midpoint between the target leading person and the other pedestrian at a time when the index value is maximum.
program.

Images