JP6354496B2 - Robot control method - Google Patents

Description

  The present invention relates to a robot control method for remotely operating a robot.

  A robot remote operation system is known in which a robot executes a task by autonomous control, and when the execution of the task fails, the central controller selects an appropriate operation terminal and requests remote operation from the operation terminal. (For example, refer to Patent Document 1).

JP 2007-190659 A

  In the robot remote operation system, when the task execution of the robot fails, a remote manual operation is requested to an appropriate operation terminal. At this time, since the operation mode for the user to manually operate the operation of the robot is not clear, there is a problem that it takes time and effort to select the optimum operation mode.

  The present invention has been made to solve such a problem, and since an optimal operation mode is automatically selected, the user can save time and effort to start the robot operation smoothly. The main purpose is to provide

  One aspect of the present invention for achieving the above object includes a step of autonomously controlling a robot to execute a task, and when the robot fails to execute the task, the task is performed according to a user instruction from the autonomous control. Switching the control of the robot to manual control for performing the operation, wherein the user manually operates the operation of the robot during manual control of the robot, and an operation mode of the robot The robot control method includes a step of selecting an operation mode corresponding to an operation when the operation of the robot fails or an operation nearest to the operation, the operation being preset in association with each other.

  According to the present invention, since the optimum operation mode is automatically selected, it is possible to provide a robot control method in which the user can smoothly start the operation of the robot without having to make the selection.

It is a figure showing a schematic structure of a robot control system concerning one embodiment of the present invention. 1 is a block diagram showing a schematic system configuration of a robot according to an embodiment of the present invention. It is a figure which shows an example of the operation screen of an operation terminal. It is a figure which shows an example of the operation screen in hand XY movement mode. It is a figure which shows an example of the operation screen in hand XY rotation mode. It is a block diagram showing a schematic system configuration of a control device concerning one embodiment of the present invention. It is a flowchart which shows the control processing flow of the robot control method which concerns on one Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing a schematic configuration of a robot control system according to an embodiment of the present invention. A robot control system 10 according to the present embodiment includes a robot 1 and an operation terminal 7 for remotely operating the robot 1. The robot 1 is configured, for example, as an autonomous work robot that autonomously executes tasks (work).

  FIG. 2 is a block diagram showing a schematic system configuration of the robot according to the embodiment of the present invention. The robot 1 according to the present embodiment includes a robot body 2, a moving device 3 that moves the robot body 2, a gripping operation device 4 that grips and moves an object, and a control that controls the moving device 3 and the gripping operation device 4. A device 5 and a distance sensor 6 are provided.

The moving device 3 rotates a plurality of wheels by driving a motor or the like, for example, according to a control signal from the control device 5, and moves the robot body 2 to a desired position.
The gripping operation device 4 includes, for example, a gripping part (hand) 41 that grips an object, a plurality of links 43 connected via joint parts 42 such as a wrist joint, an elbow joint, and a shoulder joint, and each joint part 42. It is configured as an articulated arm comprising an actuator such as a motor to be driven.

  The distance sensor 6 detects distance information with respect to obstacles and operation objects existing around the robot 1. The distance sensor 6 transmits the detected distance information to the control device 5. The distance sensor 6 includes, for example, a camera, an ultrasonic sensor, a millimeter wave sensor, and the like. The configuration of the robot 1 is an example and is not limited to this.

  The operation terminal 7 displays various operation screens for operating the robot 1. The operation terminal 7 displays distance information transmitted from the distance sensor 6 and information indicating the state of the robot 1 (an operation mode and sensor information described later). The operation terminal 7 is provided with operation input units such as a joystick (software or hardware), a touch panel, a mouse, and a keyboard. The user can input operation information for operating the robot 1 to the operation terminal 7 via the operation input unit. The operation terminal 7 is, for example, a portable terminal such as a tablet terminal, a PC (Personal Computer), or a smartphone. The operation terminal 7 and the control device 5 are connected by wireless or wired communication, and can exchange data with each other.

  The control device 5 causes the robot 1 to execute a task by controlling the moving device 3 and the gripping operation device 4 based on work information input via the operation terminal 7 and distance information output from the distance sensor 6. .

The control device 5 switches between autonomous control that causes the robot 1 to execute tasks autonomously and manual control that causes the robot 1 to execute tasks in response to user instructions. In the autonomous control, the control device 5 autonomously executes the set task by controlling the moving device 3 and the gripping operation device 4 based on the distance information output from the distance sensor 6, for example.
On the other hand, the control device 5 controls the moving device 3 and the gripping operation device 4 in accordance with information transmitted from the operation terminal 7 in manual control.

  For example, the user inputs operation information to the operation terminal 7 via the operation screen of the operation terminal 7 as shown in FIG. Further, the user selects various operation modes for manually operating the operation of the robot 1 via buttons on the operation screen. The operation terminal 7 displays an operation screen corresponding to the selected operation mode. The user inputs operation information indicating the operation of the robot 1 by operating the operation input unit while viewing the display screen of the operation terminal 7. The operation terminal 7 transmits the selected operation mode and operation information to the control device 5. The control device 5 controls the moving device 3 and the grip operation device 4 based on the operation mode and operation information transmitted from the operation terminal 7.

  For example, as shown in FIG. 4, when the user selects an operation mode called the hand XY movement mode on the operation screen of the operation terminal 7, the operation terminal 7 displays an operation screen corresponding to the selected hand XY movement mode. While viewing the display screen of the operation terminal 7, the user operates the operation input unit and inputs operation information indicating movement of the hand in the XY directions. The operation terminal 7 transmits the selected operation mode and operation information of the operation input unit to the control device 5. The control device 5 controls the grip operation device 4 according to the operation mode and operation information transmitted from the operation terminal 7 to move the hand of the robot 1 in the XY directions.

  Similarly, as shown in FIG. 5, when the user selects an operation mode called the hand XY rotation mode on the operation screen of the operation terminal 7, the operation terminal 7 displays an operation screen corresponding to the selected hand XY rotation mode. . While viewing the display screen of the operation terminal 7, the user operates the operation input unit and inputs operation information indicating the rotation of the hand in the XY directions. The operation terminal 7 transmits the selected operation mode and operation information of the operation input unit to the control device 5. The control device 5 controls the grip operation device 4 according to the operation mode and operation information transmitted from the operation terminal 7 to rotate the hand of the robot 1 in the XY directions. The above-described operation mode is an example, and the present invention is not limited to this. Various operation modes for manually operating the operation of the robot 1 are set in the operation terminal 7 and the robot 1 in advance.

  By the way, when operating a robot, there are various operation modes for performing the operation. Further, skill is required for the user to select the optimum operation mode by judging the situation of the robot. Furthermore, when an autonomous control robot fails to execute a task (task execution error), it is necessary to switch the robot control from autonomous control to manual control, and to recover the error state by manual operation. At this time, a general user does not know what operation mode exists and what operation mode is optimal at present. For this reason, it is a great effort to select an optimal operation mode for the user.

  In contrast, the robot control system 10 according to the present embodiment selects an operation mode corresponding to an operation when the robot 1 fails to execute a task. As a result, since the optimum operation mode is automatically selected, the user can smoothly start the operation of the robot 1 without having to make the selection.

  FIG. 6 is a block diagram illustrating a schematic system configuration of the control device according to the present embodiment. The control device 5 according to the present embodiment includes a task execution unit 51, an error determination unit 52, a mode setting unit 53, a mode change transmission unit 54, and a command control unit 55.

  The control device 5 includes, for example, a CPU (Central Processing Unit) 5a that performs control processing, arithmetic processing, and the like, a ROM (Read Only Memory) 5b that stores a control program executed by the CPU 5a, an arithmetic program, processing data, and the like. The hardware configuration is centered on a microcomputer including a RAM (Random Access Memory) 5c and the like.

  The task execution unit 51 controls the moving device 3 and the gripping operation device 4 so that the robot 1 executes the set task. The task execution unit 51 defines a task in a domain description language such as PDDL (Planning Domain Description Language), and inputs the defined task to the planner. Then, the task execution unit 51, for example, performs the task “open refrigerator” “1. Take the handle of the refrigerator door”, “2. Hold the refrigerator door handle”, “3. Divide into fine-grained actions such as “open”. The task execution unit 51 controls the moving device 3 and the gripping operation device 4 so as to execute each divided operation (hereinafter, divided operation). The task execution unit 51 outputs the division operation being executed to the mode setting unit 53.

  The error determination unit 52 determines whether or not the robot 1 has failed to execute the task. The error determination unit 52 determines that the task execution of the robot 1 has failed when the task completion condition of the robot 1 is not satisfied or when the task cannot be continued autonomously.

  For example, the error determination unit 52 may determine that the task execution has failed when the goal state is set in the task defined in the domain description language and the goal state cannot be satisfied.

  Here, the case where the state where the refrigerator door is opened by 60 degrees or more is set as the goal state. In this case, when the error determination unit 52 recognizes that the door opening angle is 30 degrees, the error determination unit 52 determines that the task execution failure of the robot 1 has failed because the goal state is not satisfied. Assume that the state where the refrigerator drawer is set to be opened by 0.5 m or more is set as the goal state. In this case, when the error determination unit 52 recognizes that the opening of the drawer is 0.2 m, the error determination unit 52 determines that the task execution of the robot 1 has failed because the goal state is not satisfied.

  In addition, the error determination unit 52 may determine that the task execution of the robot 1 has failed when detecting a failure in motion planning (motion planning) of the robot 1. The error determination unit 52 determines that the task execution of the robot 1 has failed when, for example, in RRT (Rapidly-exploring Random Trees), a solution (such as a joint trajectory necessary for operation) cannot be obtained within a realistic time. To do. More specifically, the error determination unit 52 may determine that the task execution of the robot 1 has failed when the motion planning for opening the door or pulling out the door is not possible.

  Further, the error determination unit 52 sets an allowable external force in each operation of the robot 1, and when the external force exceeding the set value of the external force is detected by the sensor of the robot 1, the task execution failure of the robot 1 is determined. You may judge.

  The failure determination method by the error determination unit 52 is an example, and is not limited to this. For example, the error determination unit 52 may determine the task execution failure of the robot 1 by arbitrarily combining the above-described determination methods. When the error determination unit 52 determines that the task execution of the robot 1 has failed, the error determination unit 52 outputs the determination result to the mode setting unit 53.

  The mode setting unit 53 selects and sets an operation mode corresponding to the operation when the robot 1 fails to execute the task. For example, the mode setting unit 53 sets an operation mode at the time of task execution failure based on the determination result from the error determination unit 52, the task division operation from the task execution unit 51, and the setting file information.

  The setting file information is, for example, information in which a plurality of task division operations are associated with a plurality of optimum operation modes set when each task division operation fails. The setting file information is set in advance in the ROM 5b, the RAM 5c, and the like. For example, the task “open refrigerator” is divided into division operations “1. movement”, “2. revolving door opening”, and “3. drawer opening”. In this case, the setting file information includes an operation mode “mobile operation instruction mode” for the division operation “1. movement”, an operation mode “hinge constraint rotation instruction mode” for the division operation “2. The operation mode “hand XY movement mode” is associated with “.

  When the mode setting unit 53 determines that the task execution has failed based on the determination result from the error determination unit 52, the mode setting unit 53 reads and sets the optimum operation mode corresponding to the task division operation at that time from the setting file information. In this way, the operation mode corresponding to the operation when the robot 1 fails to execute the task is automatically selected.

  The mode setting unit 53 transmits the set operation mode to the operation terminal 7. The operation terminal 7 displays an operation screen corresponding to the operation mode transmitted from the mode setting unit 53. For example, when the operation mode transmitted is the “movement control instruction mode”, the operation terminal 7 moves forward when the joystick is tilted forward, and the movement device 3 turns when the joystick is tilted laterally. Display the operation screen. When the transmitted operation mode is the “hand XY movement mode”, the hand terminal moves in the X direction when the joystick is tilted in the X direction, and the hand moves in the Y direction when the joystick is tilted in the Y direction. Display the operation screen.

Note that the mode setting unit 53 may set the operation mode at the time of task execution failure based on the most recent operation profile executed at the time of task execution failure.
For example, when the task of the robot 1 is executed, the mode setting unit 53 moves the motion profile closest to the currently executed motion (“movement of the moving device 3 in the front-rear direction”, “turning of the moving device 3”, “front-rear direction of the hand”). ”,“ Movement of the hand in the horizontal direction ”, etc.) are stored. The mode setting unit 53 extracts a divided motion that approximates the motion profile closest to the motion that was being executed when the task execution of the robot 1 failed. Then, the mode setting unit 53 selects and sets an operation mode for the extracted divided operation based on the setting file information.

  More specifically, when the motion profile closest to the motion that was being executed when the task execution of the robot 1 failed is “movement of the moving device 3 in the front-rear direction”, the mode setting unit 53 divides the motion profile by approximation. The operation “1. movement” is extracted. The mode setting unit 53 selects and sets the operation mode “movement control instruction mode” for the extracted division operation “1. movement” based on the setting file information. In addition, when the operation profile closest to the operation that was being executed when the task execution failed is “movement in the front-rear direction of the hand”, the mode setting unit 53 extracts the divided operation “3. drawer opening” that approximates the operation profile. To do. The mode setting unit 53 selects and sets the operation mode “hand XY movement mode” for the extracted division operation “1. movement” based on the setting file information. In this way, the operation mode corresponding to the most recent operation when the robot 1 fails to execute the task is automatically selected.

The mode setting unit 53 outputs the set operation mode to the mode change transmission unit 54.
When the mode change transmission unit 54 receives the operation mode from the mode setting unit 53, the mode change transmission unit 54 transmits ID information corresponding to the operation mode and error information indicating task execution failure to the operation terminal 7. The mode change transmission unit 54 transmits the ID information and error information to the operation terminal 7 via, for example, WiFi (registered trademark), wired or wireless LAN.

  The command control unit 55 is based on the operation mode transmitted from the operation terminal 7 or set by the mode setting unit 53 and the operation information transmitted from the operation terminal 7. To control. For example, when the robot 1 fails to execute the task, the command control unit 55 uses the operation mode set by the mode setting unit 53 and the operation information transmitted from the operation terminal 7 to The grip operation device 4 is controlled. Thereby, the task which the robot 1 failed can be recovered.

  More specifically, the command control unit 55 includes the operation mode “hand XY movement mode” set by the mode setting unit 53 and the operation information “joystick tilted forward (X direction) transmitted from the operation terminal 7. Based on the information, the joint angles of the gripping operation device 4 when the hand has advanced α [m] forward in the X direction are calculated using inverse kinematics (IK). Then, the command control unit 55 outputs a control signal corresponding to each calculated joint angle to the gripping operation device 4.

  The operation terminal 7 includes a mode change unit 71 that receives ID information and error information transmitted from the mode change transmission unit 54. When the mode change unit 71 receives the ID information from the mode change transmission unit 54, the mode change unit 71 displays an operation screen corresponding to the operation mode of the ID information. For example, the mode changing unit 71 displays an operation screen including an image display such as a tablet and a software joystick. The user can easily input the operation information by tapping the image display on the operation screen or tapping the software joystick. The operation terminal 7 may be composed of a mechanical joystick and an LED that lights the selected operation mode.

FIG. 7 is a flowchart showing a control processing flow of the robot control method according to the present embodiment.
The task execution unit 51 of the control device 5 autonomously controls the moving device 3 and the gripping operation device 4 so as to execute each division operation of the task (step S101). The task execution unit 51 outputs the division operation being executed to the mode setting unit 53.

  The error determination unit 52 determines whether or not the robot 1 has failed in task execution (step S102). If the error determination unit 52 determines that the robot 1 has failed to execute the task (YES in step S102), the error determination unit 52 outputs the determination result to the mode setting unit 53 (step S103).

  The mode setting unit 53 sets the operation mode at the time of task execution failure based on the determination result from the error determination unit 52, the task division operation from the task execution unit 51, and the setting file information (step S104). ). The mode setting unit 53 outputs the set operation mode to the mode change transmission unit 54.

  When receiving the operation mode from the mode setting unit 53, the mode change transmission unit 54 transmits ID information and error information corresponding to the operation mode to the operation terminal 7 (step S105).

When the mode change unit 71 of the operation terminal 7 receives the ID information from the mode change transmission unit 54, the mode change unit 71 displays an operation screen corresponding to the operation mode of the ID information (step S106).
The user inputs operation information on the operation screen of the operation terminal 7 (step S107). At this time, when task execution fails, an optimal operation mode is automatically set, and an operation screen corresponding to the operation mode is displayed. The user can easily recover a task that failed to be executed according to the operation screen. In particular, a user with little experience does not need to select an operation mode again, so that labor can be greatly reduced.

The operation terminal 7 transmits the input operation information to the command control unit 55 of the control device 5 (step S108).
The command control unit 55 controls the moving device 3 and the gripping operation device 4 based on the operation mode set by the mode setting unit 53 and the operation information transmitted from the operation terminal 7 (step S109).

  As described above, in the robot control system 10 according to the present embodiment, the operation mode corresponding to the operation when the robot 1 fails to execute the task or the operation closest to the operation is selected. As a result, since the optimum operation mode is automatically selected, the user can smoothly start the operation of the robot 1 without having to make the selection.

  Note that the present invention is not limited to the above-described embodiment, and can be changed as appropriate without departing from the spirit of the present invention.

  In the present invention, for example, the processing shown in FIG. 7 can be realized by causing the CPU 5a to execute a computer program.

  The program may be stored using various types of non-transitory computer readable media and supplied to a computer. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (for example, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (for example, magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / W and semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (random access memory)) are included.

  The program may be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

  1 robot, 2 robot body, 3 moving device, 4 gripping operation device, 5 control device, 6 distance sensor, 7 operation terminal, 10 robot control system, 51 task execution unit, 52 error determination unit, 53 mode setting unit, 54 mode Change transmission unit, 55 command control unit

Claims (1)

A step of autonomously controlling the robot to execute a task;
When the robot fails to execute the task, switching the control of the robot from the autonomous control to manual control for executing the task in response to a user instruction;
A robot control method including:
The operation mode for the user to manually operate the operation of the robot during manual control of the robot and the operation of the robot are preset in association with each other,
Determining whether the robot has failed in task execution;
When the robot determines that the task execution has failed, the task operation and the task operation based on setting file information in which a plurality of optimum operation modes to be set when the task operation fails are associated, Setting an operation mode associated with the latest operation of the operation or an operation approximate to the latest operation to an operation mode at the time of task execution failure;
Displaying an operation screen corresponding to the set operation mode at the time of task execution failure on the operation terminal;
On the operation screen of the operation terminal, inputting operation information indicating the operation of the robot;
Controlling the robot based on the set operation mode at the time of task execution failure and the input operation information .
A robot control method characterized by the above.

Images