WO2018221053A1

WO2018221053A1 - Remote operation robot control system and remote operation robot control method

Info

Publication number: WO2018221053A1
Application number: PCT/JP2018/016084
Authority: WO
Inventors: 隆浩長井; 亮介小林
Original assignee: 日立Ｇｅニュークリア・エナジー株式会社
Priority date: 2017-06-02
Filing date: 2018-04-19
Publication date: 2018-12-06
Also published as: JP2018202541A; JP6807280B2

Abstract

The present invention makes it possible to control multiple remote operation robots with a small number of operators. A system is provided with: work robots 7; an operation control device 4 for generating an operation signal for a work robot 7 from an operation signal of an operator 1; joint angle sensors 8 installed on each driving part of the work robots 7; an autonomous monitoring robot 10 that has an optical camera 11 and monitors remote work by the work robots 7; a line-of-sight measuring device 2 for detecting the behavior of the operator 1; a data-processing device 5 for generating a movement signal necessary for autonomous monitoring movement of the autonomous monitoring robot 10 on the basis of sensor information from the line-of-sight measuring device 2 and the joint angle sensors 8; and a monitoring video display unit 45 for displaying the monitoring video taken by the optical camera 11 of the autonomous monitoring robot 10.

Description

Remote work robot control system and remote work robot control method

The present invention relates to a remote work robot control system and a remote work robot control method.

As a technique related to the control of an autonomous monitoring robot, for example, in Patent Document 1, a number of operation variables that can be operated by an operator are selected from a combination of control modes, and other operations are performed according to changes in the operated operation variables. A technique for indirectly controlling the manipulated variable is disclosed.

Patent Document 2 discloses a technique in which a computer intentionally understands a situation where a human wearing a wearable system is placed, and a 3D robot system performs based on the understanding of the intention.

Further, Patent Document 3 discloses a technique for providing an operation detection unit in an autonomous mobile robot to realize movement and posture change intended by the operator.

JP 2004-276123 A JP 2002-361576 A Japanese Patent Laid-Open No. 2002-036362

In order to improve the conditions of harsh environments such as disaster sites where it is difficult for humans to enter, remote operators use remote work robots to grasp, cut, and drill unknown work objects. It is necessary to perform such work.

At that time, the remote operator performs the remote operation while watching the monitoring video. In order to obtain this monitoring image, conventionally, in order to provide a monitoring image, a monitoring image robot for providing a monitoring image operated by another operator is used separately from the remote work robot.

In such a conventional method, a monitoring operator is required separately, and it takes time for communication between operators. Therefore, there is a demand for a robot that autonomously provides support such as monitoring.

Here, in Patent Documents 1 to 3 as described above, the operation support based on the intention estimation in one robot and the work that makes it easy to define the intention such as movement are targeted. However, there is a problem that it is difficult to estimate the intention of the remote operator when monitoring the contact operation of an unknown object because there are many uncertain factors.

The present invention provides a remote work robot control system and a remote work robot control method capable of controlling a plurality of remote work robots with a small number of operators.

The present invention includes a plurality of means for solving the above-described problems. For example, a work robot disposed in a remote place different from the place where the operator is located, and an operation of the work robot by the operator are provided. An operation device for instructing, an operation control device for generating an operation signal for operating the work robot in response to an input of the operation signal of the operator by the operation device, and work for converting the operation signal into a command signal for the work robot A robot control device; a work robot sensor attached to each drive unit of the work robot; a monitoring camera that images the work robot; an autonomous monitoring robot that monitors remote work by the work robot; An operator sensor for detecting behavior, and the operator sensor and the work robot sensor. A data processing device that generates a control operation signal for an autonomous monitoring operation by the autonomous monitoring robot based on sensor information from at least one of the sensors, and the autonomous monitoring of the control operation signal generated by the data processing device An autonomous monitoring robot control device for converting into a robot command signal and a display device for displaying a monitoring image captured by the monitoring camera of the autonomous monitoring robot are provided.

According to the present invention, a plurality of remote work robots can be controlled by a small number of operators. Problems, configurations, and effects other than those described above will be clarified by the following description of embodiments.

It is a figure which shows the equipment configuration of a working environment and the remote work robot control system which concerns on this embodiment. It is a functional block diagram which shows the whole structure of the remote work robot control system which concerns on this embodiment. FIG. 3 is a PAD (Problem Analysis Diagram) diagram schematically showing processing contents of operation control processing of the work robot and the autonomous monitoring robot in the remote work robot control system of the present embodiment. It is a PAD figure which shows the conversion processing flow from the work robot joint angle sensor signal to the autonomous monitoring operation | movement in the remote work robot control system of this embodiment. It is a figure which shows the work phase determination example by a sensor signal in the remote work robot control system of this embodiment. It is a figure which shows the work phase determination example by a sensor signal in the remote work robot control system of this embodiment. It is a figure which shows the example of relationship of the geometric arrangement used as the target in the remote operation robot control system of this embodiment. It is a figure which shows the example of determination of the target geometric arrangement in the remote operation robot control system of this embodiment. It is a figure which shows the example of determination of the target geometric arrangement in the remote operation robot control system of this embodiment.

Embodiments of a remote work robot control system and a remote work robot control method according to the present invention will be described with reference to FIGS. 1 to 6C.

In the present embodiment, a remote for controlling a remote work robot (work robot 7 and autonomous monitoring robot 10) while illustrating a situation in which an object is unknown and a work device performs a carrying work in a high radiation environment. The contents of the work robot control system 1000 will be described.

First, an outline of the remote work robot control system 1000 will be described with reference to FIGS. 1 and 2. FIG. 1 is a diagram showing a device configuration in a work environment according to the present embodiment. FIG. 2 is a functional block diagram showing the overall configuration of the remote robot control system for the work robot.

In FIG. 1 and FIG. 2, a work robot 7 exists in a work environment (remote place) 13 different from the place where the operator 1 is, and works on the object 12 by an arm mechanism (grip arm) 71 and a moving mechanism. 72. In this embodiment, the work robot 7 has a crawler type as the moving mechanism 72 and a manipulator type as the arm mechanism 71 as the work mechanism, but is not limited thereto.

The work robot 7 is connected to the work robot control device 6 by wire and / or wireless, and the work robot control device 6 is connected to the data processing device 5.

A joint angle sensor (work robot sensor) 8 is attached to each drive unit of the arm mechanism 71 of the work robot 7 for work phase determination. The joint angle sensor 8 is, for example, a laser distance meter, encoder, potentiometer, inclinometer, geomagnetic sensor, gyro sensor as a detection device that is mounted in the work robot 7 and detects the amount of movement and rotation of each drive unit. Any one or more of them. In addition to the joint angle sensor 8, as a detection device for detecting the interaction with the object 12, a camera that acquires image data of the surrounding environment, an ultrasonic distance meter, the shape of the surrounding environment and the object 12 are measured. Any one or more of: a laser distance meter, a force / torque sensor that measures the force and torque applied to the hand of the robot, a thermometer, a pressure sensor, and a current sensor as a detection device that detects the operation of each drive unit Can be used, and any one or more of these sensors can be used instead of the joint angle sensor 8.

The operator 1 who manages and monitors the work inputs a work instruction to the operation control device 4 using a joystick controller (operation device) 3 that instructs the operation of the work robot 7 by the operator 1 to operate the work robot 7.

In the environment where the operator 1 is present, an operator sensor used to determine the work phase by detecting the behavior of the operator 1 is used to set the gaze point of the operator 1 in the monitoring video displayed on the monitoring video display unit 45. A line-of-sight measuring device 2 for detection is provided.

In addition to the line-of-sight measuring device 2, the operator sensor is a force / torque sensor that detects an operator's operation, a pressure-sensitive sensor, a geomagnetic sensor, a gyro sensor, an acceleration sensor, or a non-contact type sensor such as a motion capture. One or more sensors can be used.

The autonomous monitoring robot 10 is a robot that monitors the remote work performed by the work robot 7, and includes a moving mechanism 102 that moves the autonomous monitoring robot 10, a monitoring camera that captures the work robot 7, and an arm mechanism 101 that holds the monitoring camera. And a joint angle sensor (autonomous monitoring robot sensor) 100 attached to each drive unit of the autonomous monitoring robot 10. The monitoring camera can be any one of the optical camera 11, an ultrasonic scanner, and a laser scanner. Here, the optical camera 11 is used.

The autonomous monitoring robot 10 is connected to the autonomous monitoring robot control device 9 in a wired and / or wireless manner, and the autonomous monitoring robot control device 9 is connected to the data processing device 5.

In the autonomous monitoring robot 10, the work phase is determined in the data processing device 5 based on the signals from the line-of-sight measuring device 2 worn by the operator 1 and the joint angle sensor 8 of the work robot 7, and in the autonomous monitoring robot control device 9. The operation of the autonomous monitoring robot 10 is generated. During the operation, the operator 1 can monitor the progress of the operation, the state of the apparatus, and the like through the monitoring video from the optical camera 11 attached to the autonomous monitoring robot 10.

The monitoring video display unit (display device) 45 is a display such as a liquid crystal that displays the monitoring video captured by the optical camera 11 of the autonomous monitoring robot 10.

The operation control device 4 is a device that generates an operation signal for operating the work robot 7 in response to the input of the operation signal of the operator 1 by the joystick controller 3, and includes an operation input unit 40, a target value calculation unit 41, and a gazing point position pattern detection. Section (pattern detection section) 42, monitoring video output section 43, and data transmission / reception section 44.

The operation input unit 40 is a part in which the operator 1 inputs an operation input using the joystick controller 3 to the operation control device 4.

The target value calculation unit 41 is a part that calculates the target motion of the work robot 7 based on the operation input captured by the operation input unit 40.

The gazing point position pattern detection unit 42 is a part that detects a work position pattern that the operator 1 gazes in the monitoring image displayed on the monitoring image display unit 45 by the line-of-sight measurement device 2 worn by the operator 1.

The monitoring video output unit 43 is a part that outputs a monitoring video for the operator 1 to the monitoring video display unit 45.

The data transmission / reception unit 44 is a part that manages data transmission / reception with the data processing device 5.

The data processing device 5 is a device that generates a control operation signal for an autonomous monitoring operation by the autonomous monitoring robot 10 based on the sensor information from the line-of-sight measurement device 2 and the joint angle sensor 8. A determination unit 51, a storage unit 53, a camera position / posture generation unit 52, and an autonomous operation calculation unit 55 are included.

The data transmission / reception unit 50 is a part that manages data exchange with the operation control device 4, the work robot control device 6, and the autonomous monitoring robot control device 9.

The work phase determination unit 51 is a part that determines a work phase. In this work phase determination unit 51, the operation speed of the arm mechanism 71 of the work robot 7 is obtained from the signal of the joint angle sensor 8 mounted on the work robot 7, the operation speed of the arm mechanism 71, the arm mechanism 71, the object 12, The work phase is determined from the relationship with the distance, and a control operation signal suitable for the determined work phase is generated. Further, the work phase determination unit 51 determines and generates a work phase based on the gaze point of the operator 1 acquired by the line-of-sight measurement device 2, and outputs the determined work phase to the camera position / posture generation unit 52.

The storage unit 53 is a device that includes a hard disk and a RAM that store work training data 54 used to determine the work phase in the work phase determination unit 51.

The camera position / posture generation unit 52 is a part that generates the position of the optical camera 11 from the estimation result of the work phase and the geometric positional relationship between the work robot 7 and the object 12.

The autonomous movement calculation unit 55 is a part that generates an operation signal necessary for the movement of the autonomous monitoring robot 10 for placing the optical camera 11 at the camera target position.

The work robot control device 6 is a device that converts an operation signal input by the joystick controller 3 and generated in the operation control device 4 into a command signal of the work robot 7, and includes a data transmission / reception unit 60, a command voltage calculation unit 61, And a robot controller 62.

The data transmission / reception unit 60 is a part that exchanges data with the data processing device 5.

The command voltage calculation unit 61 receives the target control amount (the crawler rotation speed of the moving mechanism 72, the crawler rotation speed of the moving mechanism 72, the arm mechanism 71) from the command signal of the working robot 7 transmitted from the data processing device 5. This is a part for calculating the joint angle of each joint.

The robot control unit 62 uses the calculated target value and the current angle (current posture) calculated from the signal of the joint angle sensor 8 built in each drive unit of the work robot 7 to set the drive unit of each work robot 7. This is the part that generates the target command voltage.

The autonomous monitoring robot control device 9 is a device that converts the control operation signal generated by the data processing device 5 into a command signal of the autonomous monitoring robot 10, and includes a data transmission / reception unit 90, a command voltage calculation unit 91, a robot control unit 92, And a monitoring video acquisition unit 93.

The data transmission / reception unit 90 is a part that exchanges data with the data processing device 5.

The command voltage calculation unit 91 receives the target control amount (the crawler rotation speed of the movement mechanism 102, the arm mechanism) from the command signal of the autonomous monitoring robot 10 transmitted from the data processing device 5 based on the movement mechanism 102 and the arm mechanism 101 of the autonomous monitoring robot 10. (Joint angle of each joint 101) is calculated.

The robot control unit 92 drives each of the autonomous monitoring robots 10 using the calculated target value and the current angle (current posture) calculated from the signal of the joint angle sensor 100 built in each driving unit of the autonomous monitoring robot 10. This is a part for generating the target command voltage of the part.

The monitoring video acquisition unit 93 is a part that acquires a video from the optical camera 11 attached to the arm mechanism 101 of the autonomous monitoring robot 10.

The operation control device 4, the data processing device 5, the work robot control device 6, and the autonomous monitoring robot control device 9 can be appropriately integrated and divided, and are not limited to the configurations shown in FIGS. For example, the operation control device 4 and the data processing device 5 may be integrated into a single device, or the work robot control device 6 may be divided into a movement mechanism and a work mechanism. In addition, each device is connected by a wired cable, but a wireless system configuration may be used.

Next, a method for controlling the remote work robot including the work robot 7 and the autonomous monitoring robot according to the present embodiment will be described with reference to FIGS. 3 to 6C.

FIG. 3 is a PAD diagram schematically showing the contents of the operation control process and the autonomous monitoring robot control process of the work robot by the remote work robot control system of the present embodiment. FIG. 4 is a PAD showing a conversion process flow from the work robot joint angle sensor signal to the autonomous monitoring operation in the remote work robot control system of the present embodiment.

First, the overall flow of the remote robot control process will be described with reference to FIG. In the remote work robot control system 1000, the processing shown in FIG. 3 is continuously executed while the power is on.

In FIG. 3, the remote work robot control system 1000 first initializes the joint angle sensor 8 of the work robot 7 (step S1).

Next, when it is recognized that an operation instruction has been input by the operator 1 (step S10), the following processing is started.

First, the moving direction / moving speed of the moving mechanism 72 is calculated according to the input signal (step S11), and a control amount (voltage command value of a crawler or the like) of the moving mechanism 72 is generated (calculated) (step S12). . In addition, the position and angle of the target joint is calculated (step S13) using inverse kinematics that obtains the joint coordinates of the arm mechanism 71 from the arm hand position indicated by the operator 1, and the control amount of the arm mechanism 71 ( A voltage command value for each joint motor is generated (calculated) (step S14).

Next, the operation generation process of the work robot 7 is started (step S20). This step S20 is a work robot control step for converting the operation signal of the work robot 7 according to the operation instruction of the operator 1 into the command signal of the work robot 7, and the robot control calculated from the work instruction in the above-described steps S12 and S14. Based on the quantity, a target action of the work robot 7 is generated.

In step S20, first, when the voltage command values of the moving mechanism 72 and the arm mechanism 71 calculated in steps S12 and S14 are read (step S21), the voltage command values to each motor are output (step S22). Thereafter, based on the information of the joint angle sensor 8 mounted on the work robot 7, the state of the work robot 7 is detected (step S 23), and the joint angle sensor 8 is directed toward the operation control device 4 via the data processing device 5. The acquired data is transmitted (step S24).

Next, based on the information obtained from the work robot controller 6, the work situation is analyzed (step S30). This step S30 is for an autonomous monitoring operation by the autonomous monitoring robot 10 based on sensor information from the line-of-sight measuring device 2 that detects the behavior of the operator 1 and the joint angle sensor 8 attached to each drive unit of the work robot 7. This is a data processing step for generating the control operation signal.

In step S30, first, the acquired data of the joint angle sensor 8 of the work robot 7 output from the data processing device 5 is received (step S31). Next, the posture of the work robot 7 is derived from the acquired data by forward kinematics (step S32). Thereafter, an operation signal (operation speed of the arm mechanism 71) such as a positional relationship between the posture of the work arm and the object 12 (a relationship between the distance between the arm mechanism 71 and the object 12), a time change of the posture of the work arm, and the like. The work phase is determined based on the gazing point position pattern (gaze point of the operator 1) (step S33). Thereafter, the target position / orientation of the optical camera 11 is generated from the positional relationship between the work robot 7 and the object 12 according to the work phase determined in step S33 (step S34). Next, the autonomous operation (the control amount of the moving mechanism 102, etc.) of the autonomous monitoring robot 10 that calculates the target position / posture of the optical camera 11 generated in step S34 is calculated (step S35). Thereafter, a control amount (voltage command value of each joint motor) of the arm mechanism 101 of the autonomous monitoring robot 10 is generated (step S36).

Next, based on the control amount of the autonomous monitoring robot 10 calculated in step S30, an action generation process for generating a target action of the autonomous monitoring robot 10 is started (step S40). This step S40 is an autonomous monitoring robot control step for converting the control operation signal generated in step S35 in the data processing step into a command signal for the autonomous monitoring robot 10.

In step S40, first, when the voltage command values of the moving mechanism 102 and the arm mechanism 101 calculated in step S35 are read (step S41), the voltage command values to each motor are output (step S42). Thereafter, the monitoring video is acquired by the optical camera 11 mounted on the autonomous monitoring robot 10 (step S43), and the acquired monitoring video is transmitted to the data processing device 5 (step S44).

Next, presentation of the monitoring video to the operator 1 is started (step S50). This step S50 is a display step for displaying a monitoring video imaged by the optical camera 11 of the autonomous monitoring robot 10.

In step S50, first, the monitoring video output unit 43 of the operation control apparatus 4 receives the monitoring video output in step S44 (step S51), and then outputs the monitoring video to the monitoring video display unit 45 (step S51). Step S52).

Here, the details of the work phase determination process based on the arm operation signal and the gaze position pattern in step S33 will be described with reference to FIG.

In FIG. 4, first, the data of the joint angle sensor 8 mounted on the work robot 7 is read, and the gaze position pattern of the operator 1 in the monitoring video displayed on the monitoring video display unit 45 is read from the line-of-sight measuring device 2 ( Step S100).

Thereafter, the position / posture / movement speed of the work robot 7 is derived (step S101), and the geometric positional relationship between the object 12 and the arm mechanism 71 is detected (step S102).

Thereafter, the work robot in the work training data 54 acquired in advance by training or the like and stored in the storage unit 53 with the current position / posture / movement speed and geometrical positional relationship of the current work robot 7 derived and detected. The work phase is determined from the movement signal such as the position of 7, posture, moving speed, positional relationship with the object 12 and the operator 1 gazing point position (step S 103).

Also, in order to update the work training data 54 for the current work, a combination of the current operation signal and the work phase is recorded (step S104). The above is the basic process of the work phase determination process.

Next, an example of the determination of the work phase in step S33 in step S30 in FIG. 3 will be described in detail with reference to FIGS. 5A to 6C.

FIG. 5A and FIG. 5B are diagrams showing an example of work phase determination based on sensor signals in the remote work robot control system of this embodiment. FIG. 6A is a diagram illustrating a relationship example of a target geometric arrangement in the remote operation robot control system of the present embodiment, and FIGS. 6B and 6C are diagrams illustrating examples of determining a target geometric arrangement.

First, using FIG. 5A and FIG. 5B, determination of the work phase based on the relationship between the operation speed of the arm mechanism 71 of the work robot 7 and the distance between the arm mechanism 71 and the object 12 when the remote work is gripped. An example of processing will be described.

In the remote robot control method according to the present embodiment, the work phase until the completion of gripping is performed in advance as an “approach phase” in which the arm mechanism 71 approaches or separates from the object 12, and the object is detected by the arm mechanism 71. This is a “position adjustment phase” in which a subtle position for gripping 12 is adjusted, and a case where the speed of the arm mechanism 71 is higher than the proximity of the distance between the object 12 and the work robot 7 and is not a normal operation. It is set in three stages of “abnormal operation phase”. Further, the boundaries between the respective work phases are obtained from data at the time of remote training by the operator 1 in advance, and are stored in the storage unit 53 as work training data 54 as shown in FIG. 5A.

At the time of actual remote work, the combination of the distance value between the arm mechanism 71 of the work robot 7 and the object 12 and the speed value of the arm mechanism 71 of the work robot 7 in the work phase determination unit 51 is shown in FIG. 5A. It is determined online where the position is, and the work phase to which the determined position belongs is output to the camera position / posture generation unit 52.

At this time, when the arm mechanism 71 of the work robot 1 is 0, it is determined that the operation is stopped and the current state maintenance phase is in progress.

When the distance between the target object and the work robot is 0, it is determined that the gripping operation is being performed and that it is in the “work phase”. In this case, it is desirable to output a signal directed to the monitoring video display unit 45 so that “monitoring is possible” is displayed on the monitoring video display unit 45 of the operation control device 4.

The work phase determination unit 51 in this step records the combination of the work phase determined to be the current operation signal as shown in step S104 of FIG. 4, and the operator 1, work environment as shown in FIG. 5B. It is desirable that the boundary value of the work phase is updated according to 13 and stored in the storage unit 53.

In the abnormal operation phase, the operation of the arm mechanism 71 and the moving mechanism 72 is stopped or an operation input signal from the joystick controller 3 of the operator 1 is received in order to suppress the occurrence of an unexpected situation. It is desirable to output a signal for invalidation via the camera position / posture generation unit 52.

Next, an example of work phase determination processing using the gaze position pattern of the operator 1 acquired by the gaze measuring device 2 worn by the operator will be described with reference to FIGS. 6A to 6C.

As shown in FIG. 6A, the work phase determination unit 51 divides the position of the gazing point in the monitoring video into three regions: a work robot body J1, a work robot hand J2, and a target object J3.

When the gazing points are thus divided, the work phase determination unit 51 obtains for each gazing point how long the operator 1 has gazed at each area, as shown in FIG. 6B and FIG. The ratio (gaze ratio) at which each position is watched per unit time for operating 7 is obtained. Then, the work phase is determined from the obtained stop ratio.

For example, as shown in FIG. 6B, when each area of the work robot body J1, the work robot hand J2, and the target object J3 is relatively evenly watched, it is determined to be in the “approach phase”. Further, as shown in FIG. 6C, when the ratio of gazing at the work robot hand J2 and the object J3 is higher than that of the work robot body J1, it is determined that it is the “position adjustment phase”.

When it is determined that the ratio of the gazing point is different from J1, J2, and J3 is high, or when the operator 1 is not gazing at the monitoring image and there is almost no gazing point per unit time. When it is determined, it is desirable to determine that it is in the “current status maintenance phase”.

In addition, the work phase determination unit 51 in this step records the combination of the current gaze ratio and the work phase, updates the boundary value of the work phase, and stores it in the same manner as shown in step S104 of FIG. 4 or FIG. 5B. It is desirable to store it in the unit 53.

5A and 5B, the determination result of the work phase based on the relationship between the operation speed of the arm mechanism 71 of the work robot 7 and the distance between the arm mechanism 71 and the object 12, and FIGS. 6A to 6C. When the work phase determination result based on the gaze point of the operator 1 acquired by the line-of-sight measurement device 2 as shown in FIG. 2 is different, the work phase determination unit 51 may prioritize which one and give priority. I can leave.

Next, an example of a method for calculating the position of the optical camera 11 provided in the autonomous monitoring robot 10 in step S34 in step S30 in FIG. 3 will be described.

In the camera position / posture generation unit 52, when the work phase determination unit 51 determines that the work phase is “approach phase”, the entire work robot 7 and the object 12 are captured in the monitoring image from the optical camera 11, and the positional relationship Generates a location that is easy to understand. Further, when the determined work phase is “position adjustment phase” or “work phase”, a position where the optical camera 11 zooms the object 12 and the arm mechanism 71 of the work robot 7 is generated. Furthermore, when it is determined as the “current state maintenance phase”, a position signal is generated so as to maintain the current position.

On the other hand, when the determined work phase is “abnormal operation”, the position and orientation of the optical camera 11 of the autonomous monitoring robot 10 are not calculated so as to maintain the current position and orientation. . In this case, it is desirable to output a signal to the monitoring video display unit 45 so as to display a warning to the effect that “abnormal operation” has been performed on the monitoring video display unit 45 of the operation control device 4.

Next, the effect of this embodiment will be described.

According to the remote work robot control system 1000 and the remote work robot control method of the embodiment of the present invention described above, the operator is based on sensor information from at least one of the line-of-sight measurement device 2 and the joint angle sensor 8. One operation is classified, and an operation signal necessary for the autonomous monitoring operation of the autonomous monitoring robot 10 is generated. For this reason, it is possible to autonomously acquire the monitoring video necessary for the operation from the operation related to the operation of the work robot 7 while reflecting the intention of the operator 1, and it is difficult to use the conventional technique without arranging the monitoring operator. Thus, it becomes possible to flexibly cope with changes in the work environment and work situation that occur during work. In addition, since no monitoring operator is required, it is not necessary to spend time for communication between operators, and a plurality of robots can be operated by a small number of operators.

Further, the data processing device 5 determines the work phase from the relationship between the operation speed of the arm mechanism 71 of the work robot 7 and the distance between the arm mechanism 71 and the object 12, and outputs a control operation signal suitable for the determined work phase. Therefore, it is possible to acquire a monitoring video reflecting the intention of the operator 1 with high accuracy, and to respond more flexibly to changes in the work environment and work situation that occur during work.

Further, the operator sensor is a line-of-sight measuring device 2 that detects the gaze point of the operator 1 in the monitoring video displayed on the monitoring video display unit 45, and the data processing device 5 is the operator 1 acquired by the line-of-sight measuring device 2. It is also possible to obtain a monitoring video reflecting the intention of the operator 1 with high accuracy by determining the work phase based on the gazing point and generating a control operation signal suitable for the determined work phase. It is possible to respond more flexibly to changes in the work environment and work situation that occur inside.

<Others>
The present invention is not limited to the above-described embodiment, and includes various work examples (cutting, drilling, etc.). Further, the above-described embodiment has been described in detail for easy understanding of the present invention, and is not necessarily limited to all functions. Moreover, you may implement | achieve part or all of said each structure, function, etc., for example by designing with an integrated circuit. Each of the above-described configurations, functions, and the like may be realized by software by interpreting and executing a program that realizes each function by the processor.

DESCRIPTION OF SYMBOLS 1 ... Operator 2 ... Eye-gaze measuring device 3 ... Joystick controller 4 ... Operation control device 5 ... Data processing device 6 ... Work robot control device 7 ... Work robot 8 ... Joint angle sensor 9 ... Autonomous monitoring robot control device 10 ... Autonomous monitoring robot 11 ... Optical camera 12 ... Object 13 ... Work environment 40 ... Operation input unit 41 ... Target value calculation unit 42 ... Gaze point position pattern detection unit 43 ... Monitor video output unit 44 ... Data transmission / reception unit 45 ... Monitor video display unit 50 ... Data Transmission / reception unit 51 ... Work phase determination unit 52 ... Camera position / posture generation unit 53 ... Storage unit 54 ... Work training data 55 ... Autonomous motion calculation unit 60 ... Data transmission / reception unit 61 ... Command voltage calculation unit 62 ... Robot control unit 71 ... Arm Mechanism 72 ... Movement mechanism 90 ... Data transmission / reception unit 91 ... Command voltage calculation unit 92 ... Robot control unit 93 ... Monitoring video acquisition unit 1 0 ... joint angle sensor 101 ... arm mechanism 102 ... moving mechanism 1000 ... remote working robot control system

Claims

A work robot located at a remote location different from the location of the operator,
An operation device for instructing the operation of the work robot by the operator;
An operation control device that receives an input of the operation signal of the operator by the operation device and generates an operation signal for operating the work robot;
A work robot controller that converts the operation signal into a command signal of the work robot;
A work robot sensor attached to each drive unit of the work robot;
An autonomous monitoring robot having a monitoring camera for imaging the work robot, and monitoring a remote work by the work robot;
An operator sensor for detecting the behavior of the operator;
A data processing device that generates a control operation signal for an autonomous monitoring operation by the autonomous monitoring robot based on sensor information from at least one of the operator sensor and the work robot sensor;
An autonomous monitoring robot control device that converts the control operation signal generated by the data processing device into a command signal of the autonomous monitoring robot;
A remote working robot control system comprising: a display device that displays a monitoring image captured by the monitoring camera of the autonomous monitoring robot.
The remote work robot control system according to claim 1,
The data processing device determines a work phase from a relationship between an operation speed of a gripping arm of the work robot and a distance between the gripping arm and an object, and generates the control operation signal suitable for the determined work phase. Remote control robot control system characterized by
The remote work robot control system according to claim 1,
The operator sensor is a line-of-sight measuring device that detects an operator's gazing point in the monitoring video displayed on the display device,
The data processing device determines a work phase based on an operator's gaze point acquired by the line-of-sight measurement device, and generates the control operation signal suitable for the determined work phase. .
The remote work robot control system according to claim 1,
The data processing device includes:
A data transmission / reception unit for managing data exchange with the operation control device, the work robot control device, and the autonomous monitoring robot control device;
A work phase determination unit for estimating a remote work phase from signals of the operator sensor and the work robot sensor;
A storage unit for storing work training data used for determining a work phase in the work phase determination unit;
A camera position / posture generation unit that generates the position of the monitoring camera from the estimation result of the work phase and the geometric positional relationship between the work robot and an object;
A remote operation robot control system comprising: an autonomous motion calculation unit that generates an motion signal necessary for the motion of the autonomous surveillance robot for placing the surveillance camera at a target position.
The remote work robot control system according to claim 1,
The operation control device includes:
An operation input unit for capturing the operation input of the operator;
A target value calculation unit that calculates a target action of the work robot based on the captured operation input;
A monitoring video output unit for outputting a monitoring video to the operator;
A remote work robot control system comprising a data transmission / reception unit for managing data transmission / reception.
The remote work robot control system according to claim 1,
The operator sensor is at least one of a line-of-sight measurement device, a force / torque sensor, a pressure-sensitive sensor, a geomagnetic sensor, a gyro sensor, and an acceleration sensor,
The said operation control apparatus has a pattern detection part which detects the work position which the operator makes into the object in the monitoring image | video displayed on the said display apparatus. The remote work robot control system characterized by the above-mentioned.
The remote work robot control system according to claim 1,
The work robot sensor is mounted in the work robot, and a laser distance meter, encoder, potentiometer, inclinometer, geomagnetic sensor, gyro sensor, object as a detection device for detecting the movement amount and rotation amount of each drive unit Cameras, ultrasonic rangefinders, laser rangefinders, force / torque sensors, thermometers, pressure sensors, current sensors as detection devices that detect the operation of each drive unit A remote working robot control system characterized by being at least one of the above.
The remote work robot control system according to claim 1,
Work robot controller
A data transmission / reception unit for exchanging data with the data processing device;
A command voltage calculation unit for calculating a target control value of the moving mechanism and arm mechanism of the work robot from the command signal of the work robot;
A robot control unit that generates a target command voltage for each drive unit of the work robot using a calculated target control value and a current posture calculated from a signal of the work robot sensor. Robot control system.
The remote work robot control system according to claim 1,
Further comprising an autonomous monitoring robot sensor attached to each drive unit of the autonomous monitoring robot;
The autonomous monitoring robot control device includes:
A data transmission / reception unit for exchanging data with the data processing device;
A command voltage calculation unit that calculates a target control value of the movement mechanism and arm mechanism of the autonomous monitoring robot from the control operation signal;
A robot control unit that generates a target command voltage for each drive unit of the autonomous monitoring robot using the calculated target control value and the current posture calculated from the signal of the autonomous monitoring robot sensor;
A remote operation robot control system comprising: a monitoring video acquisition unit that acquires video from the monitoring camera.
The remote work robot control system according to claim 1,
The remote operation robot control system, wherein the monitoring camera includes any one of an optical camera, an ultrasonic scanner, and a laser scanner.
A control method for a remote work robot comprising a work robot arranged in a remote place different from the place where the operator is located, and an autonomous monitoring robot having a monitoring camera for imaging the work robot and monitoring the remote work by the work robot There,
A work robot control step of converting an operation signal of the work robot according to an operation instruction of the operator into a command signal of the work robot;
For autonomous monitoring operation by the autonomous monitoring robot based on sensor information from at least one of an operator sensor for detecting the behavior of the operator and a working robot sensor attached to each drive unit of the working robot. A data processing step for generating a control action signal;
An autonomous monitoring robot control step of converting the control operation signal generated in the data processing step into a command signal of the autonomous monitoring robot;
A display step of displaying a monitoring image captured by the monitoring camera of the autonomous monitoring robot.
The method of controlling a remote work robot according to claim 11,
In the data processing step, a work phase is determined from a relationship between an operation speed of a gripping arm of the work robot and a distance between the gripping arm and an object, and the control operation signal suitable for the determined work phase is generated. A method for controlling a remote working robot.
The method of controlling a remote work robot according to claim 11,
The operator sensor is a line-of-sight measuring device that detects an operator's gazing point in the monitoring video displayed by the display step,
In the data processing step, a work phase is determined based on an operator's gaze point acquired by the line-of-sight measurement device, and the control operation signal suitable for the determined work phase is generated. Method.