JP6383716B2 - Control device and control method for drone - Google Patents

Description

  The present disclosure relates to a control device and a control method for remotely controlling a drone via communication with the drone.

  Remotely controlled drones are used in various scenes in society. For example, various drones for decommissioning of nuclear reactors have been developed and put on-site for light work and investigation. By the way, if a method of performing manual remote control of an unmanned aircraft using a control device that communicates with the unmanned aircraft, communication delay between the control device and the unmanned aircraft cannot be ignored. Due to the communication delay, even if the drone transmits the actual measured value of the state of its own machine to the control device side, the operation command received by the control device side with a delay is transmitted to the drone. Therefore, it is received with a delay at the drone side. In an environment where communication delay cannot be ignored, the drone is forced to perform a control operation close to inching in actual operation, although it can move at high speed in terms of specifications. As a result, the work performed by remotely controlling the drone in such an environment is very inefficient, resulting in work delays, and conversely, the operator predicts the drone's operation by itself and performs high-speed operation. Attempts to do so will result in operator error.

  As a conventional example of the remote control system in consideration of the influence of the communication delay described above, there is a remote control system described in Patent Document 1. In the invention described in Patent Document 1, the drone compares the estimated position of the own machine estimated by an estimation algorithm or the like with the measured position of the own machine, and only when the error between the estimated position and the measured position is large. The actual measurement position is transmitted to the control device. In the invention described in Patent Document 1, the control device tracks the estimated position of the unmanned aircraft with the same estimation algorithm as that of the unmanned aircraft, and receives the measured position. Update with position.

Japanese Patent No. 5634760

  Even if a high-speed communication system is to be introduced as a solution to the above problem caused by communication delay, a communication environment capable of achieving sufficient performance cannot always be constructed, and such a communication system can be used for drone control. It is difficult to realize. For example, in a reactor building, a communication system that connects a drone and a control device via an optical fiber line capable of broadband communication is employed. However, since the optical fiber line is shared with other equipment in the reactor building and a specific drone cannot occupy the communication band, it is not possible to perform other operations during remote operation of the drone via the optical fiber line. When communication by these devices occurs, an unexpected communication delay occurs.

  In the remote control system described in Patent Document 1, the measured position is transmitted from the drone to the control device only when there is a large error between the estimated position of the drone and the measured position. The amount of communication bandwidth used is reduced, thereby reducing communication delay. However, Patent Document 1 discloses a mechanism for reflecting a feedback signal received from an unmanned aerial vehicle (for example, an actual measurement value obtained by actually measuring the state of the unmanned aircraft) in a drone control operation in consideration of a communication delay on the control device side. And does not disclose any means.

  In view of the above problems, at least one embodiment according to the present invention provides an actual measurement value received from the drone when the operator of the control device performs manual remote control of the drone while viewing image information around the drone. An object of the present invention is to obtain a control device and a control method that are reflected in the drone control operation in consideration of communication delay on the control device side.

(1) A control device according to some embodiments of the present invention includes a display device capable of sequentially displaying the current estimated state of the drone on a screen,
An operation command input by an operator based on the current estimated state of the drone displayed on the display device for remote control of the drone can be transmitted to the drone, and the drone A communication unit capable of sequentially receiving actual measurement values of the state of the own machine from the drone,
The display device displays the current estimated state of the unmanned aircraft obtained by executing a time-series operation for obtaining the current estimated state of the unmanned aircraft from the operation command based on a control model that reflects the actual measurement values. Two or more calculation units including a first calculation unit and a second calculation unit that can be provided to,
A control device comprising:
The first calculation unit and the second calculation unit are configured to alternately receive the actual measurement value, and each time the actual measurement value is received, the time calculation operation is repeatedly performed.
The first calculation unit is based on one control model reflecting one actually measured value, which is received alternately, until one time when the time series calculation catches up from the actually measured time of the one actually measured value to the current time. The time-series operations can be executed sequentially at a speed higher than the passage of real time,
Based on another control model reflecting other actually measured values received alternately, the second calculating unit is based on the time series calculation from the actually measured time of the other actually measured values to another time that catches up with the current time. The time-series operations can be executed sequentially at a speed higher than the passage of real time,
The first calculation unit is configured to provide the display device with a current estimated state of the drone from the one time to the next other time,
The second calculation unit is configured to provide the display device with a current estimated state of the drone from the other time to the next one time.

In the configuration of (1) above, when a new actual measurement value is received from the drone, a new control model is obtained by reflecting the new actual measurement value on the existing control model and updating it. The value can improve the modeling error of the control model. However, for that purpose, it is necessary to compensate for the update delay of the control model caused by the communication delay from the actual measurement time of the new actual measurement value to the current time. Therefore, in the configuration of (1) above, the operation of obtaining the current estimated state of the unmanned aircraft in time series based on the new control model starts from the actual measurement time until it catches up with the current time. Run faster.
As described above, according to the configuration of the above (1), the latest measured value is given to the unmanned machine while giving feedback for the unmanned machine control to the operator even when the new measured value cannot be received from the unmanned machine due to communication delay. The latest measured value can be reflected in the feedback to the operator in a form that compensates for the update delay of the control model caused by the communication delay. Therefore, according to the configuration of (1) above, when the operator of the control device performs manual remote control of the drone by relying on information around the drone, the measured value received from the drone is considered in communication delay. In addition, it can be reflected in the unmanned aircraft control operation on the control device side.

In the configuration of (1) above, even if the latest measured value representing the most recent state of the drone is received with a delay, the previously received measured value is not received until the delayed reception. Based on the reflected control model, the current state of the drone can be sequentially displayed on the screen while estimating the state in time series. Therefore, the current state of the drone that should be displayed on the screen in sequence until the control device receives the latest actual measured value with a delay based on the control model that reflects the previous actual measured value. The state can be estimated continuously. Therefore, since there is no blank time during which control cannot be performed while viewing the current state of the drone before the latest measured value is received by the control device late, the operator of the control device Manual remote control of the drone can be performed without being aware of communication delay.
Further, in the configuration of (1) above, after the control device receives the latest measured value representing the most recent state of the drone with a delay, based on the control model reflecting the latest measured value, The current state can be estimated and displayed on the screen. Therefore, if the operator performs manual remote control of the drone while viewing the current state of the drone estimated based on the control model on the screen of the display device, the modeling error of the control model over time It is possible to estimate the current state of the drone without accumulating.

(2) In some embodiments of the present invention, in the configuration of (1), the control model is configured to include a state estimation filter,
From the actual measurement time of the actual measurement value to the time of catching up with the current time, the time series calculation executed at a speed higher than the passage of real time includes a post-prediction operation that reflects the actual measurement value in the internal state of the state estimation filter. The state estimation filter is configured to include a high-speed state prediction operation for performing the prediction operation of the state estimation filter at a speed higher than the passage of real time based on the internal state reflecting the actual measurement value,
The process of providing the display device with the current estimated state of the drone continues the high-speed state prediction operation after the time-series operation executed faster than the passage of real time catches up with the current time The state estimation filter is configured to switch to a pre-prediction operation that displays the time-series estimation state on the screen while performing the prediction operation of the state estimation filter with the passage of real time.

In the configuration of (2) above, the control model is configured to include a state estimation filter. In the configuration of (2) above, unlike the case where the control model is expressed by a transfer function other than the state estimation filter, the process of estimating the current state of the unmanned aircraft in time series is performed in each loop by the state estimation filter. The process can be executed in the time domain as a simple iterative loop process corresponding to each time on the time axis.
In the configuration of (2), following the execution of the a posteriori prediction operation in which the newly received actual measurement value is reflected in the internal state of the state filter, the actual measurement value is calculated using the actual measurement time of the actual measurement value as a starting point. Based on the reflected internal state, the prediction operation of the state estimation filter is executed as a high-speed state prediction operation faster than real time, and when the execution of the high-speed state prediction operation catches up with the current time, the prediction operation of the state estimation filter is performed. Switch to advance prediction operation and continue as it is.
Therefore, in the configuration of (2) above, when realizing the process of estimating the current state of the drone in a time series based on the control model, each loop process in the time domain described above is completed, The current state of the drone can be directly obtained from the state variable calculated in each loop process. As a result, in the process of estimating the current state of the drone, the process of converting the internal variable value of the control model into the current state of the drone is unnecessary, and the calculation can be speeded up.
In the configuration of (2) above, in order to realize the process of estimating the current state of the drone based on the control model reflecting the latest measured value, at the time when the latest measured value is actually measured. It is only necessary to continue the prediction operation as it is after reflecting the latest measured value in the state variable value of the state estimation filter in the corresponding loop processing. As a result, in order to reflect the latest measured values in the control model, it is not necessary to convert the latest measured values into the internal representation of the control model and apply them to the internal state of the control model, speeding up the calculation. can do.

(3) In some embodiments of the present invention, in the configuration of (1) or (2), the two or more arithmetic units include a third arithmetic unit, and the third arithmetic unit includes:
Based on the one control model obtained by reflecting the one actual measurement value on the control model, a time-series operation for obtaining a current estimated state of the drone from the operation command in time series is performed. It is configured to execute faster than the passage of real time, from the time when the actual measurement value of the actual measurement passed through the current time to the time of one future.
The time series calculation by the third calculation unit is configured to be executed in parallel with the time series calculation by the first calculation unit and the second calculation unit,
The current estimated state of the drone obtained by the time-series operation by the third operation unit after the time-series operation by the third operation unit reaches the one future time point is determined as the future of the unmanned aircraft. The estimated state can be provided to the display device.

  According to the configuration of (3) above, the state of the drone is estimated in time series from the current time to the future time, and the state of the drone from the current time to the future time is superimposed on the current state of the drone. Can be displayed on the screen. As a result, according to the configuration of (3) above, when remote control of the drone is continued along the trajectory assumed by the operator, the drone touches an obstacle at a future point in time, The possibility of doing this can be detected in advance. Therefore, in such a case, the operator can correct the trajectory in advance. In particular, when the operator continues to control the drone by overlooking an obstacle hidden in the blind spot when viewed from the field of view of the area around the drone, the drone touches the obstruction due to an operator error, or the scorpio It is possible to detect the possibility of the drone in advance and prevent an operation error of the drone.

(4) In some embodiments of the present invention, in the configuration of (3), the two or more calculation units include a fourth calculation unit, and the fourth calculation unit includes:
Based on the other control model obtained by reflecting the other actual measurement value on the control model, a time series operation for obtaining the current estimated state of the drone from the operation command in time series is performed. It is configured to execute faster than the passage of real time, from the time when the actual measurement value of the actual measurement passed through the current time to another future time point,
The time series computation by the fourth computation unit is configured to be executed in parallel with the time series computation by the first computation unit, the second computation unit, and the third computation unit,
The current estimated state of the unmanned aircraft obtained by the time-series computation by the fourth computation unit after the time-series computation by the fourth computation unit reaches the other future time point is determined as the future of the unmanned aircraft. The estimated state can be provided to the display device,
The third calculation unit is configured to provide the display device with an estimated future state of the drone from the one future time point to the next other future time point,
The fourth calculation unit is configured to provide the display device with an estimated future state of the drone from the other future time point to the next one future time point.

  According to the configuration of (4) above, the fourth calculation unit time-series the current estimated state of the drone from the operation command based on another control model obtained by reflecting other actual measurement values in the control model. The time-series calculation to be obtained is executed at a speed higher than the lapse of the real time from the time when the other actual measurement value is actually measured to the other future time point through the current time. At this time, the other actual measurement value received by the fourth calculation unit is the same actual measurement value as the measurement value received alternately by the second calculation unit, so that it is more than the one actual measurement value received by the third calculation unit. It is a new measured value later. In other words, the fourth calculation unit performs the above time-series calculation based on a control model that reflects a newer measurement value than the control model used by the third actual measurement value, and passes the current time to another future time point. To run faster than real time. At this time, the other future time point is a time at which the third calculation unit completes the operation of providing the display device with the estimated future state of the drone. Then, after the time series computation by the fourth computation unit reaches the other future time point, the current estimated state of the drone obtained by the time series computation by the fourth computation unit is The estimated state of the future can be provided to the display device.

  Therefore, according to the configuration of the above (4), the fourth calculation unit performs the time series calculation based on the control model reflecting the actual measurement value newer than the control model used by the third calculation unit. Run faster than real time until time is reached. After that, the fourth calculation unit 100d performs a process of displaying the future trajectory of the unmanned aircraft on the screen based on the control model that has been updated with the actual measurement values that are newer than the control model used by the third calculation unit. Is executed after the time point. As a result, after the third arithmetic unit continues to execute the time-series operation at a higher speed than real time until it catches up with a future time point, the future trajectory of the drone is based on the estimated future state of the drone. If the process of displaying the image on the screen is continued after other future time points, the problem that the modeling error of the control model increases can be reduced.

  Further, according to the configuration of (4) above, even when the third calculation unit continues to execute the time series calculation described above at a higher speed than the real time until it catches up with the next future time point, The time-series state of the drone up to the time can be displayed on the screen while estimating the state in time-series in a form that precedes the passage of real time. Therefore, according to the configuration of the above (4), the third calculation unit precedes the real time while continuing to execute the time series calculation described above at a higher speed than the real time until it catches up with the next future time point. Thus, there is no blank time during which control cannot be performed while observing the future state of the drone estimated in time series. As a result, according to the configuration of the above (4), the third calculation unit is unmanned in the near future even while it continues to execute the time-series calculation described above faster than the real time until it catches up with the next future time point. It is possible to detect in advance the possibility that the aircraft will come into contact with an obstacle or sit down.

(5) In some embodiments of the present invention, in the configuration of (3) or (4), at least one of the two or more arithmetic units is
The third calculation unit and the fourth calculation unit are time-sequentially superimposed on a screen that displays the current estimated state of the drone obtained by the first calculation unit or the second calculation unit in time series. The estimated future state of the unmanned aircraft is automatically displayed on the screen in advance of the passage of real time.

  According to the configuration of (5) above, the operator displays on the screen the future state of the drone (future trajectory assumed to be followed by the drone) estimated in time series before the real time. Can be observed. At the same time, the operator uses the same screen as the screen for observing the future state of the drone to determine the current estimated state of the drone that was obtained in time series based on the measured values most recently received from the drone. Remote control of the drone can be performed while visually recognizing. Therefore, according to the configuration of (5) above, using the same screen as the screen used for monitoring the risk that the drone will touch an obstacle or sit at a future time, Remote manual control can be performed while viewing the screen display that displays the current estimated state.

(6) In some embodiments of the present invention, in the configuration of (3) or (4), at least one of the two or more arithmetic units is
Prior to the start of remote control of the drone, an operation for generating an operation profile including a series of operation commands for the drone by repeating trial and error based on simulation;
An unmanned operation of the drone based on the operation profile;
An operation of detecting whether or not a future estimated state of the unmanned aircraft obtained in a time series from the third arithmetic unit to a future time point in real time indicates an avoidance state;
When the avoidance state is detected, the control of the drone is controlled based on a screen display that displays the current estimated state of the drone that is obtained in time series by the first calculation unit and the second calculation unit. , Switching to manual control by the operator,
Configured to perform.

When an operator performs remote control to make an unmanned aircraft perform operations such as passing through a narrow part or continuously avoiding obstacles arranged in a complicated manner, it can fail with a slight operator error. High nature. Therefore, prior to letting the drone perform such complicated and high probability of failure, the operator must perform operation training in advance in a virtual environment in which a situation similar to the actual situation is set, and It is preferable to start the control in order to avoid risk.
Therefore, in the configuration of (6) above, prior to the start of remote control of the drone, an operation scenario when the drone is remotely operated in the virtual environment having the same operation interface as manual remote control is shown. Perform simulation by simulation. In the configuration of the above (6), an operation profile including a series of operation commands for the unmanned aircraft is generated by repeating trial and error by simulating and executing various operation scenarios by the simulation, and the unmanned aircraft is generated according to the operation profile. Is controlled semi-automatically. In addition, during the execution of semi-automatic control according to the above-described operation profile, if it is detected in advance that there is a possibility of contact with an obstacle or the drone being seated, manual remote control by the operator And the semi-automatic (semi-automatic) control can be corrected.
Therefore, according to the configuration of the above (6), the operator is trained in advance using the same operation interface as the manual remote control by repeating trial and error in which various operation scenarios are simulated and executed by simulation. be able to. If an unscheduled failure risk occurs during execution of the semi-automatic (semi-automatic) control, the semi-automatic control can be corrected manually. As a result, according to the configuration of (5) above, it is possible to reduce the risk of failure when the unmanned aircraft is operated with a complicated and high failure probability.

(7) In some embodiments of the present invention, in the configuration of (3) or (4), at least one of the two or more arithmetic units is
An operation of detecting whether or not a future estimated state of the unmanned aircraft obtained in a time series from the third arithmetic unit to a time point in the future from the real time indicates an avoidance state;
An operation of transmitting an emergency operation command to the drone to cause the drone to take an avoidance trajectory in preference to the operation command when the avoidance state is detected;
Is configured to perform further.

  According to the configuration of (7) above, when the state of the drone is estimated to the future point in time and the possibility of the drone coming into contact with the obstacle or the drone sitting in advance is detected, The avoidance control in which the calculation unit of the control device causes the unmanned aircraft to take an avoidance trajectory can be executed automatically, superior to the manual remote control. As a result, when the possibility of contact with an obstacle or the drone sitting in advance is detected in advance, the avoidance control is not left to the operator, but is operated based on the judgment of the calculation unit of the control device. Can be executed automatically without intervention by a person.

  As described above, at least one embodiment according to the present invention provides the control device with the actual measurement value received from the drone when the operator of the control device performs manual remote control of the drone while viewing the image information around the drone. This can be reflected in the drone control operation in consideration of communication delay on the side. Specifically, at least one embodiment according to the present invention can continue to provide feedback for drone control to the operator even while a new measured value cannot be received from the drone due to communication delay. In addition, at least one embodiment according to the present invention, when receiving the latest measured value from the drone, reflects the latest measured value in feedback to the operator in a form that compensates for the update delay of the control model due to communication delay. be able to.

It is a block diagram of the remote control system which concerns on one Embodiment. It is an internal block diagram of each apparatus which comprises the remote control system which concerns on one Embodiment. It is a block diagram corresponding to a part of control system which operate | moves within a control apparatus. It is a timing diagram which illustrates the time-sequential timing on the real time axis of a state estimation process and an image display process. It is a timing diagram which shows a mode that a state estimation process is traced back to measurement time according to reception of a measurement value packet. It is a figure which shows a mode that a state estimation process is traced back to measurement time according to reception of an actual value packet, when time-sequential state estimation is implement | achieved by the Kalman filter. It is a flowchart at the time of performing time-sequential state estimation as a loop process of a Kalman filter. It is a timing diagram which illustrates the time-sequential timing on the real time axis of a state estimation process and an image display process. It is a figure which shows the internal structure of the control apparatus in another embodiment. It is a block diagram of the remote control system which is performing simulation. It is a figure which shows a mode that a remote control system performs remote control of a drone using a simulation result.

Hereinafter, some embodiments of the present invention will be described with reference to the accompanying drawings. However, the dimensions, materials, shapes, relative arrangements, etc. of the components described in the embodiments or shown in the drawings are not intended to limit the scope of the present invention, but are merely illustrative examples. Absent.
For example, an expression indicating that things such as “identical”, “equal”, and “homogeneous” are in an equal state not only represents an exactly equal state, but also has a tolerance or a difference that can provide the same function. It also represents the existing state. On the other hand, the expressions “comprising”, “comprising”, “comprising”, “including”, or “having” one constituent element are not exclusive expressions for excluding the existence of the other constituent elements.
Hereinafter, first, before describing the control device and control method of the drone according to some embodiments, refer to FIG. 1 for an example of a remote control system to which the control device and the control method are applied. To explain. Next, the configuration and operation of the control device and the processing contents of the control method will be described with reference to FIGS.

FIG. 1 shows an example of a remote control system to which a control device and a control method according to some embodiments of the present invention are applied. A remote control system 1 shown in FIG. 1 is a system that may be used for the purpose of manual remote control of an unmanned aircraft while an operator of the control device looks at image information around the unmanned aircraft. 20 is comprised. When the operator inputs an operation command by operating the operation panel 150 connected to the control device 10, the operation command is transmitted by the encoder 130 such as an angle command value θ _in , an angular velocity command value ω _in, or a torque command value q _in. Is transmitted to the drone 20 side. The encoder 130 may be regarded as a command generation unit, and in addition to the operation command of the operation panel 150, the calculation is performed using the actual angle, the actual angular velocity, or the actual torque information, and the angle command value θ _in and the angular velocity command value ω _in. or it may be converted into the command value signal representative of such a torque command value q _in. The above-described angle command value θ _in , angular velocity command value ω _in or torque command value q _in transmitted to the unmanned aircraft 20 as the operation command value drives the joints and traveling wheels of the manipulator provided in the unmanned aircraft 20. You may use as control target values, such as an angle of a drive motor, angular velocity, and a torque. In the unmanned aircraft 20, the command value signal transmitted from the control device 10 is received by the unmanned aircraft control device 21 in the unmanned aircraft 20, and the unmanned aircraft control device 21 receives the servo mechanism 22 based on the command value signal. Drive servo control operation. When the servo control operation is driven by the unmanned aerial vehicle control device 21, the servo mechanism 22 servo-controls actuators such as traveling wheels and manipulators provided in the unmanned aircraft 20.

The servo mechanism 22 is connected to sensors for actually measuring the current state of the drone 20. Specific examples of the sensors include a tachometer that measures the rotational speed of a drive motor that drives a joint and a traveling wheel of a manipulator provided in the unmanned aircraft 20, a camera that captures the surroundings of the unmanned aircraft 20, and the current state of the unmanned aircraft 20. A position measurement sensor or the like for measuring the position may be included. When the unmanned aerial vehicle side control device 21 receives the measured values of the sensors described above from the servo mechanism 22, the actual actual angle θ _out , the actual angular velocity ω _out and the actual torque value q of the driving motor of the unmanned aircraft 20 are determined from the measured values. You may ask for _out etc. Thus, the current actual angle, actual angular velocity, actual torque value, etc. of the drive motor obtained by the unmanned aircraft control device 21 based on the measured values received from the servo mechanism 22 are measured based on the current state of the unmanned vehicle 20. The measured value is transmitted from the drone 20 to the control device 10 as a feedback signal. The above-described actual angle θ _out , actual angular velocity ω _out, actual torque value q _{out and the} like transmitted from the drone 20 to the control device 10 are the angle commands transmitted from the control device 10 to the drone 20 as operation command values. It may be used as a feedback signal for closed loop control with respect to the value θ _in , the angular velocity command value ω _in and the torque command value q _in .

  Next, the detailed configuration of the remote control system 1 will be described together with the internal configurations of the control device 10 and the drone 20 shown in FIG. 1 with reference to FIG. As shown in FIG. 2, the control device 10 and the drone 20 are communicably connected via an optical fiber line 30, and the optical fiber line 30 is further connected to a surveillance camera and other drones 91 and 92. And other control devices and monitoring devices 93 are also connected. The control device 10 includes two or more arithmetic units 100 (100a to 100n), a communication unit 110, a storage unit 120, an encoder 130, and an interface 140, and is connected to an operation panel 150 and a display device 160. Yes. In the control device 10, two or more arithmetic units 100 (100a to 100n), a communication unit 110, a storage unit 120, an encoder 130, and an interface 140 are interconnected by a bus (not shown) so as to communicate with each other. The operation panel 150 and the display device 160 are connected to two or more arithmetic units 100 (100a to 100n), a communication unit 110, a storage unit 120, and an encoder 130 via an interface 140.

The operation panel 150 is an operation panel used by an operator to input an operation input for remotely operating the drone 20 to the control device 10, and includes a joystick, operation buttons, and the like. Display device 160 may be an image display device that can sequentially display the current estimated state of the drone. The communication unit 110 may be a communication interface circuit used for the control device 10 to communicate with other devices via the optical fiber line 30. The communication unit 110 is configured to be able to transmit an operation command input by the operator based on the current estimated state of the drone 20 displayed on the screen of the display device 160 to remotely control the drone 20 to the drone 20. ing. Furthermore, the communication unit 110 is configured to be able to receive from the unmanned aircraft 20 the measured values obtained by the drone 20 sequentially measuring the state of the own device. The encoder 130 uses the actual angle, the actual angular velocity, or the actual torque information to convert the operation command input by the operator by operating the operation panel 150 into the angle command value θ _in , the angular velocity command value ω _in, or the torque command. It is converted into a command value signal representing the value q _{in and the} like and output to the communication unit 110 and the calculation unit 100.

  The two or more calculation units 100 (100a to 100n) include a first calculation unit 100a and a second calculation unit 100b. As will be described later with reference to FIGS. 3 to 5, the first calculation unit 100 a and the second calculation unit 100 b are unmanned aircraft obtained by executing time-series calculations for obtaining the current estimated state of the drone 20. The 20 current estimated states can be provided to the display device 160 (can be displayed on the screen). At that time, the current estimated state of the drone 20 is based on a control model that reflects the actual measurement values received from the drone 20 by the first calculation unit 100a and the second calculation unit 100b, respectively. It is determined by estimating the current state. The process of displaying the estimated state of the drone 20 obtained in time series by the first calculation unit 100a and the second calculation unit 100b on the display device 160 may be executed as follows. First, the environment around the drone 20 is imaged based on the captured image and measurement information from the sensors / cameras 221 provided in the drone 20. Subsequently, the current position of the drone 20, the relative positional relationship between the drone 20 and other structures, the attitude of the drone 20 (pitch angle, roll angle, yaw angle) and the drone 20 are provided. The attitude of the manipulator is calculated from the estimated state of the drone 20 obtained by the first calculation unit 100a and the second calculation unit 100b in time series. Finally, a background image obtained by imaging the environment around the drone 20 is displayed on the screen of the display device 160, and the image of the drone 20 taking the posture calculated as described above is displayed as the current position of the drone 20. Is displayed at a position in the background image corresponding to. In an exemplary embodiment, two or more arithmetic units 100 (100a to 100n) may be realized as two or more separate arithmetic circuits, and each arithmetic circuit may be a general-purpose processor, a microcontroller, or a A digital signal processor or the like may be used. In an alternative embodiment, each processor core included in a single multi-core CPU may be operated as each of two or more arithmetic units 100 (100a to 100n).

  The drone 20 includes a drone-side control device 21 and a servo mechanism 22, and the servo mechanism 22 is connected to sensors / cameras 221 and actuators 222 provided in the drone 20. . The unmanned aircraft control device 21 is connected to the servo mechanism 22 and includes a communication unit 211 on the unmanned aircraft side and a control calculation unit 212 on the unmanned aircraft side. Specific examples of the sensors 221 include a tachometer that measures the rotation speed of a drive motor that drives a joint and a traveling wheel of a manipulator provided in the drone 20, a camera that captures the surroundings of the drone 20, and the current state of the drone 20. A position measurement sensor or the like for measuring the position may be included. Specific examples of the actuators 222 may include a joint of a manipulator provided in the drone 20 and a drive motor that drives a traveling wheel. The communication unit 211 on the unmanned aircraft side may be a communication interface circuit for communicating with the control device 10 via the optical fiber line 30. In the unmanned aircraft control device 21, when the unmanned aircraft control calculation unit 212 receives an operation command value from the control device 10 via the unmanned vehicle communication unit 211, the servo mechanism 22 is based on the operation command value. A control command is given to. The servo mechanism 22 together with the sensors / cameras 221 and the actuators 222 forms part of a closed loop control system for performing servo control of manipulators and traveling wheels provided in the unmanned aircraft 20, and the unmanned aircraft side control device 21. The servo control may be executed based on the control command from.

  2, in addition to the control device 10 and the drone 20, the optical fiber line 30 includes a large number of devices including a monitoring camera, other unmanned vehicles 91 and 92, other control devices and a monitoring device 93. The communication band and communication resources of the optical fiber line 30 are shared by these many devices. Accordingly, communication between the control device 10 and the drone 20 may be delayed by using a communication band on the optical fiber line 30 by another device. In this case, the control device 10 and the drone 20 Communication delay may occur between the two. Due to the communication delay, even if the drone 20 transmits the actual measurement value M obtained by actually measuring the state of the drone 20 to the control device 10 side, the drone 20 is received with a delay on the control device 10 side. The transmitted operation command is received with a delay at the drone 20 side. In an environment where the communication delay cannot be ignored, the drone 20 is forced to perform a control operation close to inching in actual operation, although it can move at high speed in terms of specifications. As a result, work performed by remotely controlling the drone in such an environment is very inefficient, resulting in work delays, and conversely, the operator predicts the action of the drone 20 by itself and operates at high speed. Attempting to do so will result in operator error.

  FIG. 3 is a block diagram corresponding to a part of a control system that operates in the control apparatus 10 to remotely control the drone 20. The block diagram of FIG. 3 shows the current estimated state of the unmanned aerial vehicle 20 obtained by the first computing unit 100a and the second computing unit 100b performing time-series computations for obtaining the current estimated state of the unmanned aircraft 20. The flow of the information signal until it is provided to the display device 160 is shown. In FIG. 3, the first calculation unit 100a and the second calculation unit 100b among the two or more calculation units shown in FIG. 2 are a first control model 401, a second control model 402, and a switching unit generated as functional modules. 411 and switching means 412 are executed. These functional modules may be, for example, functional modules generated when the first arithmetic unit 100a and the second arithmetic unit 100b read and execute dedicated software from the storage unit 120. In an exemplary embodiment, the first control model 401 and the second control model 402 may be executed separately by the first calculation unit 100a and the second calculation unit 100b. In that case, as will be described later with reference to FIGS. 4 and 5, the first calculation unit 100 a performs time-series state estimation processing based on the first control model 401, and the first calculation unit 100 b Based on the second control model 402, time-series state estimation processing is executed.

  A first control model 401 and a second control model 402 shown in FIG. 3 are functional modules that output an estimated output signal of the drone 20 when an operation command for the drone 20 is given as an input. The first control model 401 and the second control model 402 describe response characteristics (behavioral characteristics) of the unmanned aerial vehicle 20 observed based on actual measurement values fed back from the unmanned aerial vehicle 20 as mathematical formula models. Based on this, an estimated output signal of the drone 20 is output from an operation command to the drone 20. In an exemplary embodiment, the mathematical model may be described as a state estimation filter that includes a Kalman filter. The switching unit 411 outputs the actual measurement value received from the drone 20 via the communication unit 110 as the actual measurement value obtained by actually measuring the current state of the drone 20 to the first control model 401 side. It is a functional module that switches whether to output to the side. The switching unit 412 is a functional module that switches whether an output signal to be displayed on the display device 160 is output from the first control model 401 or the second control model 402.

In FIG. 3, the operation command and the actual angle, actual angular velocity, or actual torque information input from the operation panel 150 are operated by the encoder 130 to represent the angle command value θ _in , the angular velocity command value ω _{in, the} torque command value q _in, or the like. It is converted into a command value signal and output to the communication unit 110 and the calculation unit 100. The operation command value signal from the encoder is input to the first control model 401 and the second control model 402 via the signal paths 42 and 44, and as a result, the estimated angle θ ′ representing the current estimated state of the drone 20. An estimated state signal such as estimated angular velocity ω ′ or estimated torque q ′ is output. Subsequently, the estimated state signal reaches the selector switch 52 via the signal paths 43 and 45, and the first control model 401 or the second control model 402 of the first control model 402 depends on the switching direction of the selector switch 52 by the switching means 412. Either one is output to the display device 160. The estimated state signal (such as the estimated angle θ ′, the estimated angular velocity ω ′, or the estimated torque q ′) indicates the current position of the drone 20, the current posture of the drone 20, the current state of the manipulator included in the drone 20, and the like. Is displayed on the screen of the display device 160.

  The measured value signal received from the drone 20 by the communication unit 110 as the measured value of the current state of the drone 20 depends on the first control model 401 or the first control model 401 according to the switching direction of the changeover switch 51 by the switching unit 411. It is output to either one of the two control models 402. When the first control model 401 or the second control model 402 receives an actual measurement value from the drone 20, the first control model 401 or the second control model 402 performs a process of applying the actual measurement value to a mathematical model representing the response characteristic (behavior characteristic) of the drone 20. As a result, the actual measurement value from the drone 20 is reflected in the first control model 401 or the second control model 402.

  From the above, the outline of the signal processing operation executed by the first arithmetic unit 100a and the second arithmetic unit 100b from the information signal flow shown in FIG. 3 will be described as follows. That is, when the first calculation unit 100a and the second calculation unit 100b each receive the actual measurement value received from the drone 20 from the communication unit 110, the first calculation unit 100a and the second calculation unit 100b reflect the actual measurement value in the first control model 401 or the second control model 402. . In parallel with this, when receiving the operation command output from the encoder 130, the first calculation unit 100a and the second calculation unit 100b are based on the first control model 401 and the second control model 402 reflecting the actual measurement values. The current estimated state of the drone 20 is output with the operation command as an input. Subsequently, the first calculation unit 100a and the second calculation unit 100b display an image representing a result of estimating the current state of the drone 20 based on the estimated state output by the first control model 401 or the second control model 402. The screen is displayed by the display device 160.

Next, in the control system shown in the block diagram of FIG. 3, the first calculation unit 100a and the second calculation unit 100b perform unattended calculation by performing time-series calculation for determining the current estimated state of the drone 20. A series of operation flows for outputting the current estimated state of the machine 20 to the display device 160 will be described with reference to FIG. FIG. 4 shows a time series on the real time axis of a process for obtaining the current estimated state of the drone 20 in time series and a process for displaying the current estimated state of the drone 20 obtained in time series on the screen. The execution timing is shown. In FIG. 4, each time the first calculation unit 100 a and the second calculation unit 100 b alternately (alternatively) receive the actual measurement value from the drone 20 and receive the actual measurement value, A state in which the above-described time series calculation for obtaining the estimated state in time series is repeatedly executed is shown. Specifically, at times T ₁ , T _3, and T ₅ shown in FIG. 4, the first calculation unit 100a receives actual measurement values from the drone 20 and applies the measured values to the first control model 401 shown in FIG. An operation to reflect the received actual measurement value is performed. Note that at the times T ₁ , T _3, and T ₅ , the changeover switch 51 in FIG. 3 is switched to the signal path 46 connected to the first control model 401. On the other hand, at times T ₂ , T ₄ and T ₆ shown in FIG. 4, the second arithmetic unit 100b receives the actual measurement value from the drone 20, and receives the actual measurement received from the second control model 402 shown in FIG. Perform an action that reflects the value. At times T ₂ , T _4, and T ₆ , the changeover switch 51 in FIG. 3 is switched to the signal path 47 connected to the second control model 402.

  Further, in FIG. 4, the first calculation unit 100 a described above for obtaining the current estimated state of the drone 20 based on the new first control model 401 ′ reflecting the one actually measured value alternately received. A state in which the time-series operations are sequentially executed is shown. At this time, the first calculation unit 100a executes the above-described time-series calculation from the actual measurement time of the one actual measurement value to one time that catches up with the current time, faster than the passage of real time. Further, in FIG. 4, the second calculation unit 100 b described above for obtaining the current estimated state of the drone 20 based on the new second control model 402 ′ reflecting other measured values received alternately. A state in which the time-series operations are sequentially executed is shown. At that time, the second calculation unit 100b executes the above-described time-series calculation from the actual measurement time of the other actual measurement value to another time that catches up with the current time at a higher speed than the passage of real time.

Specifically, the first arithmetic unit 100a is based on a first control model 401 ′ that reflects the actual measurement values M ₁ , M _3, and M ₅ received at times T ₁ , T _3, and T ₅ shown in FIG. Thus, time series operations for obtaining the current estimated state of the drone 20 are sequentially executed. At that time, the first calculation unit 100a is first of all, the measured value _{M 1} received at time _{T 1} by reflecting the first control model 401 and the new first control model 401 '. Subsequently, the first calculation unit 100a goes back to the actual measurement time T _m ^1, which is the time when the actual measurement value M ₁ received at the time T ₁ is actually measured by the drone 20. Subsequently, the first calculation unit 100a executes the above-described time-series calculation from the actual measurement time T _m ¹ to the time t ₁ (corresponding to the period represented by p1 in FIG. 4) at a higher speed than the passage of real time. . Here, the time t ₁ is series operations when executed on the high-speed by the first calculation unit 100a corresponds to the time to catch up to the current time Tc. Further, the time difference between the reception time T ₁ when the actual measurement value M ₁ is received and the actual measurement time T _m ¹ corresponds to a communication delay between the drone 20 and the control device 10.

Similarly, the first calculation unit 100a is, for the measured values _{M 3} and _{M 5} respectively received at time _{T 3} and _{T 5,} faster than the course of the measured time _T ^{m 3} and _T ^m real time back to ⁵ The above time-series operations are respectively performed, and the time-series operations executed at high speed are continued until the times t ₃ and t ₅ (corresponding to the periods represented by p 3 and p _{5 in} FIG. 4) that catch up with the current time Tc. Here, the time difference between the reception time T ₃ when the actual measurement value M ₃ is received and the actual measurement time T _m ³ and the time difference between the reception time T ₅ when the actual measurement value M ₅ is received and the actual measurement time T _m ⁵ are: This corresponds to a communication delay between the drone 20 and the control device 10, respectively.

On the other hand, the second calculation unit 100b determines the current estimated state of the drone 20 based on the second control model 402 ′ reflecting the actual measurement values received at the times T ₂ , T ₄ and T ₆ shown in FIG. Time series operations for obtaining are sequentially performed. At that time, the second arithmetic unit 100b includes, first, a measured value _{M 2} received in time _{T 2,} by reflecting the second control model 402 and the new second control model 402 '. Subsequently, the second calculation unit 100b goes back to the actual measurement time T _m ^2, which is the time when the actual measurement value M ₂ received at the time T ₂ is actually measured by the drone 20. Subsequently, the second calculation unit 100b is a series operations when the aforementioned, from the measured time T _m ² to time t ₂ (corresponding to the period represented by p2 in FIG. 4), to run faster than the elapsed real time . Here, the time t ₂ is series operations when executed on the high-speed by the second calculation unit 100b corresponds to the time to catch up to the current time Tc. Further, the time difference between the reception time T _{2 at} which the actual measurement value M ₂ is received and the actual measurement time T _m ² corresponds to a communication delay between the drone 20 and the control device 10.

Similarly, the second calculation unit 100b is, for the measured values respectively received at time T ₄ and T _6, the measured time T _m ⁴ and T _m the time-series operation faster than the elapsed real-time back to ⁶ each run, until the time t ₄ and t ₆ series operations when executed on the high-speed catches up with the current time Tc (corresponding to the period represented by p4 and p6 in Fig. 4) is continued, respectively. Here, the time difference between the receipt time T ₆ which has received the time difference and the measured value M ₆ and the measured time T _m ⁶ between the receiving time T _4, which has received the measured value M ₄ and the measured time T _m ^4, This corresponds to a communication delay between the drone 20 and the control device 10, respectively. In the following description, it is assumed that each index value k (1 ≦ k ≦ K) has a one-to-one correspondence with each time sequentially arranged on the time axis.

In the state estimation process described above with reference to FIGS. 3 and 4, when a new actual measurement value M _k (1 ≦ k ≦ K) is received from the drone 20, the first calculation unit 100 a and the second calculation unit 100 b The following processing operations are performed. That is, a new first control model 401 ′ or second control model 402 ′ is reflected by reflecting the new actual measurement value M _k (1 ≦ k ≦ K) in the existing first control model 401 or second control model 402. obtain. As a result, the first calculation unit 100a or the second calculation unit 100b can improve the modeling error of the control model by the new actual measurement value M _k (1 ≦ k ≦ K). However, for that purpose, the update of the first control model 401 or the second control model 402 caused by the communication delay from the actual measurement time T _m ^{k of the} new actual measurement value M _k (1 ≦ k ≦ K) to the current time Tc. It is necessary to compensate for the delay. Therefore, in the state estimation process described above with reference to FIGS. 3 and 4, the current estimated state of the drone 20 is time-sequentially based on the new first control model 401 ′ or the second control model 402 ′. The obtained time series calculation is executed at a speed higher than the lapse of the real time until the current time Tc is caught up from the actual measurement time T _m ^k .

Accordingly, after reflecting the newly received actual measurement value in the control model, the time series operation for obtaining the current estimated state of the unmanned aircraft 20 in time series catches up with the current time Tc until the current time elapses. It can be said that the control model immediately after execution at a higher speed is a control model that has been updated by the new actual measurement value. For example, the first control model 401 ″ in which the update operation using the actual measurement value M _k is completed is obtained as follows. First, the new first control model is reflected by reflecting the actual measurement value M _k in the first control model 401. After that, the above-described time-series operation based on the new first control model 401 ′ is executed at a speed higher than the elapse of real time until it catches up with the current time Tc. It can be said that the first control model 401 ″ immediately after the execution of is performed at a speed higher than the elapse of the real time is the first control model 401 ″ in which the update operation using the actual measurement value _Mk is completed.

Next, with reference to FIG. 4, a process for displaying the estimated state of the drone 20 obtained in time series on the screen by the first calculation unit 100 a and the second calculation unit 100 b will be described from a specific example. As described with reference to FIG. 4, the first calculation unit 100 a updates the first control model 401 with the actual measurement value M ₁ , starting from the actual measurement time T _m ¹ when the actual measurement value M ₁ was actually measured in the drone 20. Therefore, the above time series operation is continued until time t ₁ when the high speed execution catches up with the current time Tc. Also, starting from the actual measurement time T _m ² at which the actual measurement value M ₂ was actually measured by the drone 20, the second calculation unit 100b performs the above-described time series calculation for updating the second control model 402 with the actual measurement value M ₂ . run faster than in real time, the fast execution continues to series operation when the above until time t ₂ to catch up to the current time Tc. In this case, during period from the time t ₁ to time t _2, the second arithmetic unit 100b continues waiting for measured value M ₂ is received until time T _2, measured value M ₂ is received Then, the time series calculation is executed at a speed higher than the actual time from the actual measurement time T _m ² to the time t ₂ . Therefore, during the period from time T ₂ to time t 2, the second arithmetic unit 100 b determines the unmanned aerial vehicle 20 obtained in time series based on the second control model 402 in accordance with the passage of real time and the pace. The current estimated state of cannot be displayed on the screen. The reason is as follows. In order to display the time series of the estimated state of the unmanned aerial vehicle 20 calculated by the above time series calculation calculated at a higher speed than the real time on the screen in real time, a part of the time series of the estimated state It is necessary to display the screen in the form of thinning out only the estimated state. However, when the screen is displayed in such a manner that only a part of the estimated states is thinned out from the time series of the estimated states, the movement of the drone 20 on the screen becomes a discontinuous movement without smoothness. Further, in parallel with the process of executing the time series calculation for obtaining the estimated state of the drone 20 in time series at a speed higher than the real time, the estimated state of the drone 20 obtained in time series is sequentially displayed on the screen. This is because the displaying process may put an excessive load on the arithmetic unit 100. Therefore, during the period from time t ₁ to time t ₂ , the first calculation unit 100a is based on the first control model 401 ″ in which the update operation using the actual measurement value M ₁ received at time T ₁ is completed. An operation for displaying the current estimated state of the drone 20 obtained in time series on the screen is executed, and the selector switch 52 in FIG. 3 switches to the signal path 43 side connected to the first control model 401. It has been.

Also, starting from the actual measurement time T _m ² at which the actual measurement value M ₂ was actually measured by the drone 20, the second calculation unit 100b performs the above-described time series calculation for updating the second control model 402 with the actual measurement value M ₂ . run faster, the fast execution continues to series operation when the above until time t ₂ to catch up to the current time Tc. The first calculating unit 100a the measured time T _m ³ where measured values M ₃ is measured in the drone 20 as origin, the time series operation for updating the first control model 401 by Found M ₃ run faster than in real time, the fast execution continues to series operation when the above until time t ₃ when catch up to the current time Tc. In this case, during period from the time t ₂ to time t _3, the first calculation unit 100a continues waits for the measured value M ₃ is received until time T _3, the measured value M ₃ is received Then, the time series calculation is executed at a speed higher than the actual time from the actual measurement time T _m ³ to the time t ₃ . Accordingly, during the period from time T ₃ to time t ₃ , the first calculation unit 100a is a drone that is obtained in time series based on the first control model 401 in accordance with the passage of real time and the pace. 20 current estimated states cannot be displayed on the screen. Therefore, during the period from time t ₂ to time t ₃ , the second calculation unit 100b is based on the second control model 402 ″ in which the update operation using the actual measurement value M ₂ received at time T ₂ has been completed. An operation for displaying the current estimated state of the drone 20 obtained in time series on the screen is performed, and at this time, the changeover switch 52 in FIG. 3 switches to the signal path 45 side connected to the second control model 402. It has been.

The above description related to the screen display processing with reference to FIG. 4 is generalized as follows. Here, for convenience of explanation, when the measured value M _k (1 ≦ k ≦ K) is described above by the first calculation unit 100a from the actual measurement time T _m ^k (1 ≦ k ≦ K) measured in the drone 20 as described above. The time t _k (1 ≦ k ≦ K) at which the sequential calculation is executed at high speed to catch up with the current time Tc is referred to as one time. In addition, the actual measurement value M _{k + 1} indicates the time t _{k + 1 at} which the above-described time series calculation is performed at high speed by the first calculation unit 100a from the actual measurement time T _m ^{k + 1} actually measured in the drone 20, and catches up with the current time Tc. I will call it another time. In that case, the first calculation unit 100a provides the current estimated state of the drone to the display device 160 for screen display from one time (time t _k ) to the next other time (time t _{k + 1} ). Configured to do. Also, assuming that one time that comes next to the one time (time t _k ) is time t _{k + 2} , the second calculation unit 100b has the next time (time t _{k + 1} ) from another time (time t _{k + 1} ). _{up to k + 2} ), the current estimated state of the drone 20 is provided to the display device 160 for screen display.

As a result, as shown in the lower part of FIG. 4, during the period represented by q1 and p2 between the time t ₁ to time t _2, the first calculation unit 100a may update work by Found M ₁ determined time series of the estimated state of the drone 20 based on the first control model 401 "has been completed, the time-series manner obtained estimated state is displayed on the screen in the display device 160. in addition, from time t ₂ during the period represented by q2 and p3 between up to t _3, the second calculation unit 100b is, the estimated state of the drone 20 based on the second control model 402 "which updating actually measured value M ₂ is completed determined chronological order, the time-series manner determined estimated state is displayed on the screen in the display device 160, during the period represented by q3 and p4 between the time t ₃ to time t _4, the first Calculation unit 100a , Chronologically determined the estimated state of the drone 20 based on the first control model 401 "which updating by the measured value M ₃ is completed, the time-series manner determined estimated state is displayed on the screen in the display device 160 Similarly, the first calculation unit 100a and the second calculation unit 100b alternately alternate (alternatively) the current estimated state of the drone 20 obtained in time series. 160 is displayed on the screen.

By the way, when either one of the first calculation unit 100a or the second calculation unit 100b receives the actual measurement value from the drone 20, the process of obtaining the estimated state of the drone 20 in time series is performed by using the actual calculation value. This is executed retroactively to the actual measurement time actually measured on the 20 side. Therefore, the time for which the estimated state of the drone 20 is obtained may correspond to a time point retroactive (retroactive) with respect to the passage of real time when viewed from the current time. Therefore, in the following description, since the time for which the estimated state of the drone 20 is obtained is different from the real time in the real time, a virtualized time ( It is assumed to be abstracted as virtual time in (virtual time). 5, actually measured the state estimation process chronologically obtaining the current estimated state of the drone 20 according to the received measured value M _k from the drone 20 ₍₁ ≦ k ≦ K) time T _{m k} ⁽¹ It is a figure which shows a mode that it can go back to (<= k <= K). In FIG. 5, the transition of the virtual time when the first arithmetic unit 100a obtains the current estimated state of the drone 20 in time series based on the first control model 401 is shown as the virtual time J1. In FIG. 5, the transition of the virtual time when the second calculation unit 100b obtains the current estimated state of the drone 20 in time series based on the second control model 402 is shown as the virtual time J2. . The time _{T 1,} T 2, _{T 3} shown in FIG. 5, ... and time _{t 1,} t 2, _{t 3,} ..., the time _{T 1,} T 2, _{T 3} shown in FIG. 4, ... and time _{t 1} , T ₂ , t ₃ ,...

In FIG. 5, the operation of the first calculation unit 100a will be described in order along the virtual time J1 as follows. The first calculation unit 100a, based on the first control model 401 reflecting the actually measured value M ₁ received at time T ₁ shown in FIG. 5, the time series operation for obtaining the current estimated state of the drone 20 Run. At that time, the first calculation unit 100a is first of all, the measured value _{M 1} received at time _{T 1} by reflecting the first control model 401 and the new first control model 401 '. Subsequently, the first calculating unit 100a includes a series operation when the above-measured value M ₁ received at time T ₁ is to retroactively until the measured time T _m ¹ is the time that is actually measured in the drone 20. At this time, the actual measurement time T _m ¹ is a time retroactive to the actual time by the time width indicated by τ ₁₁ in FIG. 5 from the reception time T ₁ at which the actual measurement value M ₁ was received. Subsequently, the first calculation unit 100a executes the above-described time-series calculation at a speed higher than the lapse of real time, and performs the high-speed execution only from the actual measurement time T _m ¹ to the time t ₁ (in FIG. 5). τ only during the period represented by ₁₂₎ continue to run. Here, the time t ₁ is series operations when executed faster than the elapsed real time in virtual time J1 is a time to catch up to the current time Tc. Similarly, the first calculation unit 100a is, for the measured values _{M 3} received at time _{T 3,} the measured time _T ^m back to ³ the time-sequential faster than the elapsed real time (tau ₃₁ shown in FIG. 5) The high-speed execution is continued until time t ₃ when the current time Tc catches up (τ _{32 in} FIG. 5).

In FIG. 5, the operation of the second arithmetic unit 100b will be described in order along the virtual time J2 as follows. The second calculation unit 100b, based on the second control model 402 reflecting the actually measured value M ₂ received in time T _2, as shown in FIG. 5, the time series operation for obtaining the current estimated state of the drone 20 Run. At that time, the second arithmetic unit 100b includes, first, a measured value _{M 2} received in time _{T 2,} by reflecting the second control model 402 and the new second control model 402 '. Subsequently, the second calculation unit 100b is a series operations when the above-measured value M ₂ received in time T _2, is thereby retroactively until the measured time T _m ² is the time that is actually measured in the drone 20. At this time, the actual measurement time T _m ² is a time retroactively from the reception time T ₂ at which the actual measurement value M ₂ was received in a manner that reverses the actual time by the time width indicated by τ ₂₁ in FIG. Subsequently, the second calculation unit 100b is a series operations when mentioned above, run faster than the elapsed real time, in only (FIG. 5 between the high-speed run from the measured time T _m ² to time t ₂ τ only during the period represented by ₂₂₎ continue to run. Here, the time t ₂ is series operations when executed faster than the elapsed real time in virtual time J2 is a time to catch up to the current time Tc. Similarly, the second calculation unit 100b also goes back to the actual measurement time T _m ⁴ (τ ₄₁ shown in FIG. 5) with respect to the actual measurement value M ₄ received at time T ₄ faster than the actual time. The high-speed execution is continued until time t ₄ when the current time Tc catches up (τ _{42 in} FIG. 5).

3 to the embodiment described above using Figure 5 receives a new measured value M _k from the drone 20, new measured values M _k existing first control model 401 or the second control model 402 As a result, a new first control model 401 ′ or second control model 402 ′ is obtained. Therefore, in this embodiment, the modeling error of the control model can be improved by the new actual measurement value _Mk . However, to do this, it is necessary to compensate for the new actual value update delay control model generated by communication delay from the measured time T _m ^k of M _k to the current time Tc. Therefore, in this embodiment, an operation for obtaining the current estimated state of the drone 20 in time series based on the new first control model 401 ′ or the second control model 402 ′ is performed as the actual measurement time T _m ^k. From the beginning until the current time Tc is caught up, execution is performed at a speed higher than the passage of real time.

As described above, according to this embodiment, the latest measured value M _k is _obtained while giving feedback for the drone control to the operator even when the new measured value M _k cannot be received from the drone 20 due to communication delay. When received from the drone 20, the latest measured value _Mk may be reflected in the feedback to the operator in a form that compensates for the update delay of the first control model 401 and the second control model 402 caused by the communication delay. it can. Therefore, according to this embodiment, when the operator of the control device 10 performs manual remote control of the drone 20 while viewing the image information around the drone, the measured value M _k received from the drone 20 is communicated. In consideration of the delay, it can be reflected in the unmanned aircraft control operation on the control device 10 side.

Further, in this embodiment, even if the control device 10 receives the latest measured value M _k representing the most recent state of the drone 20 with a delay, the measured value received before until the delay is received. Based on the most recent first control model 401 or second control model 402 reflecting M _k−1, the current state of the drone 20 can be sequentially displayed on the screen while estimating the state in time series. Accordingly, the current state of the drone 20 to be sequentially displayed on the screen until the control device 10 receives the latest actual measurement value M _k with a delay until the previous actual measurement value M _k−1 is reflected. Based on the first control model 401 or the second control model 402, state estimation can be continued in time series. Accordingly, there is no blank time during which control cannot be performed while viewing the current state of the drone 20 until the latest measured value _Mk is received by the control device 10 with a delay. The operator can perform manual remote control of the drone 20 without being aware of the communication delay.

Furthermore, in this embodiment, after the latest measured value _Mk representing the most recent state of the drone 20 is received with a delay, the first control model 401 ′ reflecting the latest measured value _Mk is received. Alternatively, based on the second control model 402 ′, the current state of the drone 20 can be estimated and displayed on the screen. Accordingly, when the operator performs manual remote control of the drone 20 while viewing the current state of the drone 20 estimated based on the control model on the screen of the display device 160, the first over time. It is possible to estimate the current state of the drone 20 without accumulating modeling errors of the control model 401 and the second control model 402.

  In an exemplary embodiment, the first control model 401 and the second control model 402 described above with reference to FIGS. 3-5 may be configured to include a state estimation filter. In this embodiment, the time series calculation for the first control model 401 and the second control model 402 to obtain the current estimated state of the drone 20 in a time series can be executed as follows. That is, the state estimation filter can execute the time-series operation in the time domain as a simple iterative loop process in which each loop process corresponds to each time on the time axis. Note that each time sequentially arranged on the time axis has a one-to-one correspondence with each index value k (1 ≦ k ≦ K). Therefore, each loop process repeatedly executed by the state estimation filter also has each index value k ( 1 ≦ k ≦ K). Therefore, in this embodiment, when realizing the process of estimating the current state of the drone 20 in time series based on the first control model 401 and the second control model 402, each time domain described above Each time the loop process ends, the current state of the drone 20 can be directly obtained from the state variable calculated in each loop process. As a result, in the process of estimating the current state of the drone 20, the process of converting the internal variable values of the first control model 401 and the second control model 402 into the current state of the drone 20 becomes unnecessary, and the calculation is performed. The speed can be increased.

Hereinafter, in order to realize the processing described above with reference to FIGS. 3 to 5 as time series operations for obtaining the current estimated state of the drone 20 in time series, the first operation unit 100a and the second operation are performed. First, the flow of operations performed by the unit 100b will be described from a specific example. In this embodiment, starting from the measured time T _m ^k measured value M _k newly received, to run faster than real-time status prediction operation of the state estimation filter based on the measured values M _k newly received . This state prediction operation is an operation for performing time-series state estimation at a higher speed than the real time by going back to the actual measurement time T _m ^k that is the starting point of the state estimation on the virtual time axis. This is called a retrospective state prediction operation. Retrospective state prediction operation, retrospectively was measured time T _m and posterior predictive operation to reflect the actual value M _k to the control model in ^k, measured on the basis of a new control model measured value Mk is reflected time T _It consists of a high-speed state prediction operation that executes time-series state estimation at a speed higher than real time from catching up to the current time Tc from _m ^k . Newly received measured values M _k on posterior predictive operation performed by the state estimation filter based on the new to reflect the actual value M _k newly received to the first control model 401 or the second control model 402 first This corresponds to the operation of obtaining the first control model 401 ′ or the second control model 402 ′. The high-speed state prediction operation performed by the state estimation filter is a time series in which the current estimated state of the drone 20 is obtained in time series based on the new first control model 401 ′ or the new second control model 402 ′. It corresponds to the processing that executes the arithmetic operation faster than the real time. When the fast state prediction operation executed at a speed higher than the real time catches up with the current time Tc, the state prediction operation of the state estimation filter is switched to the prior prediction operation and continued as it is. The prior prediction operation executes state estimation processing that is substantially the same as the high-speed state prediction operation, except that the prior prediction operation is executed in synchronism with the passage of real time while displaying time-series state estimation results on the screen. The pre-prediction operation executed by the state estimation filter is performed after the first control model 401 or the second control model 402 implemented as the state estimation filter is updated with newly received actual values (updated first Start) after the control model 401 "or the updated second control model 402" is obtained. That is, the state estimation filter calculates and executes the high-speed state prediction operation at a speed higher than the real time until it catches up with the current time Tc. A time series calculation for obtaining the state in time series is executed in accordance with the passage of real time and the pace. As a result, the estimated state of the drone 20 output by the prior prediction operation is displayed on the screen on the display device 160.

で表し、時刻ｋ（１≦ｋ≦Ｋ）における入力をｕ（ｋ）、時刻ｋ（１≦ｋ≦Ｋ）における観測雑音をｗ（ｋ）、システム雑音をｖ（ｋ）、システム雑音の分散をσ_ｖ ^２と表した場合、当該制御系システムの状態方程式、観測方程式および共分散は以下のように表される。

なお、上記式（３）によって求められる共分散は、カルマンフィルタが現在時刻ｋにおける無人機２０の状態ｘ（ｋ）と状態観測値Ｙ（ｋ）＝［ｙ_０，…，ｙ_ｋ］との関係から未来の時刻ｋ＋１の状態推定値を求める予測動作モードで動作する際に使用する共分散である。 FIG. 6 shows a state in which the state estimation process is traced back to the actual measurement time in response to reception of the actual measurement packet when the state estimation filter is configured as a Kalman filter and the time series state estimation of the drone 20 is executed by the Kalman filter. Show. Note that each time sequentially arranged on the time axis has a one-to-one correspondence with each index value k (1 ≦ k ≦ K). Therefore, in the following description, for simplicity, an index on the time axis is used. The time corresponding to k is referred to as time k. The first control model 401 and the second control model 402 that model the control response characteristics of the drone 20 are regarded as independent control system systems, and the estimated state of the drone 20 at time k (1 ≦ k ≦ K) is determined.

The input at time k (1 ≦ k ≦ K) is u (k), the observation noise at time k (1 ≦ k ≦ K) is w (k), the system noise is v (k), and the system noise variance Is expressed as σ _v ² , the state equation, observation equation and covariance of the control system are expressed as follows.

The covariance obtained by the above equation (3) is the relationship between the state x (k) of the drone 20 at the current time k by the Kalman filter and the state observation value Y (k) = [y ₀ ,..., Y _k ]. The covariance used when operating in the predictive operation mode for obtaining the estimated state value at the future time k + 1 from.

  In FIG. 6, the left side of the vertical axis corresponding to the time axis represents a state in which an actual measurement value obtained by actually measuring the state of the drone 20 changes in time series on the drone 20 side as the real time elapses. In FIG. 6, the right side of the vertical axis corresponding to the time axis shows the estimated state of the drone 20 based on the state estimation filter when the first arithmetic unit 100a and the second arithmetic unit 100b progress as the virtual time elapses. Represents how it is obtained serially.

が、実時間の経過に伴う無人機２０側の実測値の時系列的推移として示されている。図６に示す縦軸の右側では、状態推定フィルタが実行する事前予測動作により、仮想時間軸上の時刻ｋ−４、時刻ｋ−３、時刻ｋ−２、時刻ｋ−１および時刻ｋにおいて無人機２０の時系列的な推定状態を以下のように求めている。

上記状態推定フィルタが実行する上記事前予測動作は、時刻ｋ−４、時刻ｋ−３、時刻ｋ−２、時刻ｋ−１および時刻ｋにそれぞれ対応する時系列的なループ処理として実行され、直前の時刻ｈ−１における無人機２０の推定状態

を上記式（１）の状態方程式に適用することにより、時刻ｋにおける無人機２０の推定状態

を求めるものである。つまり、上記状態推定フィルタが実行する上記事前予測動作は、以前に受信した実測値Ｍ_ｋ−１による更新作業が完了した第１制御モデル４０１”または第２制御モデル４０２”に基づいて無人機２０の推定状態を時系列的に求める時系列的演算に相当する処理である。 On the left side of the vertical axis shown in FIG. 6, the real state of the drone 20 at time k-4, time k-3, time k-2, time k-1 and time k on the real time axis.

Is shown as a time-series transition of actual measurement values on the unmanned aerial vehicle 20 side with the passage of real time. On the right side of the vertical axis shown in FIG. 6, unattended at time k-4, time k-3, time k-2, time k-1 and time k on the virtual time axis by the prior prediction operation executed by the state estimation filter The time-series estimated state of the machine 20 is obtained as follows.

The prior prediction operation executed by the state estimation filter is executed as a time-series loop process corresponding to each of time k-4, time k-3, time k-2, time k-1, and time k. Estimated state of drone 20 at time h-1

Is applied to the state equation of the above equation (1), so that the estimated state of the drone 20 at the time k

Is what you want. In other words, the prior prediction operation performed by the state estimation filter is based on the first control model 401 ″ or the second control model 402 ″ that has been updated with the previously received actual measurement value M _k−1. This is a process corresponding to a time-series operation for obtaining the estimated state in a time-series manner.

When the control device 10 receives a new measured value _Mk from the drone 20 at time k, the state estimation filter first performs a post-prediction operation. In this post-prediction operation, a new actual measurement value M _k received at time k is reflected in the first control model 401 or the second control model 402 to obtain a new first control model 401 ′ or second control model 402 ′. Corresponds to the action. Subsequently, the state estimation filter shifts to a state in which the high-speed state prediction operation described above is executed as a time-series loop process. In this high-speed state prediction operation, a new first control model 401 ′ or second control model 402 obtained by reflecting the new actual measurement value M _k received at time k on the first control model 401 or the second control model 402. This corresponds to a process of executing a time-series operation for obtaining the estimated state of the drone 20 in a time-series manner at a higher speed than the passage of real time based on '.

の新たな値を、実測値Ｍ_ｋに基づいて更新する処理を行う。これは、時刻ｋにおいて受信した新たな実測値Ｍ_ｋを第１制御モデル４０１または第２制御モデル４０２に反映させて、新たな第１制御モデル４０１’または第２制御モデル４０２’を得る動作に対応する。その場合、状態推定フィルタは、時刻ｋにおいて受信した新たな実測値Ｍ_ｋが表す状態観測値

に基づいて時刻ｋ−３における無人機２０の新たな推定状態を、

として求める。 Specifically, the state estimation filter performs the following processing operations. When the state estimation filter that has received the new actual measurement value _Mk at time k shifts to the above-described retrospective prediction operation, the state prediction operation is performed up to k-3, which is the actual measurement time point when the actual measurement value _Mk was actually measured on the drone 20 side. Retroactively. In the above description with reference to FIG. 5, the flow of the virtual times J1 and J2 goes back to the actual time until the actual measurement time T _m ^k of the actual measurement value M _k when the new actual measurement value M _k is received. And respond to retroactive behavior. Therefore, the time difference between the time k and the time k-3 corresponds to a communication delay between the control device 10 and the drone 20. When the virtual time is retroactive as described above, the state estimation filter first performs a posterior prediction operation as follows. That is, the state estimation filter, when the state prediction operation is retroactive to k-3, which is the actual measurement time point at which the actual measurement value M _k was actually measured, newly received the new value of the internal state variable in the state estimation filter. Calculate based on _Mk . Specifically, the state estimation filter was obtained as the estimated state of the drone 20 at time k-3 by the above-described prior prediction operation.

Is updated based on the actual measurement value _Mk . This is an operation of reflecting the new measured value M _k received at time k in the first control model 401 or the second control model 402 to obtain a new first control model 401 ′ or second control model 402 ′. Correspond. In that case, the state estimation filter is the state observation value represented by the new actual measurement value M _k received at time k.

Based on the new estimated state of the drone 20 at time k-3,

Asking.

に基づいて時刻ｋ−ｍにおける無人機２０の新たな推定状態

をカルマンフィルタが求める処理は以下のように定式化することができる。

上記式（４）において、ｇ（ｋ−ｍ）は、カルマンゲインであり、事前予測動作において上記式（３）に基づいて求められた共分散を使用して、以下の式により算出されてもよい。

上記式（５）において、σ_ｗ ^２は、観測雑音ｗ_ｋの分散を表す。なお、上記式（５）に基づいて無人機２０の状態観測値から無人機２０の新しい推定状態を求める計算は、カルマンフィルタが無人機２０の現在時刻ｋにおける状態ｘ（ｋ）と状態観測値Ｙ（ｋ）＝［ｙ_０，…，ｙ_ｋ］との関係から現在時刻ｋの状態推定値を求めるフィルタリング動作モードに対応する。カルマンフィルタがフィルタリング動作モードで動作している場合に算出する共分散は、上記式（３）で求められる共分散とは異なり、以下の式で求められる事後共分散である。

The actual measured value M _k is actually measured at the actual measurement time k−m on the drone 20 side, and the state observation value represented by the actual measurement value M _k when the Kalman filter receives the actual measurement value M _k as the new actual measurement value at the time k.

Based on the new estimated state of the drone 20 at time km

The process in which the Kalman filter calculates can be formulated as follows.

In the above equation (4), g (km) is the Kalman gain, and may be calculated by the following equation using the covariance obtained based on the above equation (3) in the prior prediction operation. Good.

In the above equation (5), σ _w ² represents the variance of the observation noise w _k . The calculation for obtaining a new estimated state of the drone 20 from the state observation value of the drone 20 based on the above equation (5) is performed by the Kalman filter using the state x (k) and the state observation value Y at the current time k of the drone 20. (K) = [y ₀ ,..., Y _k ], which corresponds to the filtering operation mode for obtaining the state estimated value at the current time k. The covariance calculated when the Kalman filter is operating in the filtering operation mode is the posterior covariance obtained by the following equation, unlike the covariance obtained by the above equation (3).

を順次算出する。上記状態推定フィルタが実行する上記高速状態予測動作は、時刻ｋ−３、時刻ｋ−２、時刻ｋ−１および時刻ｋにそれぞれ対応する時系列的なループ処理として実行され、例えば、時刻ｈ−１における無人機２０の推定状態

を上記式（１）の状態方程式に適用することにより、時刻ｋにおける無人機２０の推定状態

を求めるものである。 Subsequently, the state estimation filter that reflects the new actual measurement value M _k received at time k in the internal state variable executes the high-speed state prediction operation at time k-3 faster than the passage of real time. As already described, the calculation performed by the state estimation filter in the high-speed state prediction operation is almost the same as the above-described pre-prediction operation except that the calculation is executed faster than the real time. First, as the time-series estimated state of the drone 20 at time k-3, time k-2, time k-1 and time k,

Are calculated sequentially. The high-speed state prediction operation executed by the state estimation filter is executed as a time-series loop process corresponding to each of time k-3, time k-2, time k-1 and time k. Estimated state of drone 20 in 1

Is applied to the state equation of the above equation (1), so that the estimated state of the drone 20 at the time k

Is what you want.

つまり、実時間よりも高速に実行される当該高速状態予測動作が現在時刻Ｔｃに追い付いたら、状態推定フィルタは、状態予測動作を事前予測動作に切り替えてそのまま継続する。これは、第１演算部１００ａと第２演算部１００ｂにより実行される以下の処理に対応する。すなわち、実時間の経過よりも高速に実行される上記時系列的演算が現在時刻Ｔｃに追い付いた後に、実時間の経過に歩調を合わせて上記時系列的演算により無人機２０の推定状態を時系列的に求め、画面表示のために表示装置１６０に出力する処理に対応する。 The state estimation filter that reflects the newly measured value M _k received at time k in the internal state variable executes the high-speed state prediction operation faster than real time until time k, and the high-speed state prediction operation until time k is performed. When finished, the high-speed state prediction operation executed at high speed catches up with the current time Tc. Subsequently, the state estimation filter returns to a state in which the prior prediction operation after time k + 1 is executed as a time-series loop process, and the estimated state of the drone 20 at time k + 1, k + 2, k + 3,. Calculate in time series.

That is, when the high-speed state prediction operation executed at a speed higher than the real time catches up with the current time Tc, the state estimation filter switches the state prediction operation to the prior prediction operation and continues as it is. This corresponds to the following processing executed by the first calculation unit 100a and the second calculation unit 100b. That is, after the time-series operation executed at a higher speed than the passage of real time catches up with the current time Tc, the estimated state of the drone 20 is set by the time-series operation in time with the passage of real time. This corresponds to processing that is obtained in series and output to the display device 160 for screen display.

To generalize the above description, the first calculation unit 100a and the second calculation unit 100b configure the state estimation filter as follows. From the measured time T _m ^k measured value M _k until the time to catch up to the current time, to perform the time-series calculation based on measured values M _k newly received faster than the elapsed real time, the first operation The unit 100a and the second calculation unit 100b configure the state estimation filter so as to include a retrospective state prediction operation for performing the prediction operation of the state estimation filter based on the actual measurement value _Mk . This retrospective state prediction operation, a posterior predictive operation to reflect the actual value M _k the internal state of the state estimation filter, the prediction operation of the state estimation filter based on the internal state reflecting the measured values M _k in real time And a high-speed state prediction operation performed at a speed higher than the elapsed time. That is, the state estimation filter moves to a high-speed state prediction operation in which the time series calculation is executed faster than the real time after executing the post-prediction operation as a process of reflecting the actual measurement value _Mk in the control model. Then, after the time-series operation (high-speed state prediction operation), which is executed at a higher speed than the passage of real time, catches up with the current time Tc, the state estimation filter In addition, the state estimation operation of the state estimation filter is configured to be switched to the prior prediction operation in a form that continues as it is (after the execution speed is reduced), and the estimated state of the drone 20 output by the prior prediction operation is: Provided to display device 160 for screen display.

In the above-described configuration using the state estimation filter shown in FIG. 6, starting from the measured time T _m ^k measured value M _k newly received, a prediction operation of the state estimation filter based on the measured values M _k newly received When the high-speed state prediction operation is executed at a speed higher than the real time and the execution of the high-speed state prediction operation catches up with the current time Tc, the prediction operation of the state estimation filter is switched to the prior prediction operation and continued as it is. Therefore, in the above configuration using the state estimation filter shown in FIG. 6, when realizing the process of estimating the current state of the drone 20 in time series based on the first control model 401 and the second control model 402. In addition, every time each loop process in the time domain described above is completed, the current state of the drone 20 can be directly obtained from the state variable calculated in each loop process. As a result, in the process of estimating the current state of the drone 20, it is not necessary to convert the internal variable values of the first control model 401 and the second control model 402 into the current state of the drone, and the calculation is performed at high speed. Can be

In the above configuration using the state estimation filter shown in FIG. 6, the current state of the drone is changed based on the first control model 401 ′ and the second control model 402 ′ reflecting the latest measured value M _k. to realize the processing of estimating the actual state represented by the latest measured values M _k the latest measured values M _k in the state variable value of the state estimation filter in the loop process corresponds to the time T _m ^k which is actually measured values It is only necessary to continue the state prediction operation as it is after reflecting the above. As a result, the latest measured values M _k to reflect the first control model 401 and a second control model 402, the first control model on which converts the latest measured values M _k in the internal representation of the control model The processing applied to the internal state of 401 and the second control model 402 becomes unnecessary, and the calculation can be speeded up.

FIG. 7 is a flowchart showing the operation flow of the state estimation filter described above with reference to FIG. In the flowchart shown in FIG. 7, the process starts initially at step S01, the time and the T = T ₀ and the initial setting, for k = 1 and the initial setting for the index value k corresponding to the respective times. In the following description, each index value k corresponds to each of the virtual times arranged on the time axis in the virtual time described above with reference to FIG. Subsequently, the processing of the flowchart of FIG. 7 proceeds to step S02. Step S02 continues to wait for a new actual measurement value _Mk to be received from the drone 20, and when the new actual measurement value _Mk is received, the first control model 401 reflecting the new actual measurement value _Mk. This corresponds to a stage in which time series calculation for obtaining the estimated state of the drone 20 in time series based on 'or the second control model 402' is executed faster than the passage of real time. For example, the step S02, the operation carried out during the second operation unit 100b for performing a second control model 402 in FIG. 4, the operation and the time _{t 3} is performed from time _{t 1} to time _{t 2} to _{t 4} Corresponding to Also for example, step S02 is performed between the first computing unit 100a that performs a first control model 401 in FIG. 4, the operation and the time _{t 4} performed during the period from the time _{t 2} to time _{t 3} to _{t 5} Corresponds to the action.

から現在時刻ｋにおける無人機２０の推定状態

を上記式（１）に従って算出する事前予測動作を行う。 Step S02 includes sub-steps S71 to S76. When the execution of step S02 is started, the processing of the flowchart in FIG. 7 executes sub-step S71 to sub-step S76. When the process of the flowchart of FIG. 7 proceeds to sub-step S71, the state estimation filter executes a process while waiting for a new measured value _{Mk to} be received from the drone 20. Specifically, in sub-step S71, the state estimation filter determines the estimated state of the drone 20 at the immediately preceding time k-1.

To the estimated state of the drone 20 at the current time k

Is performed in advance according to the above equation (1).

となるので、この状態観測値から上記式（４）に基づいて時刻ｋ−ｍにおける無人機２０の新たな推定状態

を求める。続いて、状態推定フィルタは、仮想時刻を実測値Ｍ_ｋの実測時刻ｋ−ｍまで遡らせるために、ｋ＝ｋ−ｍとする。 Subsequently, the process proceeds to sub-step S72 and waits until the clock hand advances from the current time k to the next time k + 1. Subsequently, the process proceeds to sub-step S73, and the state estimation filter determines whether or not a new actual measurement value M _k is received from the drone 20, and if not, increments the value of the index k. After setting k = k + 1, the process returns to sub-step S71, and if received, the process proceeds to sub-step S74. In sub-step S74, the state estimation filter determines the new value of the internal state variable of the state estimation filter based on the state observation value of the drone 20 newly represented from the actual measurement value _Mk received from the drone 20 at time k. Is calculated. Specifically, if the newly received actual measurement value M _k is an actual measurement value actually measured at the actual measurement time k−m on the drone 20 side, the drone represented by the newly received actual measurement value M _k. The 20 state observations are

Therefore, from this state observation value, a new estimated state of the drone 20 at time k-m based on the above equation (4)

Ask for. Subsequently, the state estimation filter sets k = km so that the virtual time can be traced back to the actual measurement time km of the actual measurement value _Mk .

から次の時刻ｋ−ｍ＋１における無人機２０の推定状態

を求める。続いて、サブステップＳ７６においては、インデックスｋの値をインクリメントしてｋ＝ｋ＋１とした上で、サブステップＳ７５に戻る。 Subsequently, the process proceeds to sub-step S75, and the state estimation filter determines whether or not the time k has caught up to the current time Tc in real time. For example, the process ends the execution of step S02 and proceeds to step S03. In sub-step S76, the estimated state of drone 20 at time km is performed by performing the same processing as in sub-step S71.

To the next time k−m + 1 from the estimated state of the drone 20

Ask for. Subsequently, in sub-step S76, the value of index k is incremented to k = k + 1, and the process returns to sub-step S75.

  Unlike the loop process in which sub-step S71 to sub-step S73 are repeatedly executed, in the loop process in which sub-step S75 and sub-step S76 are repeatedly executed, a process of waiting until the clock hand moves from time k to time k + 1 is performed. There is no intervention. Therefore, if the execution speed in terms of hardware performance of the arithmetic unit 100 is sufficiently high, the loop processing that repeatedly executes the sub-step S75 and the sub-step S76 will be executed faster than the passage of real time.

から次の時刻ｋ＋１における無人機２０の推定状態

を求める。続いて、サブステップＳ７７においては、インデックスｋの値をインクリメントしてｋ＝ｋ＋１とした上で、サブステップＳ７８に進み、サブステップＳ７８においては、時計の針が現在時刻ｋから次の時刻ｋ＋１に進むまで待つ。 In step S03, the estimated state of the drone 20 is obtained in time series based on the first control model 401 ″ or the second control model 402 ″ that has been updated with the actual measurement value M _k received most recently from the drone 20. This corresponds to a process of displaying the time series calculation and the estimated state of the drone 20 obtained in time series on the screen. For example, the step S03, the operation carried out during the first operation unit 100a that performs a first control model 401 in FIG. 4, the operation and the time _{t 3} is performed from time _{t 1} to time _{t 2} to _{t 4} Corresponding to Also for example, step S03 is performed between the second operation unit 100b for performing a second control model 402 in FIG. 4, the operation and the time _{t 4} performed during the period from the time _{t 2} to time _{t 3} to _{t 5} Corresponds to the action. Step S03 includes sub-steps S77 to S79. When the execution of step S03 is started, the processing of the flowchart of FIG. 7 executes sub-step S77 to sub-step S79. When the process of the flowchart of FIG. 7 proceeds to sub-step S77, the state estimation filter performs the same process as in sub-step S71, so that the estimated state of the drone 20 at the current time k is calculated.

The estimated state of the drone 20 at the next time k + 1 from

Ask for. Subsequently, in sub-step S77, the value of index k is incremented to k = k + 1, and the process proceeds to sub-step S78. In sub-step S78, the clock hand moves from the current time k to the next time k + 1. Wait until it goes.

  Subsequently, the process proceeds to sub-step S79, and a new actual measurement value is received from the drone 20, or the sub-step S75 and the sub-step S76 are repeatedly executed by another state estimation filter in the other arithmetic unit 100. If the loop process catches up with the current time Tc, the process proceeds to step S04. If a new actual measurement value has not been received and the above repeated loop processing by another state estimation filter being executed by another arithmetic unit 100 has not caught up to the current time Tc, the processing returns to sub-step S77. . In step S04, it is determined whether or not the process of the state estimation filter should be terminated. If not, the process returns to sub-step S71 constituting step S02, and the above-described series of processes is performed again. Otherwise, the state estimation filter processing is terminated.

Next, another embodiment of a control device and a control method for remotely controlling the drone 20 will be described. In this embodiment, at least one third calculation unit 100c among the two or more calculation units 100 (100a to 100n) is independent of the first calculation unit 100a and the second calculation unit 100b. It is added as a calculation unit that obtains the estimated state in time series. Further, the third arithmetic unit 100c executes the third control model 403 as a functional module separately from the first arithmetic unit 100a and the second arithmetic unit 100b. In this embodiment, when the third calculation unit 100c receives the actual measurement value M _k measured at time T _m ^k , the first calculation unit 100a and the second calculation unit 100b display the estimated states of the drone 20 in time series. The following processing is executed in parallel with the time-series operation that is obtained automatically. That is, based on the new third control model 403 ′ obtained by reflecting the actual measurement value M _k in the third control model 403, a time series operation for obtaining the estimated state of the drone 20 in time series is performed at time T Execute from _m ^k to a point in the future rather than real time. The time series calculation by the third calculation unit 100c is executed at a higher speed than the passage of real time until a future time point is reached. When the third calculation unit 100c executes the time series calculation at a high speed until the future time point, the third calculation unit 100c displays the estimated state of the drone 20 from the current time Tc to the future time point. A screen is displayed on the device 160. The operation for displaying the estimated state of the drone 20 up to the future point in time is an image displaying the current estimated state of the drone 20 obtained in time series by the first calculation unit 100a and the second calculation unit 100b. The thin shadow of the drone 20 is displayed in a superimposed manner.

FIG. 8 is a timing chart for explaining how the third calculation unit 100c executes the above-described time-series calculation at a higher speed than the real time from the time T _m ^k to a future time point rather than the real time. It is. FIG. 9 is a diagram showing an internal configuration of the control device 10 used for remotely controlling the drone 20 in this embodiment. The control device 10 shown in FIG. 9 is substantially the same as the control device 10 shown in FIG. 2 except that the work environment model 121, the simulation program 122, and the future trajectory prediction data 123 are stored in the storage unit 120. It has the composition of. The work environment model 121, the simulation program 122, and the future trajectory prediction data 123 will be specifically described below. Further, each of the first calculation unit 100a, the second calculation unit 100b, the third calculation unit 100c, the fourth calculation unit 100d, the fifth calculation unit 100e, and the sixth calculation unit 100f included in the control device 10 illustrated in FIG. The calculation unit is the same as each of the two or more calculation units 100 (100a to 100n) illustrated in FIG. Although not shown in FIG. 9, the third calculation unit 100 c and the fourth calculation unit 100 d use the third control model 403 and the fourth control model 404 as functional modules that model the control response characteristics of the drone 20. Each running.

  As described above with reference to FIGS. 3 to 5, the first calculation unit 100 a and the second calculation unit 100 b indicate the current state of the drone 20 based on the first control model 401 and the second control model 402. It is a calculation part calculated | required in time series. As will be described later with reference to FIG. 8, the third calculation unit 100 c and the fourth calculation unit 100 d are based on the third control model 403 and the fourth control model 404, and the future time point of the drone 20. It is a calculating part which predicts until the state in time series. Further, as will be described later, the fifth arithmetic unit 100e is configured so that the drone 20 contacts an obstacle based on a result of time-series prediction of the state of the drone 20 from the present to the future time point. The avoidance determination process is executed to detect in advance the possibility of sitting. In addition, as will be described later with reference to FIGS. 10 and 11, the sixth arithmetic unit 100 f performs virtual work while the control device 10 interrupts communication with the drone 20 and is in an offline state. A simulation for simulating the remote control of the drone 20 is executed assuming the environment.

Hereinafter, with reference to FIG. 8 and FIG. 9, the third calculation unit 100 c performs the time-series calculation described above from the actual measurement time T _m ^k of the actual measurement value M _{k to} a time point in the future (future virtual time). The flow of processing that is executed at a speed higher than real time will be described. In FIG. 8, the first calculation unit 100 a performs the current estimation of the drone 20 based on the first control model 401 ′ that reflects the actual measurement values received at times T ₁ , T _3, and T ₅ shown in FIG. 4. In order to obtain the state, time series operations similar to the scenario described above in FIG. 4 are sequentially executed. More specifically, the first calculation unit 100a, similarly to the scenario shown in FIG. 4, the measured value M ₁ received at time T ₁ by reflecting the first control model 401 and the new first control model 401 ' . Subsequently, the first calculation unit 100a goes back to the actual measurement time T _m ^1, which is the time when the actual measurement value M ₁ received at the time T ₁ is actually measured by the drone 20. Subsequently, the first calculation unit 100a executes the above-described time-series calculation from the actual measurement time T _m ¹ to the time t ₁ (corresponding to the period represented by p1 in FIG. 4) at a higher speed than the passage of real time. . Here, the time t ₁ is series operations when executed on the high-speed by the first calculation unit 100a corresponds to the time to catch up to the current time Tc. Similarly, first calculating unit 100a is, for the measured values _{M 3} and _{M 5} respectively received at time _{T 3} and _{T 5,} it performs the same process. Further, the second calculation unit 100b determines the current estimated state of the drone 20 based on the second control model 402 ′ reflecting the actual measurement values received at times T ₂ , T ₄ and T ₆ shown in FIG. In order to obtain it, the same time series operation as the scenario described above with reference to FIG. 4 is sequentially executed.

On the other hand, the third calculation unit 100c includes a series operation when the above-described reflecting the measured value M ₁ received at time T ₁ to the control model, run from the measured time T _m ¹ faster than the elapsed real time This is the same as the above-described operation of the first calculation unit 100a, but the following points are different. In other words, in parallel with the first operation unit 100a and a second calculating unit 100b, the third calculation unit 100c includes a measured value _{M 1} received at time _{T 1} by reflecting the third control model 403 new third control model 403 ′. Subsequently, the third computation section 100c has a series operations when the above-measured value M ₁ received at time T ₁ is to go back to the actual time T _m ¹ is the time that is actually measured in the drone 20. Subsequently, the third computation section 100c is starts series operation when the aforementioned than real time is executed at high speed from the measured time T _m ^1, time series operation the current time Tc is performed on the high-speed even after the time t ₁ to catch up to and then continue with the time series operations performed in the high speed. As a result, the third calculation unit 100c executes a series operation when the above is performed faster than real time, to the virtual time t ₃ of the future past the current time, faster than the elapsed real time (Corresponding to the period represented by τ _R in FIG. 8). Therefore, the third calculation unit 100c is on the high speed until _p only point in the future time width τ than the time t ₁ of the first arithmetic unit 100a finishes the operation to be executed faster than real time the time-series operation Continue the time-series operations to be performed. Third arithmetic unit 100c is, the series operation when the above is performed faster than real-time, the finish to run until future virtual time t _3, the third calculation unit 100c, the virtual future the following operations to run until the time _{t 4.} That is, the third calculation unit 100c, based on the third control model 403 "which updating by the measured value M ₁ is completed, the series operation when determining the estimated state of the drone 20 in time series, the real-time In parallel with this, the third calculation unit 100c displays the estimated state of the drone 20 obtained in time series on the display device 160 on the screen. As will be described later, this screen display corresponds to a process of displaying the future trajectory of the unmanned aerial vehicle 20 based on the estimated state of the unmanned aerial vehicle 20 in a form that precedes the passage of real time.

  As described above, according to the embodiment shown in FIG. 8, the state of the drone 20 is estimated in time series until a future time point, and the state of the drone 20 from the current time Tc to the future time point is determined. Can be displayed on the screen superimposed on the current state. As a result, according to this embodiment, when remote control of the drone 20 is continued along the trajectory assumed by the operator, the drone 20 becomes an obstacle at a future time (future virtual time). It is possible to detect in advance the possibility of touching or sitting down. Therefore, in such a case, the operator can correct the trajectory in advance. In particular, when the operator continues to control the drone by overlooking an obstacle hidden in the blind spot when viewed from the field of view around the drone 20, the drone 20 may come into contact with the obstacle due to an operator error. It is possible to detect in advance the possibility of sitting down and to prevent an operation error of the drone 20.

  In addition, the process which detects the possibility that the drone 20 contacts an obstacle or sits down at a future time point is executed by the fifth arithmetic unit 100e that operates as an avoidance determination function unit in FIG. The first calculation unit 100e shown in FIG. 9 is expected to be followed by the drone 20 based on the result of the state estimation of the state of the drone 20 up to a future time point by the third calculation unit 100c. Predict the future trajectory. Subsequently, the fifth arithmetic unit 100 e writes data representing the predicted future trajectory into the future trajectory prediction data 123 stored in the storage unit 120. Subsequently, the fifth arithmetic unit 100e is based on the spatial relationship between the work environment represented by the work environment description model 121 stored in the storage unit 120 and the future trajectory of the drone 20 represented by the future trajectory prediction data 123. Then, it is determined whether or not the predicted trajectory indicates a contact between the drone 20 and another structure or a saddle of the drone 20. Here, in the work environment description model 121, other structures existing in the work environment of the drone 20, unevenness of the ground, the shape of the wall surrounding the drone 20, and the like are geometric data. It is described by a dynamic modeling language. As described above, when the fifth arithmetic unit 100e detects in advance the possibility that the drone 20 will contact an obstacle or sit down, the fifth arithmetic unit 100e that operates as an avoidance determination function unit. Can issue a warning to the operator that the drone 20 needs to take an avoidance trajectory.

Next, still another embodiment of a control device and a control method for remotely controlling the drone 20 will be described from a specific example with reference to FIGS. 8 and 9. As described above in the scenario illustrated in FIG. 8, the third calculation unit 100 c starts the above-described time-series calculation that is executed at a speed higher than the real time from the actual measurement time T _m ^1, but is executed at the high speed. time series operation is continued as a series operations when executed on the high-speed even after the time t ₁ to catch up to the current time Tc. As a result, the third calculation unit 100c includes a series operation when the above is performed faster than real time, to the virtual time t ₃ of the future past the current time, i.e., the period indicated by tau _R in FIG. 8 Run faster than the passage of real time. In parallel with this, during the period from time t ₁ to time t ₃ , as described above with reference to FIG. Are executed while the first calculation unit 100a and the second calculation unit 100b alternate with each other. At this time, the trajectory of the future of the drone 20 based on the estimated future state of the drone 20, the process of screen display in the form preceding the passage of real time, virtual time t ₃ of the future by the third calculation unit 100c It is performed on until the virtual time t ₄ of the future.

However, the third arithmetic unit 100c is not received new measured values since receiving a measured value M ₁ at time T ₁ is a screen in the form of a preceding track of the future of the drone 20, the elapsed real-time If the display process is continued after the future virtual time t ₄ , the modeling error of the third control model 403 naturally increases. Because for the virtual time t ₄ after the future, the third calculation unit 100c is without further update the third control model 403 by a new measured value M ₂ received from the unmanned aircraft 20 to the measured value M ₁ and later, This is because the process of displaying the future trajectory of the drone 20 on the screen is continued. As a result, in the above-described embodiment, for the period from the future virtual time t ₄ to the virtual time t ₅ , the third calculation unit 100 c uses the future orbit as described above based on the estimated future state of the drone 20. The preceding screen cannot be displayed.

Therefore, in this embodiment, a process for displaying the future trajectory of the unmanned aerial vehicle 20 based on the estimated future state of the unmanned aerial vehicle 20 in a form that precedes the passage of real time is shown in FIG. 100d is, may be started from the time _{t 4.} In that case, the fourth arithmetic unit 100d, based on the fourth control model 404 "which updating actually measured value M ₂ received is completed at time T _2, which precedes the time t _2, the future drone 20 while seeking estimated state in time series, the trajectory of the future of the drone 20 is a display screen in a manner preceding the passage of real time. Therefore, the fourth arithmetic unit 100d is updating the actually measured value M ₂ is completed Based on the fourth control model 404 ″, the future trajectory of the drone 20 is obtained from the future virtual time t ₄ to the time t ₅ while obtaining the estimated state of the drone 20 in time series. The screen can be displayed in a form preceding the passage of real time. The operation of displaying the future trajectory of the drone 20 on the screen as described above displays the current estimated state of the drone 20 obtained in time series by the first calculation unit 100a and the second calculation unit 100b. It may be performed in such a manner that a thin shadow of the drone 20 is displayed so as to be superimposed on the image. That is, at least one of the two or more arithmetic units 100 (100a to 100n) displays a current estimated state of the drone 20 obtained in time series by the first arithmetic unit 100a or the second arithmetic unit 100b. The estimated future state of the drone 20 obtained in time series by the third calculation unit 100c and the fourth calculation unit 100d is displayed on the screen of the display device 160 in a form preceding the passage of real time. It is configured as follows.

The third calculation unit 100c is, real-time to reach the virtual time t ₅ the time-series operation based on the new third control model 403 'which reflects the measured value M ₃ received at time T ₃ After execution at a higher speed, the fourth calculation unit 100d may exchange the processes executed with the third calculation unit 100c. In this case, during the period from the future virtual time t ₅ to the time t ₆ , the third arithmetic unit 100c determines the future of the drone 20 based on the third control model 403 ″ updated with the actual measurement value M ₃ . the preceding screen display of future trajectory as described above based on the estimated state may be performed. on the other hand, the fourth arithmetic unit 100d, a new fourth control that reflects the measured value M ₄ received at time T ₄ based on the model 404 'may be the processing to be executed faster than real-time to reach the time-series operation on the virtual time t ₆ is the fourth arithmetic unit 100d executes.

Generalizing the above description with reference to FIG. 8 and FIG. 9, the third calculation unit 100c and the fourth calculation unit 100d included in the two or more calculation units 100 (100a to 100n) have the following operations. Are configured to execute in parallel with the first arithmetic unit and the second arithmetic unit. First, the fourth arithmetic unit 100d, in addition to the measured value received in the next one of the measured values received by the third arithmetic unit (time T ₁ measured value M ₁ received _in) (e.g., found M ₂ ) Is reflected in the fourth control model 404, and the following calculation is executed based on the new fourth control model 404 ′ obtained. That is, a time series calculation for obtaining the current estimated state of the drone 20 from an operation command in a time series is a time (for example, time T _m ² ) when another actual measurement value (for example, actual measurement value M ₂ ) is actually measured. From the current time Tc to another future time point (for example, future virtual time t ₄ ), the execution is performed at a speed higher than the real time. Subsequently, after the time-series calculation by the previous fourth calculation unit 100d reaches another future time point (for example, the future virtual time t ₄ ), the unattended obtained by the time-series calculation by the fourth calculation unit 100d The current estimated state of the machine 20 is provided as the estimated future state of the drone 20 to the display device 160 until the next future time point (for example, the future virtual time t ₅ ). Subsequently, the third computation unit 100c is the next one future point in time (e.g., future virtual time t _{5 in)} from to other future time point of the next (e.g., future virtual time t _6), the drone 20 The fourth computing unit 100d provides the next estimated future state (for example, the future virtual time t ₆ ) to the next next future time (for example, Until the future virtual time t ₇ ), the future estimated state of the drone 20 is provided to the display device 160.

From the above, according to this embodiment, after the third arithmetic unit 100c continued to run faster than real-time to catch up to the time T ₃ in the future the series operation when the above-described estimation of the future of the drone 20 can modeling error of the well to continue the time T ₄ after the processing for the screen display of the trajectory of the future of the drone 20 third control model 403 to alleviate the problem of increases based on the state. The reason is as follows. The fourth calculation unit 100d performs the time-series calculation based on the fourth control model 404 ′ reflecting the actual measurement value that is newer than the third control model 403 ″ used by the third calculation unit 100c in the future virtual time T _4. The fourth calculation unit 100d then performs the above-described operation based on the fourth control model 404 ″ that has been updated with the actually measured values that are newer than the third control model 403 ″. executed in the processing time T ₄ or later to the future of the orbit the screen display of the drone 20.

Further, according to this embodiment, even while the third arithmetic unit 100c continues to run faster than real-time to catch up to the series operation future point in time t ₅ from time t ₄ in the future when the above-future the series state when the drone 20 from time t ₄ to time t ₅ of the next future, in a form preceding the elapse of real time, be screen display with time series state estimation it can. Therefore, according to this embodiment, the third between the arithmetic unit 100c continues to run faster than real-time to catch up to the time t ₅ in the future the series operation when the above-described, prior to than real-time time There is no blank time during which control cannot be performed while observing the future state of the drone 20 whose state has been serially estimated. As a result, according to this embodiment, even while the third arithmetic unit 100c continues to execute the series operation when the above faster than real-time to catch up to the time t ₅ for the next future, unmanned in the near future It is possible to detect in advance the possibility that the machine 20 may come into contact with an obstacle or sit down.

In addition, during the period from time t ₁ to time t ₃ shown in FIG. 8 (period indicated by τ _p in FIG. 8), the third calculation unit 100c is unattended in order to catch up with the future time t ₃ as soon as possible. The time series calculation for obtaining the estimated state of the machine 20 in time series must be executed as fast as possible. Accordingly, during that time, the third calculation unit 100c cannot perform processing for displaying the estimated state of the drone 20 obtained in time series on the screen. Therefore, in an alternative embodiment, during the period from time t ₁ to time t ₃ shown in FIG. 8 (period indicated by τ _p in FIG. 8), the fourth arithmetic unit 100d shown in FIG. A process of displaying the future trajectory of the unmanned aerial vehicle 20 based on the estimated state of the future in a form preceding the passage of real time is performed instead of the third arithmetic unit 100c. Specifically, in this embodiment, the fourth arithmetic unit 100d is, during a period from time t ₁ shown in FIG. 8 to time t _3, the trajectory of the future drone 20 based on the estimated future state of the drone 20 Is displayed on the screen in advance of the passage of real time. At that time, the fourth arithmetic unit 100d obtains the future estimated state of the drone 20 in a time series based on the fourth control model 404 ″ in which the update operation using the actual measurement value M ₁ received at the time T ₁ is completed. On the other hand, the future trajectory of the drone 20 is displayed on the screen in advance of the passage of real time, and the future trajectory of the drone 20 is displayed on the screen as described above. 2 The calculation unit 100b may superimpose the current estimated state of the drone 20 obtained in time series on the displayed image and display a thin shadow of the drone 20.

  Next, still another embodiment of a control device and a control method for remotely controlling the drone 20 will be described. In the remote control system described above with reference to FIGS. 1 and 2, remote control for causing the drone 20 to perform an operation of passing through a narrow part or continuously avoiding obstacles arranged in a complicated manner is performed. When performed by an operator, there is a high possibility of failure due to a slight operator error. Therefore, prior to causing the drone 20 to perform such an operation with a high probability of failure, after the operator has previously performed remote operation training in a virtual environment in which a situation similar to the reality is set, It is preferable to start actual remote control in order to avoid risk. Therefore, in this embodiment, the sixth calculation unit 100f, which is at least one of the two or more calculation units 100 (100a to 100n), is configured to perform the following operations.

  First, prior to the start of remote control of the drone 20, the sixth arithmetic unit 100 f generates an operation profile including a series of operation commands for the drone 20 by repeating trial and error based on simulation. FIG. 10 shows a simulation execution state in which the control device 10 is executing an operation of generating an operation profile 125 including a series of operation commands for the drone 20 by repeating trial and error based on simulation. As shown in FIG. 10, in the simulation execution state, communication between the control device 10 and the drone 20 is interrupted, and during that time, the control device 10 performs remote operation of the drone 20 by simulation. The actual remote control is not performed only by performing the simulation. In the simulation execution state shown in FIG. 10, the operation command input by the operator performing simulated remote control by simulation is stored in the form of being accumulated in the operation profile 125 stored in the storage unit 120. Go. Further, in the simulation execution state shown in FIG. 10, a virtual environment in which a situation similar to that in reality is set in order to perform a simulation is described by a work environment description model 121 stored in the storage unit 120. In the work environment description model 121, other structures existing in the work environment of the drone 20, the unevenness of the ground, the shape of the wall surrounding the drone 20, etc. are used as geometric data as a geometric modeling language. It is described by.

  10 and 11, only one operation profile 125 is illustrated, but the sixth arithmetic unit 100f generates a plurality of (S) operation profiles 125s (1 ≦ s ≦ S). However, it may be stored in the storage unit 120. For example, when the operator repeatedly executes simulated remote control of the drone 20 by a plurality of times through trial and error, the sixth calculation unit 100f individually sets the operation profile for each of the repeated executions. 125 may be generated. In that case, a plurality (S) of operation profiles 125s (1 ≦ s ≦ S) corresponding to the number of times that the simulated remote control of the drone 20 is repeatedly performed through trial and error are generated, It is stored in the storage unit 120.

  Subsequently, as shown in FIG. 11, the control device 10 is communicably connected to the drone 20 via the optical fiber line 30, and the sixth arithmetic unit 100f is generated in the simulation execution state shown in FIG. From among a plurality of operation profiles 125s (1 ≦ s ≦ S), an operation profile 125 ′ that is assumed to be able to successfully complete a series of operations is selected. Subsequently, as illustrated in FIG. 11, the sixth calculation unit 100 f performs semi-automatic control (semi-automatic control) on the drone 20 based on the operation profile 125 ′. Subsequently, the sixth calculation unit 100f determines that the estimated state of the drone 20 obtained by the third calculation unit 100c in a time series from the real time to a future time point is an avoidance state (for example, contact with an obstacle or the aforementioned It is detected whether or not it indicates a drone scorpio. Subsequently, when the avoidance state (for example, contact with an obstacle or the saddle of the unmanned aircraft) is detected, the above-described semi-automatic control being executed based on the operation profile 125 ′ is manually performed by the operator. Intervene remote control. Specifically, the sixth arithmetic unit 100f superimposes the control command generated by the manual operation of the operator on the control command generated and differenced by the semi-automatic control (semi-automatic control). Thereby, the sixth calculation unit 100f can correct the control command transmitted to the drone 20. Thereafter, the operator manually controls the drone 20 based on the screen display of the estimated state of the drone 20 obtained in time series by the first calculation unit 100a and the second calculation unit 100b.

  That is, in this embodiment, prior to the start of the remote control of the drone 20, an operation scenario when the drone 20 is remotely operated in the virtual environment having the same operation interface as the manual remote control is simulated. Run a simulation. The simulation may be performed by causing the sixth arithmetic unit 100f to read out and execute the simulation program 122 from the storage unit 120. In this embodiment, an operation profile including a series of operation commands for the drone 20 is generated by repeating trial and error to simulate and execute various operation scenarios by the simulation, and the drone 20 is semi-automatically generated according to the operation profile. (Semi-automatic) control. In addition, during the execution of semi-automatic control according to the above-described operation profile, if it is detected in advance that there is a possibility of contact with an obstacle or the drone being seated, manual remote control by the operator And the semi-automatic (semi-automatic) control can be corrected.

  In addition, the process which detects the possibility that the drone 20 contacts an obstacle or sits down at a future time point is executed by the fifth arithmetic unit 100e that operates as an avoidance determination function unit in FIG. The first calculation unit 100e shown in FIG. 9 is expected to be followed by the drone 20 based on the result of the state estimation of the state of the drone 20 up to a future time point by the third calculation unit 100c. Predict the future trajectory. Subsequently, the fifth arithmetic unit 100 e writes data representing the predicted future trajectory into the future trajectory prediction data 123 stored in the storage unit 120. Subsequently, the fifth arithmetic unit 100e is based on the spatial relationship between the work environment represented by the work environment description model 121 stored in the storage unit 120 and the future trajectory of the drone 20 represented by the future trajectory prediction data 123. Then, it is determined whether or not the predicted trajectory indicates a contact between the drone 20 and another structure or a saddle of the drone 20.

  Therefore, according to this embodiment, by repeating trial and error in which various operation scenarios are simulated and executed by simulation, an operator can perform operation training in advance using the same operation interface as manual remote control. . If an unscheduled failure risk occurs during execution of the semi-automatic (semi-automatic) control, the semi-automatic control can be corrected manually. As a result, according to this embodiment, it is possible to reduce the risk of failure when causing the drone 20 to perform a complicated operation with a high probability of failure.

  Next, still another embodiment of a control device and a control method for remotely controlling the drone 20 will be described. In this embodiment, at least one of the two or more arithmetic units 100 (100a to 100n) may be configured to execute the following operation. First, whether the estimated state of the drone 20 obtained in a time series from the real time to the future time point by the third arithmetic unit 100c indicates a avoidance state (for example, contact with an obstacle or the drone of the drone). Detect whether or not. Subsequently, when an avoidance state (for example, contact with an obstacle or a saddle of the unmanned aircraft) is detected, an emergency operation that causes the unmanned aircraft 20 to take an avoidance path in preference to the operation command input by the operator. A command is transmitted to the drone 20.

  As described above, according to this embodiment, when the state of the drone 20 is estimated to the future time point, when the possibility of contact with an obstacle or the drone being seated is detected in advance, It is possible to automatically execute avoidance control in which the calculation unit 100 of the control device 10 causes the unmanned aircraft 20 to take an avoidance trajectory, superior to manual remote control by a person. As a result, when it is detected in advance that the drone 20 may come into contact with an obstacle or the drone 20 sits down, the avoidance control is not left to the operator. It can be automatically executed without intervention of the operator based on the judgment of the calculation unit 100.

DESCRIPTION OF SYMBOLS 1 Remote control system 10 Control apparatus 20 Unmanned machine 21 Unmanned machine side control apparatus 22 Servo mechanism 30 Optical fiber line 42,43,45,46,47 Signal path 51,52 Changeover switch 91,92 Surveillance camera / other unmanned machine 93 Monitoring device 100 (100a, 100b, 100c, 100d, 100e, 100f) Arithmetic unit 110 Communication unit 120 Storage unit 121 Work environment model 121 Work environment description model 122-Program 123 Future trajectory prediction data 125 Operation profile 130 Encoder 140 Interface 150 Operation panel 160 Display device 211 Communication unit 212 on unmanned aircraft side Control calculation unit 221 on unmanned aircraft side Sensor / cameras 222 Actuators 401 First control model 402 Second control model 403 Third control model 404 Fourth control model 411 12 switching means J1, J2 virtual time Mk (M1, M3, M4, M5, M6) measured value T1, T2, T3, T4, T5, T6 reception time Tc current time Tmk (Tm1, Tm3, Tm4, Tm5, Tm6) Actual measurement time Y, y State observation value k Index value

Claims (14)

A display device capable of sequentially displaying the current estimated state of the drone on the screen;
An operation command input by an operator based on the current estimated state of the drone displayed on the display device for remote control of the drone can be transmitted to the drone, and the drone A communication unit capable of sequentially receiving actual measurement values of the state of the own machine from the drone,
The display device displays the current estimated state of the unmanned aircraft obtained by executing a time-series operation for obtaining the current estimated state of the unmanned aircraft from the operation command based on a control model that reflects the actual measurement values. Two or more calculation units including a first calculation unit and a second calculation unit that can be provided to,
A control device comprising:
The first calculation unit and the second calculation unit are configured to alternately receive the actual measurement value, and each time the actual measurement value is received, the time calculation operation is repeatedly performed.
The first calculation unit is based on one control model reflecting one actually measured value, which is received alternately, until one time when the time series calculation catches up from the actually measured time of the one actually measured value to the current time. The time-series operations can be executed sequentially at a speed higher than the passage of real time,
Based on another control model reflecting other actually measured values received alternately, the second calculating unit is based on the time series calculation from the actually measured time of the other actually measured values to another time that catches up with the current time. The time-series operations can be executed sequentially at a speed higher than the passage of real time,
The first calculation unit is configured to provide the display device with a current estimated state of the drone from the one time to the next other time,
The second computing unit is configured to provide the display device with a current estimated state of the unmanned aircraft from the other time to the next one time. apparatus. The control model is configured to include a state estimation filter;
From the actual measurement time of the actual measurement value to the time of catching up with the current time, the time series calculation executed at a speed higher than the passage of real time includes a post-prediction operation that reflects the actual measurement value in the internal state of the state estimation filter. And a high-speed state prediction operation for performing the prediction operation of the state estimation filter based on the internal state reflecting the actual measurement value at a speed higher than the lapse of real time,
The process of providing the display device with the current estimated state of the drone continues the high-speed state prediction operation after the time-series operation executed faster than the passage of real time catches up with the current time The state estimation filter is configured to include a pre-prediction operation that displays the estimated state in time series while performing the prediction operation of the state estimation filter in accordance with the passage of real time.
The unmanned aerial vehicle control device according to claim 1. The two or more arithmetic units include a third arithmetic unit,
The third calculation unit includes:
Based on the one control model obtained by reflecting the one actual measurement value on the control model, a time-series operation for obtaining a current estimated state of the drone from the operation command in time series is performed. It is configured to execute faster than the passage of real time, from the time when the actual measurement value of the actual measurement passed through the current time to the time of one future.
The time series calculation by the third calculation unit is configured to be executed in parallel with the time series calculation by the first calculation unit and the second calculation unit,
The current estimated state of the drone obtained by the time-series operation by the third operation unit after the time-series operation by the third operation unit reaches the one future time point is determined as the future of the unmanned aircraft. The unmanned aircraft control device according to claim 1, wherein the estimated state can be provided to the display device. The two or more calculation units include a fourth calculation unit,
The fourth calculation unit includes:
Based on the other control model obtained by reflecting the other actual measurement value on the control model, a time series operation for obtaining the current estimated state of the drone from the operation command in time series is performed. It is configured to execute faster than the passage of real time, from the time when the actual measurement value of the actual measurement passed through the current time to another future time point,
The time series computation by the fourth computation unit is configured to be executed in parallel with the time series computation by the first computation unit, the second computation unit, and the third computation unit,
The current estimated state of the unmanned aircraft obtained by the time-series computation by the fourth computation unit after the time-series computation by the fourth computation unit reaches the other future time point is determined as the future of the unmanned aircraft. The estimated state can be provided to the display device,
The third calculation unit is configured to provide the display device with an estimated future state of the drone from the one future time point to the next other future time point,
The fourth calculation unit is configured to provide the display device with an estimated future state of the drone from the other future time point to the next one future time point.
The unmanned aerial vehicle control device according to claim 3. At least one of the two or more arithmetic units is:
The third calculation unit and the fourth calculation unit are time-sequentially superimposed on a screen that displays the current estimated state of the drone obtained by the first calculation unit or the second calculation unit in time series. Configured to display the estimated future state of the drone obtained on the screen in a form that precedes the passage of real time,
The unmanned aerial vehicle control device according to claim 4. At least one of the two or more arithmetic units is:
Prior to the start of remote control of the drone, an operation for generating an operation profile including a series of operation commands for the drone by repeating trial and error based on simulation;
An unmanned operation of the drone based on the operation profile;
An operation of detecting whether or not a future estimated state of the unmanned aircraft obtained in a time series from the third arithmetic unit to a future time point in real time indicates an avoidance state;
When the avoidance state is detected, the control of the drone is controlled based on a screen display that displays the current estimated state of the drone that is obtained in time series by the first calculation unit and the second calculation unit. , Switching to manual control by the operator,
The drone control device according to claim 3 or 4, further comprising: At least one of the two or more arithmetic units is:
An operation of detecting whether or not a future estimated state of the unmanned aircraft obtained in a time series from the third arithmetic unit to a time point in the future from the real time indicates an avoidance state;
An operation of transmitting an emergency operation command to the drone to cause the drone to take an avoidance trajectory in preference to the operation command when the avoidance state is detected;
The drone control device according to claim 3 or 4, further comprising: The drone receives the actual measurement values obtained by sequentially measuring the state of the own machine from the drone, and the operator inputs the screen based on the sequentially displayed states while sequentially displaying the unmanned machine state on the screen. A method of operating a control device for remotely controlling the drone based on an operation command,
The first calculation unit and the second calculation unit included in the two or more calculation units in the control device, respectively, based on the control model reflecting the actual measurement value, the current estimated state of the drone from the operation command A first step of providing a display device with a current estimated state of the drone obtained by performing a time-series operation for obtaining
In the first step,
The first calculation unit and the second calculation unit alternately receive the actual measurement value, and each time the actual measurement value is received, repeatedly execute the time series calculation,
The first calculation unit is based on one control model reflecting one actually measured value, which is received alternately, until one time when the time series calculation catches up from the actually measured time of the one actually measured value to the current time. The time series operations are sequentially executed faster than the passage of real time,
Based on another control model reflecting other actually measured values received alternately, the second calculating unit is based on the time series calculation from the actually measured time of the other actually measured values to another time that catches up with the current time. The time series operations are sequentially executed faster than the passage of real time,
The first calculation unit provides the display device with a current estimated state of the drone from the one time to the next other time,
The second calculation unit provides the display device with a current estimated state of the drone from the other time to the next one time.
A method for controlling an unmanned aerial vehicle. The control model is configured to include a state estimation filter;
From the actual measurement time of the actual measurement value to the time of catching up with the current time, the time series calculation executed at a speed higher than the passage of real time includes a post-prediction operation that reflects the actual measurement value in the internal state of the state estimation filter. And a high-speed state prediction operation for performing the prediction operation of the state estimation filter based on the internal state reflecting the actual measurement value at a speed higher than the lapse of real time,
The process of providing the display device with the current estimated state of the drone continues the high-speed state prediction operation after the time-series operation executed faster than the passage of real time catches up with the current time Configured to switch to a pre-prediction operation that displays the time-series estimation state on the screen while performing the prediction operation of the state estimation filter in a form,
The method of controlling an unmanned aerial vehicle according to claim 8. The third calculation unit included in the two or more calculation units is in parallel with the time-series calculation by the first calculation unit and the second calculation unit,
Based on the one control model obtained by reflecting the one actual measurement value on the control model, a time-series operation for obtaining a current estimated state of the drone from the operation command in time series is performed. A step that is executed faster than the passage of real time from the time when the actual measurement value of the actual measurement is passed through to the future time point through the current time,
Run
After the time series computation by the third computation unit reaches the one future time point, the current estimated state of the unmanned aircraft obtained by the time series computation by the third computation unit is The drone control method according to claim 8 or 9, wherein the estimated state of the future can be provided to the display device. The fourth arithmetic unit included in the two or more arithmetic units is in parallel with the time-series arithmetic performed by the first arithmetic unit, the second arithmetic unit, and the third arithmetic unit,
Based on the other control model obtained by reflecting the other actual measurement value on the control model, a time series operation for obtaining a current estimated state of the drone from the operation command in time series is performed. A step that is executed faster than the passage of real time from the time when the actual measurement value of the actual measurement is passed through the current time to another future time point,
Run
The current estimated state of the unmanned aircraft obtained by the time-series computation by the fourth computation unit after the time-series computation by the fourth computation unit reaches the other future time point is determined as the future of the unmanned aircraft. The estimated state can be provided to the display device,
The third calculation unit provides the display device with an estimated future state of the drone from the one future time point to the next other future time point,
The fourth computing unit provides the display device with a future estimated state of the drone from the other future time point to the next future time point.
The method of controlling an unmanned aerial vehicle according to claim 10. At least one of the two or more arithmetic units is:
The third calculation unit and the fourth calculation unit are time-sequentially superimposed on a screen that displays the current estimated state of the drone obtained by the first calculation unit or the second calculation unit in time series. Displaying the estimated future state of the drone obtained on the screen in a form preceding the passage of real time,
The drone control method according to claim 11. At least one of the two or more arithmetic units is:
Prior to the start of remote control of the drone, generating an operation profile including a series of operation commands for the drone by repeating trial and error based on simulation; and
Unmanned control of the drone based on the operation profile;
Detecting whether or not the estimated future state of the drone obtained in a time series from the third computing unit to a future time point in real time indicates a avoidance state;
When the avoidance state is detected, the control of the drone is controlled based on a screen display that displays the current estimated state of the drone that is obtained in time series by the first calculation unit and the second calculation unit. , Switching to manual control by the operator,
The drone control method according to claim 10 or 11, further comprising: At least one of the two or more arithmetic units is:
Detecting whether or not a future estimated state of the drone obtained in a time series from the third computing unit to a time point in the future from the real time indicates an avoidance state;
A step of transmitting an emergency operation command to the drone to cause the drone to take an avoidance path in preference to the operation command when the avoidance state is detected;
The drone control method according to claim 10 or 11, further comprising:

Images