JP2007183332A - Operation training device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an operation training device for objectively and exactly evaluating student's skill in operation. <P>SOLUTION: A robot 20 moves in accordance with the operation of a teacher side joystick 12b. The real-time image of the robot 20 is displayed on a window in a monitor 14c for student ST. Guide information indicating the attitude of the teacher side joystick 12b and a student side joystick 14b is displayed in another window on the monitor 14c correlating with display processing of the real-time image of the robot 20. The proficiency of the student ST is calculated on the basis of the difference between attitudes of the teacher side joystick 12b and the student side joystick 14b. The calculated proficiency M is displayed on the monitor 14c and also transmitted to a teacher TC side communication terminal 12. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an operation training device, and more particularly to an operation training device that trains operation of a robot having a large number of degrees of freedom, for example.

An example of a conventional device of this type is disclosed in Patent Document 1. According to this prior art, the first handle is controlled by the instructor to move the medical instrument and the second handle is moved by the student to control the same instrument. Deviations between the movement of the handle by the instructor and the movement of the handle by the student are provided as force feedback on the handle of the student and the instructor. The student's hand is forced to match the instructor's hand movement through this force feedback.
JP 2005-515010 A

  However, the prior art does not disclose how to calculate the proficiency level of the student's operation skill.

    Therefore, a main object of the present invention is to provide an operation training apparatus capable of objectively and accurately evaluating the proficiency level of students.

  The operation training device (14) according to the invention of claim 1 operates the simulated operation device (14b) with control state information indicating the control state of the object (20) responding to the operation of the actual operation device (12b) by the teacher. First information output means (S5) for output to the student, guide information indicating the operation state of the actual operation device and the operation state of the simulated operation device for the student in relation to the output processing of the first information output means Second information output means for outputting (S11, S13), calculating means for calculating the proficiency level of the student based on the difference between the operating state of the actual operating device and the operating state of the simulated operating device (S15, S17, S19, S25, S31), and proficiency level output means (S33) for outputting the proficiency level calculated by the calculation means.

  The object responds to the operation of the actual operating device by the teacher. The control state information indicating the control state of the object is output by the first information output unit toward the student who operates the simulated operating device. The second information output means outputs operation guide information indicating the operation state of the actual operation device and the operation state of the simulated operation device to the student in association with the output process of the first information output device. The proficiency level of the student is calculated by the calculation means based on the difference between the operating state of the actual operating device and the operating state of the simulated operating device. The calculated proficiency level is output by the proficiency level output means.

  The proficiency level reflects the skill of the student who operates the simulated operating device using the control state information and the operation guide information output by the first information output means and the second information output means, respectively. This makes it possible to objectively and accurately evaluate the student's operation ability.

  The operation training apparatus according to the invention of claim 2 is dependent on claim 1, and the object is a robot.

  The operation training apparatus according to the invention of claim 3 is dependent on claim 1 or 2, wherein the operation state of each of the actual operation device and the simulated operation device is defined by the inclination direction and the inclination amount, and the calculation means includes: Discriminating means (S17) for discriminating whether or not the difference in inclination direction with the simulated operating device is within an allowable range, and when the discrimination result of the discriminating device is affirmative, the actual operating device and the simulated operating device Difference calculation means (S19) for calculating the difference in the amount of inclination between the counting means, counting means (S25) for counting the number of times the determination result of the determination means is negative, and the calculation result of the difference calculation means and the counting result of the counting means A proficiency level calculating means (S31) for calculating the proficiency level based on the above.

  The difference calculating means and the counting means are selectively activated based on the difference in inclination direction. The proficiency level is obtained based on the calculation result of the difference calculation means and the counting result of the counting means. In this way, the students' ability to operate is objectively and accurately evaluated.

  The operation training device according to the invention of claim 4 is dependent on claim 3, and when the discrimination result of the discrimination means is affirmative, the operation training device has an external force applied to the simulation operator to match the operation state of the simulation operator to the operation state of the actual operator. Further adding means (S23) for adding. Appropriate operation by the student is supported by the addition of external force by the adding means.

  The operation training apparatus according to the invention of claim 5 is dependent on claim 4, and the external force applied by the adding means differs depending on the magnitude of the difference calculated by the difference calculating means. This improves the operability of the student.

  The operation training program according to the invention of claim 6 is the control state information indicating the control state of the object (20) responding to the operation of the actual operation device (12b) by the teacher to the processor (14f) of the operation training device (14). First information output step (S5) for outputting to the student who operates the simulated operating device (14b), and first information output step for operating guide information indicating the operating state of the actual operating device and the operating state of the simulated operating device. Second information output step (S11, S13) that is output to the student in relation to the output processing of the computer, and the proficiency level of the student is calculated based on the difference between the operating state of the actual operating device and the operating state of the simulated operating device It is an operation training program for executing a calculation step (S15, S17, S19, S25, S31) and a proficiency output step (S33) for outputting the proficiency level calculated in the calculation step.

  In the operation training method according to the invention of claim 7, (a) the control state information indicating the control state of the object responding to the operation of the actual operation device by the teacher is output to the student who operates the simulated operation device. ) Operation guide information indicating the operation state of the actual operation device and the operation state of the simulated operation device is output to the student in connection with the output process of step (a), and (c) the operation state and simulation of the actual operation device. This is an operation training method that calculates the proficiency level of the student based on the difference from the operating state of the operating device, and (d) outputs the proficiency level calculated in step (c).

  According to the present invention, the proficiency level reflects the skill of the student who operates the simulated operating device using the control state information and the operation guide information as clues. This makes it possible to objectively and accurately evaluate the student's operation ability.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  Referring to FIG. 1, a training system 10 of this embodiment includes a communication terminal 12 prepared for a teacher TC, and a plurality of communication terminals 14 prepared for a plurality of students ST, ST,. 14 and so on. All of these communication terminals 12, 14, 14,... Are connected to the public Internet network 16. A server 18, a robot 20 and a camera 22 are also connected to the public internet network 16.

  For example, the server 18 and the robot 20 are installed in a company that provides this training service, and the communication terminals 12, 14, 14,... Are installed in a school. Here, the communication terminal 12 for the teacher TC may be installed in the same room as the communication terminal 14 for the student ST, or may be installed in a different room. Further, the function of the server 18 may be integrated into the robot 20.

  As shown in FIG. 2, the teacher TC communication terminal 12 includes a speaker / microphone 12a, a teacher joystick 12b, a monitor 12c, a keyboard 12d, a memory 12e, a CPU 12f, and an I / F 12g. As shown in FIG. 3, the student ST communication terminal 14 also includes a speaker / microphone 14a, a student joystick 14b, a monitor 14c, a keyboard 14d, a memory 14e, a CPU 14f, and an I / F 14g. As shown in FIG. 4, the server 18 includes a memory 18a, a CPU 18b, and an I / F 18c.

  The external appearance of the teacher joystick 12b or the student joystick 14b is shown in FIGS. 5 (A) and 5 (B). The stick STK mounted at the center of the upper surface of the substrate BS can freely tilt up, down, left and right. The inclination direction is represented by an angle on a plane (see FIG. 5A) in which the right direction, the upward direction, the left direction, and the downward direction are 0 °, 90 °, 180 °, and 270 °, respectively. The amount of inclination is represented by an angle on a plane (see FIG. 5B) in which the axis orthogonal to the upper surface of the substrate is 0 ° and the axis parallel to the upper surface of the substrate is 90 °. The posture of the stick STK, that is, the posture of the teacher-side joystick 12b or the student-side joystick 14b is specified by such an inclination direction and an inclination amount.

  The teacher-side joystick 12b is a normal control stick for operating the robot 20, and the student-side joystick 14b is a simulated control stick for training. Moreover, SideWinder Force Feedback 2 of Microsoft Corporation is used as the teacher side joystick 12b and the student side joystick 14b.

  With reference to FIGS. 1 to 5 described above and FIG. 6 showing the exchange among the communication terminal 12, the server 18 and the communication terminal 14, the robot 20 is photographed by the camera 22, and the image of the robot 22 photographed is taken. Data (control state information) is provided to the server 18 via the public Internet network 16. The CPU 18b of the server 18 takes in the given image data from the I / F 18c and writes the taken image data into the memory 18a. The CPU 18b also distributes the image data stored in the memory 18a to each of the communication terminals 12, 14, 14,... Via the I / F 18c and the public Internet network 16.

  In the communication terminal 12 on the teacher TC side, the CPU 12f fetches the given image data from the I / F 12g and writes the fetched image data into the memory 12e. The CPU 12f also gives the image data stored in the memory 12e to the monitor 12c. As a result, an image of the robot 22 is displayed on the monitor 12c.

  In the communication terminal 14 on the student ST side, the CPU 14f fetches the given image data from the I / F 14g and writes the fetched image data into the memory 14e. The CPU 14f also gives the image data stored in the memory 14e to the monitor 14c. As a result, a window W1 on which a real-time image of the robot 22 is displayed is formed on the screen as shown in FIG.

  The CPU 12f of the teacher TC communication terminal 12 repeatedly transmits posture information (operation state information) indicating the posture of the teacher joystick 12b operated by the teacher TC to the server 18 via the I / F 12g and the public Internet network 16. To do.

  The CPU 18b of the server 18 takes in the posture information transmitted from the communication terminal 12 for teacher TC via the I / F 18c, and creates control data corresponding thereto. The created control data is given to the robot 20 via the I / F 18c and the public Internet network 16. The robot 20 shows a movement corresponding to the control data. That is, the control state of the robot 20 changes in response to the operation of the teacher joystick 12b by the teacher TC.

  The CPU 18b of the server 18 also distributes the posture information given from the communication terminal 12 for the teacher TC to each communication terminal 14, 14,... For the student ST via the I / F 18c and the public Internet network 16, and the posture information. Is taken into the memory 14e via the I / F 14g. The CPU 14f displays a character Ct indicating the posture of the teacher joystick 12b on the monitor 14c based on the taken posture information. The CPU 14f also displays a character Cs indicating the posture of the student joystick 14b on the monitor 14c.

  When the teacher joystick 12b is tilted upward by a certain angle and the student joystick 14b is tilted upward by an angle smaller than the angle of the teacher joystick 12b, the characters Ct and Cs are shown in FIG. In this way, it is displayed in the window W2 on the screen.

  Subsequently, the CPU 14f detects a difference in inclination direction between the teacher-side joystick 12b and the student-side joystick 14b, and determines whether or not the detected difference is within an allowable range (for example, ± 10 °). If within the allowable range, the CPU 14f calculates a difference in the amount of inclination between the teacher-side joystick 12b and the student-side joystick 14b, and applies an external force having a magnitude corresponding to the calculated difference to the student-side joystick 14b. . The external force is applied so that the posture of the student joystick 14b matches the posture of the teacher joystick 12b. Further, difference information D indicating the absolute value of the calculated difference is accumulated in the memory 14e. On the other hand, if the difference in the tilt direction is outside the allowable range, the CPU 14f increments the variable K indicating the number of times that the determination result is obtained, and cancels the application of the external force to the student-side joystick 14b.

  The above-described processing is repeatedly executed every time posture information is fetched from the teacher TC side. As a result, the difference information D is accumulated in the memory 14e or the variable K is incremented according to the difference in operation direction between the teacher joystick 12b and the student joystick 14b.

  When the inclination amount of the teacher joystick 12b and the inclination amount of the student joystick 14b change as shown in FIG. 8 (inclination directions match), the difference information D is repeatedly detected as shown in FIG. 14e.

When the training is completed, the CPU 14f calculates a proficiency level M for the operation of the student ST based on the difference information D and the variable K stored in the memory 14e. Specifically, the calculation according to Equation 1 is executed.
[Equation 1]
M = α−β (ΣD + γK)
α, β, γ: constant According to Equation 1, the integrated value of the difference information D and the multiplied value obtained by multiplying the variable K by the constant γ are added to each other, and the multiplied value and the constant β are subtracted from the constant α. Is done. The proficiency level M is obtained by such a deduction method. Therefore, the less the number of times the difference in the tilt direction deviates from the allowable range, and the smaller the difference in the tilt amount, the higher the proficiency level M is. The CPU 14f displays the proficiency level M thus calculated on the monitor 14c and transmits it to the communication terminal 12 of the teacher TC. The proficiency level M is also displayed on the monitor 12c of the communication terminal 12, and the teacher TC can recognize the proficiency level of the student ST based on the displayed proficiency level M.

  The start / end of training is transmitted by a call using the speakers / microphones 12a and 14a.

  The CPU 14f executes processing according to the flowcharts shown in FIGS. A control program corresponding to this flowchart is stored in the memory 14e.

  In step S 1, the variable K is set to “0”. In subsequent step S 3, the image data of the robot 20 is fetched from the server 18. In step S5, a real-time image of the robot 20 based on the captured image data is displayed on the window W1 formed on the monitor 14c.

  In step S7, posture information indicating the posture of the teacher joystick 12b for the teacher TC is fetched from the server 18, and in step S9, the posture of the student joystick 14b is detected. In step S11, based on the posture information captured in step S7, the character Ct representing the posture of the teacher joystick 12b is displayed in the window W2 formed on the monitor 14c. In step S13, the character Cs representing the posture of the student joystick 14b detected in step S9 is displayed on the window W2 formed on the monitor 14c.

  In step S15, the difference in inclination direction between the teacher joystick 12b and the student joystick 14b is calculated based on the posture information captured in step S7 and the posture detected in step S9. In step S17, it is determined whether or not the calculated difference is within an allowable range. If YES, the process proceeds to step S29 through steps S19 to S23. If NO, the process proceeds to steps S25 to 27 and step S29. Proceed to

  In step S19, the difference in the amount of tilt between the teacher joystick 12b and the student joystick 14b is calculated. In step S21, difference information D indicating the absolute value of the calculated difference is accumulated in the memory 12e. In step S23, an external force having a magnitude corresponding to the calculated difference in the tilt amount is applied to the student joystick 14b so that the attitude of the student joystick 14b matches the attitude of the teacher joystick 12b. The added external force is increased as the difference in the amount of inclination is increased.

  On the other hand, in step S25, the variable K is incremented, and in step S27, the external force applied to the student joystick 14b is released. By releasing the application of the external force, the student-side joystick 14b is prevented from being damaged due to the external force being applied in a state where the posture of the student-side joystick 14b is significantly different from the posture of the teacher-side joystick 12b. Further, the erroneous operation as described above is reflected in the proficiency level M by incrementing the variable K. The training may be forcibly terminated when it is determined NO in step S17.

  In step S29, it is determined whether or not the training is completed. If NO, the process returns to step S3, whereas if YES, the process proceeds to step S31. In step S31, the proficiency level M is calculated according to the above equation 1 based on the difference information D and the variable K stored in the memory 14e. In step S33, the calculated proficiency level M is displayed on the monitor 12c and transmitted to the communication terminal 12 on the teacher TC side. Thereafter, the process ends.

  As can be understood from the above description, the robot 20 shows a movement corresponding to the operation of the teacher-side joystick (actual operating device) 12b. The real-time image (control state information) of the robot 20 is displayed on the window W1 on the monitor 14c (S5). Characters Ct and Cs (operation guide information) respectively indicating the posture (operation state) of the teacher-side joystick 12b and the posture (operation state) of the student-side joystick 14b (simulated operation device) are used for real-time image display processing of the robot 20. Relatedly, it is displayed on the window W2 on the monitor 14c. Both windows W1 and W2 are displayed toward the student ST. The proficiency level M of the student ST is calculated based on the difference in posture between the teacher joystick 12b and the student joystick 14b (S31). The calculated proficiency level M is output from the monitor 14c (S33).

  The proficiency level M reflects the skill of the student ST who operates the student joystick 14b using the real-time image of the robot 20 displayed in the window W1 and the characters Ct and Cs displayed in the window W2. Thereby, the operation ability of the student ST can be objectively and accurately evaluated.

  In this embodiment, an image (appearance) of the robot 20 captured by the camera 22 is assumed as the control state information. However, as the control state information, surrounding images captured from the robot's line of sight, reactions of switches (such as touch switches) provided on the robot, voices emitted from the robot, and the like are also conceivable.

Further, in this embodiment, the above equation 1 is used for calculating the proficiency level. However, the following formula 2 may be used instead. That is, as long as the proficiency level is calculated using the difference between the operation of the teacher and the student, any mathematical formula can be considered.
[Equation 2]
M = α / (α + ΣKD)
Further, in this embodiment, a joystick is used as an operating device for operating the robot, but a mouse or a keyboard may be used instead of the joystick. However, in this case, it is necessary to calculate the proficiency level based on a wrong button press or a delay time until a proper button is pressed.

It is an illustration figure which shows the structure of one Example of this invention. It is a block diagram which shows an example of a structure of the communication terminal for teachers applied to the FIG. 1 Example. It is a block diagram which shows an example of a structure of the communication terminal for students applied to the FIG. 1 Example. It is a block diagram which shows an example of a structure of the server applied to the FIG. 1 Example. (A) is a top view showing an appearance of a joystick applied to the embodiment of FIG. 2 or FIG. 3, and (B) is a side view showing an appearance of a joystick applied to the embodiment of FIG. 2 or FIG. 3. FIG. It is an illustration figure which shows a part of operation | movement of FIG. 1 Example. It is an illustration figure which shows an example of the image displayed on the monitor applied to the FIG. 3 Example. It is a graph which shows a part of operation | movement of FIG. 1 Example. FIG. 4 is a flowchart showing one portion of behavior of a CPU applied to the embodiment in FIG. 3. FIG. 11 is a flowchart showing another portion of behavior of the CPU applied to the embodiment in FIG. 3;

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Training system 12, 14 ... Communication terminal 18 ... Server 20 ... Robot 22 ... Camera

Claims (7)

First information output means for outputting control state information indicating a control state of an object in response to an operation of an actual operation device by a teacher toward a student who operates the simulated operation device;
Second information output means for outputting operation guide information indicating an operation state of the actual operation device and an operation state of the simulated operation device to the student in association with an output process of the first information output device;
Calculating means for calculating the proficiency level of the student based on the difference between the operating state of the actual operating device and the operating state of the simulated operating device; and a proficiency level output means for outputting the proficiency level calculated by the calculating means. An operation training device is provided.   The operation training apparatus according to claim 1, wherein the object is a robot. The operation state of each of the actual operation device and the simulated operation device is defined by an inclination direction and an inclination amount,
The calculating means is a determining means for determining whether or not a difference in inclination direction between the actual operating device and the simulated operating device is within an allowable range, and when the determination result of the determining device is affirmative Difference calculating means for calculating the difference in the amount of inclination between the actual operating device and the simulated operating device, counting means for counting the number of times the determination result of the determining means is negative, and calculation of the difference calculating means The operation training apparatus according to claim 1, further comprising a proficiency level calculating unit that calculates the proficiency level based on a result and a counting result of the counting unit.   4. The apparatus according to claim 3, further comprising an adding unit configured to apply an external force to the simulated operating device in order to match the operation state of the simulated operating device with the operating state of the actual operating device when the determination result of the determining unit is positive. Operation training device.   The operation training apparatus according to claim 4, wherein the external force applied by the adding unit is different depending on the magnitude of the difference calculated by the difference calculating unit. To the processor of the operation training device,
A first information output step for outputting control state information indicating a control state of an object in response to an operation of an actual operation device by a teacher toward a student who operates the simulated operation device;
A second information output step of outputting operation guide information indicating an operation state of the actual operation device and an operation state of the simulated operation device to the student in association with an output process of the first information output step;
A calculation step of calculating a proficiency level of the student based on a difference between an operation state of the actual operation device and an operation state of the simulated operation device, and a proficiency level output step of outputting the proficiency level calculated by the calculation step Operation training program to be executed. (a) Outputs control state information indicating the control state of the object responding to the operation of the actual operation device by the teacher to the student who operates the simulated operation device,
(b) Output operation guide information indicating the operation state of the actual operation device and the operation state of the simulated operation device to the student in association with the output process of the step (a),
(c) calculating the proficiency level of the student based on the difference between the operating state of the actual operating device and the operating state of the simulated operating device; and
(d) An operation training method for outputting the proficiency level calculated in step (c).

Images