JP2006272537A - Robot simulator and simulation program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a robot simulator and a simulation program enabling a developer to develop a robot off-line while saving extra time and labor. <P>SOLUTION: The robot simulator 100 includes a computer 102. The computer 102 displays a model of the robot on a touch screen 104. If the developer touches the model through the touch screen 104, the model reenacts communication action (reaction) previously made to correspond to the touched part. For example, if the head of the model is stroked, the model is displayed by animation so as to wave its hand after uttering "What?". By this way, it is possible to develop the robot off-line. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a robot simulator and a simulation program, and more particularly, to a robot simulator and a simulation program that support development of a communication robot that performs communication behavior including at least body motion.

For example, Patent Document 1 by the present applicant discloses a technique for visualizing the relationship between action modules, which are programs for executing individual actions provided in a communication robot. With this technology, the relationship between behavior modules can be intuitively grasped, so that, for example, rules for autonomous behavior can be appropriately created and edited. Further, after that, a test for actually operating the robot can be performed to confirm the suitability of the editing.
JP 2004-114242 A

  In this background art, it is necessary to actually operate a robot and confirm the suitability of editing or the like. However, such a robot is relatively expensive and is assigned to a plurality of developers. Therefore, in order to carry out such a test, a robot is used alternately by a plurality of developers in a predetermined time schedule through discussion. For this reason, it is necessary to perform a test in a limited time, development does not proceed quickly, and the purpose of use is different for each developer, so it is necessary to rewrite the program every time it is used. In other words, it took extra time for development.

  Therefore, a main object of the present invention is to provide a novel robot simulator and simulation program.

  Another object of the present invention is to provide a robot simulator and a simulation program that can be developed off-line while eliminating unnecessary work.

  The invention of claim 1 is a robot simulator of a communication robot that executes communication behavior including at least a body motion, a screen display means for displaying a model of the communication robot on a screen, a pointing device provided in association with the screen display means, And a robot simulator comprising animation display means for displaying an animation of the body motion of the model in response to an input from a pointing device.

  In the first aspect of the invention, the robot simulator performs a simulation (test) on the communication behavior of a communication robot that executes a communication behavior including at least a body motion such as gesture gesture. The robot simulator includes an image display unit, a pointing device, and an animation display unit. The image display means displays the communication robot model (CG) on the screen. The pointing device is provided in association with the screen display means, and the body motion of the model is displayed in an animation according to the input of the pointing device. That is, the reaction behavior of the communication robot is reproduced in the model by simulation.

  According to the first aspect of the present invention, the model of the communication robot is displayed on the screen, and the communication behavior according to the operation of the pointing device is reproduced by the animation display of the model. Therefore, the robot can be used offline without using the actual communication robot. Can be developed. For this reason, it is possible to save extra time for development.

  The invention of claim 2 is dependent on claim 1, and the communication behavior includes speech, and further includes an output means for displaying the animation of the body motion of the model and outputting the content of the speech accompanying the body motion.

  In the invention of claim 2, the communication behavior includes utterance as necessary. The output means displays the animation of the body motion of the model and outputs the contents accompanying the body motion.

  According to the invention of claim 2, since not only the body movement of the communication robot but also the utterance content can be known, it is possible to understand a more realistic reaction of the robot.

  The invention of claim 3 is a simulation program for a communication robot that executes communication behavior including at least body movements, a screen display step for displaying a model of the communication robot on a screen, and an instruction for the model displayed on the screen by the screen display step It is a simulation program for executing an animation display step for displaying an animation of the body motion of the model in response to an input.

  In the invention of claim 3 as well, as in the invention of claim 1, it is possible to save time and labor for development.

  According to the present invention, since the model of the communication robot displayed on the screen is displayed in an animation according to the operation of the pointing device, the communication behavior can be expressed without using the actual communication robot. In other words, it can be developed offline with no extra effort.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  Referring to FIG. 1, a robot simulator (hereinafter simply referred to as “simulator”) 100 of this embodiment includes a computer 102. The computer 102 is a general-purpose computer such as a personal computer or a workstation. Therefore, although not shown, the computer 102 includes components such as a CPU, a memory (hard disk, ROM, RAM) and an image processing unit (including a frame buffer and a Z buffer). Although illustration is omitted, the memory stores a program (simulation program) and data (image data such as animation data) necessary for executing a simulation process (see FIGS. 6 and 7) described later. The A general-purpose touch screen 104 and a speaker 106 are connected to the computer 102.

  In this embodiment, the touch screen 14 is provided to give a feeling that the communication robot 10 (see FIG. 2) is directly operated. However, a display device such as a CRT monitor or LCD may be connected to the computer 102 to provide another pointing device such as a touch pad, tablet, or computer mouse. In such a case, an instruction image such as a mouse pointer is displayed on the display, and a feeling that the communication robot 10 is operated indirectly can be obtained.

  In the simulator 100 configured as described above, the computer 102 displays, for example, a model (CG: Computer Graphics) of a communication robot (hereinafter simply referred to as “robot”) 10 configured as illustrated in FIG. To do. However, it is not necessary to be limited to the robot 10 as shown in FIG. 2, and any robot model can be used. When the developer of the robot 10 touches the model through the touch screen 104, the model expresses (reproduces) the communication behavior corresponding to the model. Here, the communication action refers to a body movement such as gesture gesture of the robot 10 and may include utterance. By this communication action, the robot 10 interacts (communicates) with a human, for example. Specifically, as a reaction of the robot 10, the model on the touch screen 104 is displayed as a moving image (animation display), or the sound of the content spoken by the model is output from the speaker 106.

  FIG. 2 is an external view of the robot 10, and the robot 10 includes a carriage 12 with reference to FIG. 2. Wheels 14 for autonomously moving the robot 10 are provided on the side surface of the carriage 12. The wheel 14 is driven by a wheel motor (indicated by reference numeral “16” in FIG. 3), and the carriage 12, that is, the robot 10 can be moved in any direction. Although illustration is omitted, a collision sensor is attached to the front surface of the carriage 12, and the collision sensor detects contact of a person and other obstacles to the carriage 12.

  On the carriage 12, the human body-like portion 18 is attached so as to stand upright. The whole body of the human body 18 as the robot body is covered with skin 20 made of a flexible material. The human body 18 includes a housing (not shown) such as an iron plate, for example, and houses a computer and other necessary components in the housing. Then, the skin 20 is put on the casing. A microphone 22 is provided at approximately the center of the upper part of the housing under the skin 20. The microphone 22 is for collecting ambient sounds, particularly human voices.

  The human body portion 18 includes a right arm 24R and a left arm 24L, and the right arm 24R and the left arm 24L, that is, the upper arms 26R and 26L are detachably attached to the trunk portion by shoulder joints 28R and 28L, respectively. The shoulder joints 28R and 28L have three degrees of freedom. Forearms 32R and 32L are attached to upper arms 26R and 26L by uniaxial elbow joints 30R and 30L, and hands 34R and 34L are attached to these forearms 32R and 32L. Each axis in each joint of the right arm 24R and the left arm 24L is controlled by a motor (not shown). That is, the four motors of the right arm 24R and the left arm 24L are represented as the right arm motor 36 and the left arm motor 38, respectively, in FIG.

  A head 42 is attached to the upper portion of the human body 18 via a neck joint 40 so as to be able to be elevated and rotated like a human head. The three-axis neck joint 40 is controlled by a head motor 44 shown in FIG. Two eye cameras 46 are provided at positions corresponding to the “eyes” on the front surface of the head 42, and these eye cameras 46 photograph a human face and other parts approaching the robot 10 and output the video signals thereof. take in. A speaker 48 is provided below the eye camera 46 provided on the front surface of the head 42. The speaker 48 is used by the robot 10 to communicate with a person around it by voice (utterance).

  The body, head 42 and arms of the human body 18 described above are all covered with the skin 20 made of a flexible material as described above. Although illustration is omitted, skin 20 includes a lower urethane foam, a relatively thick silicon rubber layer and a relatively thin silicon rubber layer laminated thereon. A piezo sensor sheet is embedded between the two silicon rubber layers.

  The specific configuration of the skin 20 and the detailed description thereof are disclosed in Japanese Patent Application Laid-Open No. 2004-283975 filed and published by the present applicant. I will omit it.

  FIG. 3 is a block diagram showing an electrical configuration of the robot 10 shown in FIG. Referring to FIG. 3, the robot 10 includes a microcomputer or CPU 56 that is responsible for overall control thereof, and this CPU 56 is connected to a memory 58, a motor control board 62, a sensor input / output board 64, and a sound input / output through a bus 58. An output board 66 is connected.

  Although not shown, the memory 60 includes a ROM, an HDD, and a RAM. The ROM and HDD are pre-stored with a control program for the robot 10, and voice or voice data to be generated from the speaker 48, predetermined data The angle control data of each joint axis for performing gesture gesture (body movement) is stored. The RAM is used as a temporary storage memory and a working memory.

  The motor control board 62 is constituted by a DSP (Digital Signal Processor), for example, and controls each axis motor such as each arm and head. That is, the motor control board 62 receives control data from the CPU 56, and controls three motors for controlling the angles of the three axes of the right shoulder joint 28R and one motor for controlling the angles of one axis of the right elbow joint 30R. The rotation angles of the four motors (collectively shown as “right arm motor” in FIG. 3) 36 are adjusted. Further, the motor control board 62 adjusts the rotation angle of a total of four motors (collectively shown as “left arm motor” in FIG. 3) 38, three axes of the left shoulder joint 28L and one axis of the left elbow joint 30L. The motor control board 62 also adjusts the rotation angle of the three-axis motor 44 of the head 42 (collectively shown as “head motor” in FIG. 3). The motor control board 62 controls two motors 16 that collectively drive the wheels 14 (collectively shown as “wheel motors” in FIG. 3).

  The above-described motors of this embodiment are stepping motors or pulse motors for simplifying the control except for the wheel motors 16, but may be direct-current motors similarly to the wheel motors 16.

  Similarly, the sensor input / output board 64 is configured by a DSP, and takes in signals from each sensor and camera and gives them to the CPU 56. That is, data relating to contact from each of the collision sensors (not shown) is input to the CPU 56 through the sensor input / output board 64. The video signal from the eye camera 46 is input to the CPU 56 after being subjected to predetermined processing by the sensor input / output board 64 as necessary. Further, when a human or the like touches the skin 20 of the robot 10, data relating to contact (touch, stroke, etc.) from the piezo sensor 54 provided therein is input to the CPU 56 through the sensor input / output board 64.

  Similarly, the sound input / output board 66 is configured by a DSP. The speaker 48 is provided with synthesized voice data from the CPU 56 via the sound input / output board 66, and in response to this, the speaker 48 outputs voice or voice according to the data. Further, the voice input from the microphone 22 is taken into the CPU 56 via the sound input / output board 66.

  For example, in the robot 10 having such a configuration, as described above, interaction (communication) is performed with a human by a body motion and a necessary voice. Further, when a human touches the robot 10, a communication action corresponding to the touched part (part) is executed. That is, the robot 10 reacts. Specifically, when a human strokes the shoulder of the robot 10, the robot 10 utters “what” and shakes his / her hand accordingly. When a human strokes the head of the robot 10, the robot 10 utters “Wai” and shakes the head (40) in response. Further, when the human strokes the stomach of the robot 10, the robot 10 speaks “Let's play” and raises both hands. However, these are merely examples, and need not be limited. The reaction of the robot 10 can be freely determined (set) by the developer.

  Here, in general, when the robot 10 having such a configuration or another robot is developed, or the desired communication behavior (reaction) is realized (reproduced) by the developed robot 10 or the like, it is created. The control program (program for communication behavior) is mounted on the robot 10 and actually interacts with the robot 10 to test whether the communication behavior according to the control program is as intended (appropriate).

  However, such a robot 10 is relatively expensive and is assigned to a plurality of developers. Therefore, the plurality of developers are used alternately in a predetermined time schedule through discussion. For this reason, it is necessary to perform a test in a limited time, development does not proceed quickly, and the purpose of use is different for each developer, so it is necessary to rewrite the control program every time it is used. In other words, it takes extra time for development.

  Therefore, in this embodiment, by proposing a simulator 100 that can test (simulate) the communication behavior of the robot 10 offline, it is possible to perform offline development and to save unnecessary time and effort for development.

  The control program as described above is created by a developer and incorporated in the computer 102 shown in FIG. Specifically, animation data about the body movements included in the communication behavior is stored in the internal memory (HDD, ROM, or RAM) of the computer 102 so that the model of the robot 10 expresses the communication behavior according to the control program. . For example, the model is displayed on the touch screen 104 using image data such as frame data (bone data), polygon data, and texture data. The animation data (motion data) is data indicating a time-series change in the position of a joint (joint: a connection part between bones) and the position of a bone connected to the joint. Here, the joint corresponds to each joint (28R, 28L, 30R, 30L, 40) of the robot 10 shown in FIG. In addition, voice data (speech synthesized data) about the content of the utterance included in the communication action is also stored in the internal memory of the computer 102.

  However, as will be described later, the control program executes a process so as to express a predetermined communication behavior in accordance with the part of the model touched through the touch screen 104. That is, the model is made to react to a human (developer) touch operation.

  The computer 102 displays a simulation screen 200 as shown in FIG. 4 on the touch screen 104. The simulation screen 200 displays the model 202 of the robot 10 using the image data as described above. The simulation screen 200 also displays a button (or icon) 204 for instructing the viewpoint conversion of the model 202.

  For example, the simulation screen 200 illustrated in FIG. 4 is an image of a three-dimensional virtual space including the model 202 illustrated in FIG. 5 viewed from the viewpoint (taken with a virtual camera). In FIG. 5, each axis of the coordinate system of the three-dimensional virtual space (hereinafter sometimes referred to as a robot coordinate system) is (Px, Py, Pz), and each axis of the virtual screen 210 is (Sx, Sy, Sz). It is as. However, the Sz axis of the virtual screen 210 is a direction perpendicular to the virtual screen 210 and represents its depth. Further, although it is difficult to understand in FIG. 5, the model 202 arranged in the three-dimensional virtual space has a line-of-sight direction in the Px axis direction and a viewpoint in the X point on the Px axis.

  As described above, the developer of the robot 10 tests (simulates) the communication behavior of the robot 10 using the simulator 100 as shown in FIG. Specifically, the model 202 is displayed on the simulation screen 200, and the developer touches a desired part of the model 202. That is, the touch screen 104 is touched. Then, touch position (coordinate position) data (coordinate position data) is input from the touch screen 104 to the computer 102 in accordance with the touch operation. When the computer 102 receives the input of the coordinate position data, the coordinate (position) indicated by the coordinate position data indicates which part of the model 202 or the button 204 or other than both Determine whether to instruct.

  When any part of the model 202 is touched, the computer 102 causes the model 202 to execute a communication action defined (determined) corresponding to the touched part. That is, the model 202 is displayed as an animation, and if necessary, the voice synthesis data is reproduced and the voice or voice is output from the speaker 106 so that the model 202 speaks.

  When the button 204 is touched, the viewpoint conversion process is performed in accordance with the subsequent developer operation. Although illustration is omitted, in the simulation screen 200, the viewpoint is changed every time the viewpoint conversion button 204 is turned on (touched), and the model 202 of the robot 10 is in front of the right side of the developer viewing the touch screen 104. The display is changed so that it can be seen in the order of surface, back surface, left side surface, top surface, bottom surface, front surface,. However, the developer may be able to set the viewpoint at an arbitrary position. In such a case, for example, after the viewpoint conversion button 204 is operated, the touch screen 104 can be touch-inputted with a desired position as the viewpoint, or numerical values of three-dimensional coordinates are input with an input device (keyboard or the like) not shown. Or you can. Thus, by changing the viewpoint, the reaction of the model 202 can be seen from different angles (directions).

  However, when neither the model 202 nor the button 204 is touched, for example, the model 202 is caused to execute a communication action that prompts the developer to perform a touch operation. Here, the case where neither the model 202 nor the button 204 is touched means that the developer does not perform a touch operation or the developer touches an area other than the display area of the model 202 and the button 204. If you are.

  Specifically, the computer 102 shown in FIG. 1 executes the simulation process shown in FIGS. As shown in FIG. 6, when the computer 102 starts the simulation process, it initializes in step S1. Here, the model 202 is placed at a predetermined three-dimensional position (referred to as “first initial position” for convenience of description), and the viewpoint of the model 202 is set at a predetermined three-dimensional position, here, the first initial position. Is set to a second initial position different from. Here, each joint angle of the model 202 is set to an initial value. For example, at the start of simulation, each joint angle is set so that the model 202 is in an upright state (careful posture).

  In the next step S <b> 3, the model 202 is displayed on the touch screen 104 based on the joint angle and viewpoint of the current model 202. As described above, when the initialization is performed in step S1, the simulation screen 200 in which the upright model 202 faces the front as shown in FIG. Is displayed.

  Subsequently, in step S5, it is determined whether or not the touch has been made. That is, it is determined whether coordinate data (touch coordinates) is input from the touch screen 104. If “NO” in the step S5, that is, if the touch coordinates are not input, it is determined that the touch is not performed, and the process proceeds to a step S19 shown in FIG. On the other hand, if “YES” in the step S5, that is, if touch coordinates are inputted, it is determined that the touch has been made, and whether or not the viewpoint conversion is performed is determined in a step S7. That is, it is determined whether or not the touch coordinates are included in the display area of the button 204. Here, if “YES”, that is, if the touch coordinates are included in the display area of the button 204, it is determined that the viewpoint is changed, and the process proceeds to step S21 shown in FIG. However, if “NO”, that is, if the touch coordinates are not included in the display area of the button 204, it is determined that the viewpoint conversion is not performed, and the touch position (touch coordinates) is set to the robot in step S9 as described later. (See FIG. 8).

  In the next step S11, it is determined whether or not the shoulder of the model 202 has been touched. If “YES” in the step S11, that is, if the shoulder of the model 202 is touched, in a step S13, the model 202 speaks “what”, shakes his hand, and proceeds to a step S23 shown in FIG. Specifically, in step S <b> 13, the computer 102 reads out audio data corresponding to “NANI”, performs analog conversion, amplification, and the like, and then outputs them from the speaker 106. At the same time or almost the same time, the computer 102 reads out the animation data about “waving hands” and displays the animation of the model 202 according to the animation data. An image (video) viewed from the viewpoint is displayed on the touch screen 104 as a simulation screen 200. Thus, the model 202 expresses the reaction of the robot 10 to the developer's touch operation. The same applies hereinafter.

  However, if “NO” in the step S11, that is, if the shoulder of the model 202 is not touched, it is determined whether or not the head of the model 202 is touched in a step S15. If “YES” in the step S15, that is, if the head of the model 202 is touched, in a step S17, the model 202 speaks “Wai”, swings his head, and proceeds to a step S23. On the other hand, if “NO” in the step S15, that is, if the head of the model 202 is not touched, the illustration is omitted, but other parts, for example, left (or right) hand or arm, trunk (belly) , Back) or the like is determined, and a reaction, that is, a communication action defined (determined) corresponding to the touched part is executed. However, when the developer touch position is outside the display area of the model 202 and the button 204, it is determined that there is no corresponding part, and the process proceeds to step S19.

  As described above, the touch screen 104 is not touched (“NO” in step S5), or the touch position is outside the display area of the model 202 and the button 204 (“NO” in steps S11, S15,...). In such a case, as shown in FIG. 7, in step S19, the model 202 speaks “Please touch”, presents the hand, and proceeds to step S23. Also, as described above, when viewpoint conversion is instructed (“YES” in step S7), the viewpoint is updated (converted) in accordance with the developer's instruction in step S21 shown in FIG. 7, and the process proceeds to step S23.

  In step S23, it is determined whether or not the simulation is finished. For example, it is determined whether or not the developer has instructed the computer 102 to end the simulation. If “NO” in the step S23, that is, if the simulation is not finished, the process returns to the step S3 shown in FIG. However, if “YES” in the step S23, that is, if the simulation is finished, the simulation process is finished.

  FIG. 8 is a flowchart showing the screen display process shown in step S3 of FIG. Referring to FIG. 8, when the computer 102 starts the screen display process, in step S31, the computer 102 calculates the three-dimensional coordinates of each joint of the model 202 from the joint angle of the current model 202 and the size of each part. . In the subsequent step S33, the coordinates of each part are converted into coordinates on the virtual screen 210. Specifically, each pixel to be displayed on the screen is stored in the frame buffer, and the depth value (Z value) of each pixel is stored in the Z buffer. Next, in step S35, the virtual screen 210 is drawn in order from the rear part. That is, the model 202 is drawn according to the Z buffer method. In step S37, the corresponding part (part name or identification information thereof) for each Z value stored in the Z buffer is stored in a table (hereinafter referred to as “corresponding table”) not shown. To return to the screen display process.

  FIG. 9 is a flowchart showing the robot coordinate conversion process shown in step S9 of FIG. Referring to FIG. 9, when the computer 102 starts the robot coordinate conversion process, in step S41, with reference to the correspondence table, the touched point (of model 202 displayed on screen 210 touched from the touch coordinate ( Sx, Sy, Sz) is acquired. Here, based on the touch coordinates, (Sx, Sy) is acquired from the frame buffer, and Sz is acquired from the Z buffer. In the next step S43, it is determined whether there is a corresponding point. Specifically, it is determined whether or not a part is stored corresponding to Sz included in the acquired point (Sx, Sy, Sz). Thereby, it is determined whether or not the touch coordinates are on the model 202. If “NO” in the step S43, that is, if there is no corresponding point, it is determined that the model 202 is not touched, and the robot coordinate conversion process is directly returned. However, if “YES” in the step S43, that is, if there is a corresponding point, it is determined that the model 202 is touched, and the coordinates (Sx, Sy, Sz) on the screen are converted into the robot coordinates (Sx, Sy, Sz) in a step S45. Px, Py, Pz) and the robot coordinate conversion process is returned.

  According to this embodiment, since the robot can be designed and inspected using the simulator, the trouble of performing the test using the actual robot can be saved. Further, since the simulation is executed by CG, it can be used for developing a robot that does not yet exist.

  In this embodiment, a speaker is provided in the simulator so that utterances included in the communication behavior are output by voice. However, instead of voice, text (text) may be output. In such a case, the text of the utterance content may be displayed on the touch screen. For example, in order to produce an effect as if the model is speaking, a balloon (balloon) can be displayed near the mouth of the model. In this way, the speaker can be omitted from the simulator.

FIG. 1 is an illustrative view showing one example of a robot simulator of the present invention. FIG. 2 is an illustrative view for explaining the appearance of the robot simulated by the simulator shown in FIG. FIG. 3 is an illustrative view showing an electrical configuration of the robot shown in FIG. FIG. 4 is an illustrative view showing a display example of the touch screen shown in FIG. FIG. 5 is an illustrative view showing a state in which the model of the robot shown in FIG. 2 is arranged in a three-dimensional virtual space. FIG. 6 is a flowchart showing a part of the simulation processing of the computer shown in FIG. FIG. 7 is another part of the computer simulation process shown in FIG. 1, and is a flowchart subsequent to FIG. FIG. 8 is a flowchart showing screen display processing of the computer shown in FIG. FIG. 9 is a flowchart showing robot coordinate conversion processing of the computer shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Robot simulator 102 ... Computer 104 ... Touch screen 106 ... Speaker 10 ... Communication robot 18 ... Human body part 20 ... Skin 56 ... CPU
62 ... Motor control board 64 ... Sensor input / output board 66 ... Sound input / output board

Claims (3)

A robot simulator of a communication robot that executes communication actions including at least body movements,
Screen display means for displaying a screen of the communication robot model;
A robot simulator comprising: a pointing device provided in association with the screen display means; and an animation display means for displaying an animation of a body motion of the model in response to an input of the pointing device. The communication behavior includes utterance;
The robot simulator according to claim 1, further comprising output means for displaying the animation of the body motion of the model and outputting the utterance content accompanying the body motion. A communication robot simulation program that executes communication actions including at least body movements,
The simulation program which performs the screen display step which displays the model of the said communication robot on a screen, and the animation display step which displays the animation of the body movement of the said model according to the instruction input of the model displayed on the screen by the said screen display step.

Images