JP2024068115A - Robot-mounted moving device - Google Patents

Abstract

A robot-mounted moving device is provided that can achieve highly accurate posture correction of a robot using an identification graphic and a camera.
[Solution] A robot-mounted moving device 25 includes a robot 26 having a camera 32 and an action unit 30 for an object, a moving device 35 on which the robot 26 is mounted and which can move to a work position, an identification figure D2 provided on the moving device 35 and having at least two parallel boundaries within the identification surface when viewed from a direction perpendicular to the identification surface, and a control device for controlling the movement of the robot 26. The control device analyzes the identification figure D2 in an image captured by the camera 32 from a direction different from the orthogonal direction, including the two boundaries, and calculates (i) position information of the camera 32 or (ii) position information of the action unit 30, and controls the movement of the action unit 30 acting on the object based on the calculated position information.
[Selected figure] Figure 7

Description

The present invention relates to a robot-mounted mobile device that is composed of a robot, a mobile device on which the robot is mounted, and a control device that controls at least the robot.

As an example of the above-mentioned robot-mounted moving device, a conventional automated guided vehicle with a robot arm is disclosed in JP 2017-132002 A (Patent Document 1 below). In this automated guided vehicle, an automated guided vehicle equipped with a robot moves to a work position set for a machine tool, and at this work position, the robot performs work such as attaching and detaching a workpiece to the machine tool.

In such an automated guided vehicle, a single robot that moves on the automated guided vehicle can perform tasks such as attaching and detaching workpieces to multiple machine tools, which allows for greater freedom in the layout of the machine tools compared to when the robot is fixedly installed relative to the machine tools, making it possible to set the layout of the machine tools in a way that can further increase production efficiency. Also, compared to older systems in which the robot was installed in a fixed state, a single robot can perform work on a larger number of machine tools, which allows for lower equipment costs.

On the other hand, because an automated guided vehicle is structured to move on its own using wheels, the positioning accuracy with which it stops at the work position is not necessarily high. For this reason, in order for a robot to perform accurate work on a machine tool, it is necessary to compare the posture of the robot when the automated guided vehicle is positioned at the work position with the reference posture of the robot set during so-called teaching, which serves as a control standard, to detect the amount of error, and to correct the working posture of the robot according to the amount of error.

A production system capable of correcting the posture of such a robot is disclosed in JP 2016-221622 A (Patent Document 2 below). In this production system, a visual target consisting of two calibration markers is disposed on the outer surface of the machine tool, and a camera is provided on the movable part of the robot. The relative positional relationship between the robot and the machine tool is measured based on an image obtained by capturing an image of the visual target with this camera and the position and posture of the camera, and the working posture of the robot is corrected based on the measured positional relationship.

JP 2017-132002 A JP 2016-221622 A

In a production system using the above-mentioned robot, highly accurate posture control of the robot is required to avoid unnecessary operational stops and achieve high work efficiency. For this reason, a highly accurate correction technology is required when capturing an image of the visual target (identification figure) with a camera and correcting the robot's posture based on the image obtained.

The present invention was developed in consideration of the above-mentioned circumstances, and aims to provide a robot-mounted mobile device that can achieve highly accurate posture correction of the robot using an identification figure and a camera.

In order to solve the above problems, the present invention provides a robot that has a camera for capturing an image and an action unit that acts on a predetermined object and performs a task on the object,
a moving device that carries the robot and that can move to a position where the action unit can act on the object;
an identification figure provided on the moving device, the identification figure having at least two parallel boundaries within the identification surface when viewed from a direction perpendicular to the identification surface;
A control device for controlling the movement of the robot,
The control device includes:
The present invention relates to a robot-mounted mobile device that analyzes the identification figure in an image captured by the camera from a direction different from the orthogonal direction, including the two boundaries, calculates (i) position information of the camera, or (ii) position information of an action part, and controls the movement of the action part acting on the target object based on the calculated position information.

According to this aspect (first aspect) of the robot-mounted mobile device, the identification figure provided on the mobile device has at least two parallel boundaries provided within the identification surface when viewed from a direction perpendicular to the identification surface, and the camera is configured to capture the identification figure from a direction different from the perpendicular direction. The control device then analyzes the image captured in this manner, including the two boundaries of the identification figure in the image, calculates position information of the camera or position information of the action unit, and controls the movement of the action unit acting on the target based on the calculated position information.

The camera captures an image of the identification figure from a direction different from the orthogonal direction perpendicular to the identification surface, i.e., from an oblique direction intersecting the orthogonal direction, causing the image corresponding to the two parallel boundaries within the identification figure to be distorted, and by analyzing this distorted state, the accurate positional relationship between the camera and the identification figure can be obtained. Then, based on this positional information, accurate positional information of the camera or accurate positional information of the action part can be calculated, making it possible to accurately control the movement of the action part relative to the target object.

From the above perspective, it is preferable that the identification figure is a figure having a matrix structure in which multiple square pixels are arranged two-dimensionally.

In the first aspect, the control device can adopt an aspect (second aspect) in which, when the identification figure is imaged by the camera, the posture of the robot is controlled so that the imaging optical axis of the camera is inclined at an angle of 15° to 60° toward the identification surface with respect to a direction perpendicular to the identification surface.

In addition, in the first aspect, the control device can adopt an aspect (third aspect) in which, when the identification figure is imaged by the camera, the posture of the robot is controlled so that the imaging optical axis of the camera is inclined at an angle of 30° to 60° toward the identification surface with respect to an orthogonal direction perpendicular to the identification surface.

In addition, in any of the first to third aspects, the control device can adopt a fourth aspect in which, when the camera captures an image of the identification figure, the control device controls the posture of the robot so that the imaging optical axis of the camera intersects with the parallel boundary at an angle of 15° to 75° in a direction around an axis centered on a perpendicular direction perpendicular to the identification surface.

Furthermore, in any of the first to third aspects, the control device can adopt an aspect (fifth aspect) in which, when the identification figure is imaged by the camera, the image-capturing optical axis of the camera intersects with the parallel boundary at an angle of 30° or more and 60° or less in a direction around an axis centered on a perpendicular direction perpendicular to the identification surface.

In addition, as a reference robot-mounted moving device according to the present invention, there is provided a robot having a camera for capturing images and an action part for acting on a predetermined object, and performing a task on the object;
a moving device that is mounted with the robot and is movable to a work position set with respect to the object;
A mobile device side identification figure (second identification figure) provided on the mobile device;
a control device configured to cause the robot to take at least a first image capturing posture in which an object side identification figure (first identification figure) provided corresponding to the object is captured by the camera, one or more working postures for causing the action unit to act on the object, and a second image capturing posture in which an image of a second identification figure for posture correction provided on the moving device is captured by the camera, according to an operation program including preset operation commands;
the first image capturing posture, the second image capturing posture, and the working posture are set in advance by teaching the robot under the control of the control device;
The control device includes:
a first positional relationship acquisition process for, during the teaching operation with the moving device located at the work position, operating the camera to capture an image of the first identification figure as a first reference image with the robot transitioning to a first imaging posture, and acquiring and storing a positional relationship between the first identification figure and the camera in the first imaging posture during the teaching operation based on the acquired first reference image;
a second positional relationship acquisition process for acquiring and storing a positional relationship between the second identification figure and the camera in the second imaging posture during the teaching operation, based on the second reference image acquired, while the robot is in a second imaging posture during the teaching operation and the camera is operated to capture an image of the second identification figure as a second reference image,
Furthermore, the control device
an actual operation error calculation process for, when the robot is actually operated in accordance with the operation program, with the moving device located at the work position, moving the robot to the first imaging posture, and then acquiring a positional relationship between the first identification figure and the camera in the first imaging posture during actual operation based on an image of the first identification figure during actual operation captured by the camera, and calculating an amount of error of the camera between the current posture of the robot and the posture during teaching operation based on the acquired positional relationship between the first identification figure and the camera during actual operation and the positional relationship between the first identification figure and the camera during teaching operation acquired by the first positional relationship acquisition process;
a first correction process for correcting the working posture during actual movement based on the amount of error calculated by the actual movement error calculation process;
a position change amount calculation process for calculating an amount of position change between a current position of the camera provided on the robot relative to the robot and its position at the time of the teaching operation based on an image of the second identification figure during the actual operation of the robot captured by the camera after the robot is shifted to the second imaging attitude when the robot is actually operated in accordance with the operation program, and a position change amount calculation process for calculating an amount of position change between a current position of the camera provided on the robot relative to the robot and its position at the time of the teaching operation based on the obtained positional relationship between the second identification figure and the camera at the time of the actual operation and the positional relationship between the second identification figure and the camera at the time of the teaching operation obtained by the second positional relationship acquisition process;
and a second correction process for correcting the working posture during actual movement based on the position change amount calculated by the position change amount calculation process.

As described above, according to the robot-mounted moving device of this aspect (sixth aspect), when the robot actually operates under the control of the control device in accordance with the operation program, the actual operation error calculation process is executed when the moving device is positioned at the work position and the robot is made to assume a working posture, and the working posture is corrected by the first correction process based on the calculated amount of error of the camera between the posture during the actual operation of the robot and the posture during the teaching operation.

In addition, when the position change amount calculation process is executed, the working posture during actual operation is corrected by the second correction process based on the amount of position change between the current position of the camera provided on the robot relative to the robot and the position at the time of the teaching operation, which is calculated by the position change amount calculation process.

When a camera is provided on a robot, positional deviation, i.e., positional change (mounting fluctuation), may occur between the robot and the camera as the robot operates over time. In addition, a positional change may occur between the robot and the camera due to an external force acting on the camera. When such a positional change occurs between the robot and the camera, the working posture cannot be accurately corrected by the first correction process alone.

According to the sixth aspect of the robot-mounted mobile device, the amount of position change between the robot and the camera between the time of the teaching operation (initial state) and the time of actual operation is calculated by the position change amount calculation process, and the working posture is corrected by the second correction process based on the calculated amount of position change. Therefore, even if a position change occurs between the robot and the camera, the working posture of the robot during actual operation can be accurately corrected.

The position change amount calculation process in the sixth aspect is configured to execute a process up to calculating the position change amount after the robot is moved to the second imaging posture, and then execute a process up to calculating the position change amount after the robot is moved to the second imaging posture corrected according to a correction amount determined based on the calculated position change amount, and when the difference between the obtained current positional relationship between the second identification figure and the camera and the positional relationship between the second identification figure and the camera during the teaching operation is equal to or less than a predetermined reference value, the position change amount obtained by the previous process is calculated as the desired position change amount (seventh aspect).

In addition, the position change amount calculation process in the sixth aspect can be configured to be executed in multiple cycles, with one cycle being the calculation of the position change amount after the robot is moved to the second imaging attitude, and in the second and subsequent cycles, the second imaging attitude is corrected according to a correction amount determined based on the position change amount calculated in the previous cycle, and when the difference between the current positional relationship between the second identification figure and the camera and the positional relationship between the second identification figure and the camera during the teaching operation becomes equal to or less than a predetermined reference value, the repetition of the cycle is terminated, and the position change amount calculated immediately before the termination is calculated as the final position change amount (eighth aspect). According to this aspect, the position change amount between the robot and the camera during the teaching operation (initial state) and the actual operation can be accurately calculated.

In addition, in any of the sixth to eighth aspects, the control device may be configured to execute the actual motion error calculation process after executing the position change amount calculation process, and to cause the robot to assume the working posture, and at that time, to execute the first correction process and the second correction process.

As described above, with the robot-mounted mobile device of the present invention, the camera captures an image of the identification figure from a direction different from the orthogonal direction perpendicular to the identification surface, i.e., an oblique direction intersecting the orthogonal direction, so that an accurate positional relationship between the camera and the identification figure can be obtained, and accurate positional information of the camera or the action part can be calculated based on such positional information. As a result, it becomes possible to accurately control the movement of the action part relative to the target object.

1 is a plan view showing a schematic configuration of a system according to an embodiment of the present invention. FIG. 1 is a block diagram showing a configuration of a system according to an embodiment of the present invention. 1 is a perspective view showing a robot-mounted moving device according to an embodiment of the present invention; 1 is an explanatory diagram for explaining a first image capturing posture of a robot according to the present embodiment. FIG. 4A and 4B are explanatory diagrams showing first and second identification figures according to the embodiment; 2 is a plan view showing a part of a cutaway view of an arm portion of a robot of the robot-mounted moving device according to the embodiment. FIG. 11 is an explanatory diagram for explaining a second image capturing posture of the robot according to the embodiment. FIG. 11 is an explanatory diagram for explaining a second image capturing posture of the robot according to the embodiment. FIG. 11 is an explanatory diagram for explaining a second image capturing posture of the robot according to the embodiment. FIG. 11 is an explanatory diagram for explaining the effect of the second image capturing posture of the robot according to the embodiment. FIG. 10A and 10B are explanatory diagrams for explaining a second correction process in the present embodiment. 10A and 10B are explanatory diagrams for explaining a second correction process in the present embodiment. FIG. 11 is an explanatory diagram for explaining a configuration of a robot-mounted moving device according to another embodiment of the present invention.

Specific embodiments of the present invention will be described below with reference to the drawings.

As shown in Figures 1 and 2, the system 1 according to this embodiment is composed of a machine tool 10, a material stocker 20 and a product stocker 21 as peripheral devices, and a robot-mounted moving device 25, which is composed of an automated guided vehicle 35 as a moving device, a robot 26 mounted on the automated guided vehicle 35, a camera 32 attached to the robot 26, and a control device 40 that controls the robot 26 and the automated guided vehicle 35.

As shown in FIG. 4, the machine tool 10 is a vertical NC (numerically controlled) lathe equipped with a spindle 11 on which a chuck 12 for gripping a workpiece W (W') is attached, and the spindle 11 is arranged along the vertical direction, so that the workpiece W (W') can be turned. In addition, a tool presetter 13 equipped with a contact 14 and a support bar 15 for supporting the contact 14 is provided near the spindle 11, and the support bar 15 is arranged to be able to advance and retreat from the machining area along the axis of the spindle 11. A ceramic display plate 16 is provided on the end surface on the machining area side, and the first identification figure (object side identification figure) D1 shown in FIG. 5 is drawn on the display plate 16. In this example, the display plate 16 is arranged to be located on a horizontal plane.

In this way, the machine tool 10 of this example is configured to place the first identification figure D1 within the machine tool 10, and in particular, it is preferable that the first identification image D1 is placed within the machining area.

Note that FIG. 4 shows the support bar 15 and contact 14 advanced into the processing area, but when the support bar 15 and contact 14 retract and the contact 14 and display plate 16 are stored in the storage area, the shutter 17 closes, isolating the contact 14 and display plate 16 from the processing area.

The first identification figure D1 in this example (as well as the second identification figure D2 described later) has a matrix structure in which multiple square pixels are arranged two-dimensionally, forming a rectangle as a whole, with each pixel displayed in white or black. In FIG. 5, the black pixels are shaded (as well as in FIGS. 1, 4, and 6). Such identification figures include those called AR markers and AprilTags. In addition, if the identification figure is small, a lens may be provided on the identification figure so that an enlarged image is captured by a camera 32 described later. Note that when viewed from a direction perpendicular to the identification figure D1, at least two parallel boundaries exist within the identification surface of this first identification figure D1 (as well as the second identification figure D2).

The material stocker 20 is disposed to the left of the machine tool 10 in FIG. 1 and is a device that stocks multiple materials (unmachined workpieces W) to be machined by the machine tool 10. The product stocker 21 is disposed to the right of the machine tool 10 in FIG. 1 and is a device that stocks multiple products or semi-finished products (machined workpieces W') machined by the machine tool 10.

As shown in FIG. 1, the automated guided vehicle 35 has the robot 26 mounted on its upper surface, the mounting surface 36, and is also provided with a portable operation panel 37 for an operator. The operation panel 37 is equipped with an input/output section for inputting and outputting data, an operation section for manually operating the automated guided vehicle 35 and the robot 26, and a display capable of displaying information on a screen.

The automated guided vehicle 35 is also equipped with a sensor (e.g., a distance measurement sensor using laser light) that can recognize its own position within the factory, and is configured to travel autonomously without a track within the factory, including the areas in which the machine tool 10, material stocker 20, and product stocker 21 are arranged, under the control of the control device 40, and in this example, passes through each of the work positions set for each of the machine tool 10, material stocker 20, and product stocker 21.

As shown in Figs. 1 and 3, the robot 26 of this embodiment is a multi-joint robot having three arms, a first arm 27, a second arm 28, and a third arm 29, which are connected in series via joints, and a hand 30 is attached to the tip of the third arm 29 as an end effector, and a camera 32 is attached via a support bar 31. As shown in Figs. 6 and 7, the second identification figure (moving device side identification figure) D2 is arranged near the robot 26 on the mounting surface 36 of the automatic guided vehicle 35 (see also Figs. 1 and 3).

The robot 26 is not limited to the above-mentioned configuration, but may be any configuration that includes (i) a camera, (ii) a hand unit for grasping a workpiece or tool, and (iii) an arm unit that movably connects the hand unit. There are various types of robots as is well known, and in addition to vertical articulated robots and horizontal articulated robots as in this example, Cartesian robots and the like can be applied. Also, in Figures 1, 3, 6, and 7, the schematic configuration of the robot-mounted moving device 25 is illustrated, and the specific shape and dimensions do not necessarily match.

2, the control device 40 of this example is composed of an operation program memory unit 41, a movement position memory unit 42, an operation posture memory unit 43, a map information memory unit 44, a reference image memory unit 45, a manual operation control unit 46, an automatic operation control unit 47, a map information generation unit 48, a position recognition unit 49, a correction amount calculation unit 50, and an input/output interface 51. The control device 40 is connected to the machine tool 10, the material stocker 20, the product stocker 21, the robot 26, the camera 32, the unmanned guided vehicle 35, and the operation panel 37 via the input/output interface 51. Note that the control device 40 is not limited to this form. The control device 40 only needs to have at least a control unit that controls the position of the hand 30 of the robot 26, and other storage units, etc. may be included in other devices.

The control device 40 is composed of a computer including a CPU, RAM, ROM, etc., and the functions of the manual operation control unit 46, the automatic operation control unit 47, the map information generating unit 48, the position recognition unit 49, the correction amount calculation unit 50, and the input/output interface 51 are realized by a computer program, and execute the processing described below. The operation program storage unit 41, the moving position storage unit 42, the operation posture storage unit 43, the map information storage unit 44, and the reference image storage unit 45 are composed of appropriate storage media such as RAM. In this example, the control device 40 is attached to the unmanned transport vehicle 35 and is connected to the machine tool 10, the material stocker 20, and the product stocker 21 by appropriate communication means, and is connected to the robot 26, the camera 32, the unmanned transport vehicle 35, and the operation panel 37 by wire or wirelessly. However, this is not limited to such an embodiment, and the control device 40 may be arranged at an appropriate position other than the unmanned transport vehicle 35. In this case, the control device 40 is connected to each part by appropriate communication means.

The manual operation control unit 46 is a functional unit that operates the automated guided vehicle 35, robot 26, and camera 32 according to operation signals input by an operator from the operation panel 37. That is, the operator can manually operate the automated guided vehicle 35, robot 26, and camera 32 using the operation panel 37 under the control of the manual operation control unit 46.

The operation program storage unit 41 is a functional unit that stores an automatic driving program for automatically driving the automatic guided vehicle 35 and the robot 26 during production, and a map generation program for operating the automatic guided vehicle 35 when generating map information within the factory, which will be described later. The automatic driving program and the map generation program are input, for example, from an input/output unit provided on the operation panel 37, and stored in the operation program storage unit 41.

The automatic driving program includes command codes related to the target position to which the automated guided vehicle 35 moves, the moving speed, and the direction of the automated guided vehicle 35, as well as command codes related to the sequential operations of the robot 26 and command codes related to the operation of the camera 32. The map generation program includes command codes for driving the automated guided vehicle 35 without a track throughout the factory so that map information can be generated in the map information generation unit 48.

The map information storage unit 44 is a functional unit that stores map information including the placement information of machines, devices, equipment, etc. (devices, etc.) placed in the factory where the automated guided vehicle 35 travels, and this map information is generated by the map information generation unit 48.

When the map information generating unit 48 drives the automated guided vehicle 35 in accordance with the map generation program stored in the operation program storage unit 41 under the control of the automatic driving control unit 47 of the control device 40, which will be described in detail later, the map information generating unit 48 acquires spatial information within the factory from distance data detected by the sensors, recognizes the planar shapes of devices and the like arranged within the factory, and recognizes the positions, planar shapes, and the like (arrangement information) of specific devices arranged within the factory, in this example the machine tool 10, material stocker 20, and product stocker 21, based on the planar shapes of devices and the like that have been registered in advance. The map information generating unit 48 then stores the obtained spatial information and the arrangement information of devices and the like in the map information storage unit 44 as map information within the factory.

The position recognition unit 49 is a functional unit that recognizes the position of the automated guided vehicle 35 within the factory based on the distance data detected by the sensor and the map information within the factory stored in the map information storage unit 44, and the operation of the automated guided vehicle 35 is controlled by the automatic driving control unit 47 based on the position of the automated guided vehicle 35 recognized by this position recognition unit 49.

The movement position memory unit 42 is a functional unit that stores the movement positions as specific target positions to which the automated guided vehicle 35 moves, which correspond to command codes in the operation program, and these movement positions include the work positions set for the machine tool 10, material stocker 20, and product stocker 21 described above. Note that these movement positions are set, for example, by manually driving the automated guided vehicle 35 with the operation panel 37 under the control of the manual operation control unit 46 to move it to each target position, and then storing the position data recognized by the position recognition unit 49 in the movement position memory unit 42. This operation is called a teaching operation.

The movement posture memory unit 43 is a functional unit that stores data relating to the movement postures (movement postures) of the robot 26 that change sequentially as the robot 26 moves in a predetermined order, and that correspond to command codes in the movement program. The data relating to the movement postures is rotation angle data of each joint (motor) of the robot 26 in each target posture when the robot 26 is manually operated to take each target posture by teaching operation using the operation panel 37 under the control of the manual operation control unit 46, and this rotation angle data is stored in the movement posture memory unit 43 as data relating to the movement posture.

The specific operating posture of the robot 26 is set in the material stocker 20, the machine tool 10, and the product stocker 21, respectively. In this example, a posture (second imaging posture) in which the camera 32 captures the second identification image D2 arranged on the loading surface 36 of the unmanned transport vehicle 35 is set. As shown in Figures 7, 8, and 9, the second imaging posture in which the camera 32 captures the second identification figure D2 is set to a posture in which the second identification figure D2 is captured obliquely. In this way, by setting the second imaging posture to a posture in which the second identification figure D2 is captured obliquely, as shown in Figures 3 and 7, the second imaging posture can be set by simply rotating the third arm 29 of the robot 26 in the arrow direction, and the robot 26 can be moved to the second imaging posture in a short time compared to, for example, moving the camera 32 directly above the second identification figure D2. The posture of the robot 26 shown in Fig. 3 illustrates a first imaging posture, which will be described later, and the robot 26 can transition from this first imaging posture to a second imaging posture simply by rotating the third arm 29 in the direction of the arrow. Fig. 7 shows the state in which the robot 26 has transitioned to the second imaging posture. Fig. 7 is an enlarged perspective view of the vicinity of where the second identification figure D2 is provided, so that the positional relationship between the camera 32 and the second identification figure D2 when the robot 26 is placed in the second imaging posture can be seen.

The posture of the camera 32 when capturing an image of the second identification figure D2 obliquely is set by the teaching operation such that the angle between the imaging optical axis of the camera 32 and the second identification figure D2 is a predetermined angle, and the posture of the robot 26 (second imaging posture) is set by the teaching operation. In this example, the angle of the imaging optical axis is defined by the angle Ry shown in FIG. 8 and the angle Rz shown in FIG. 9. As shown in FIG. 8, the angle Ry is an angle with respect to the z-axis, which is an axis perpendicular to the center position of the second identification figure D2, and is an angle at which the imaging optical axis is inclined toward the surface side of the second identification figure D2 with respect to the z-axis. As shown in FIG. 9, the angle Rz is a rotation angle around the axis centered on the z-axis with respect to one of the reference axes (x-axis in this example) of the x-axis and y-axis, which are reference axes parallel to the sides of the rectangle, with the center of the rectangular second identification figure D2 as the origin. The preferred ranges of the angles Ry and Rz will be described later.

In the material stocker 20, for example, the work start posture (removal start posture) when work is started in the material stocker 20, each work posture (each removal posture) for grasping the unmachined workpiece W stored in the material stocker 20 with the hand 30 and removing it from the material stocker 20, and the posture when removal is completed (removal completion posture, which in this example is the same posture as the removal start posture) are set as the removal operation posture.

In addition, the machine tool 10 is set with a work removal operation posture for removing the machined workpiece W' from the machine tool 10, and a work attachment operation posture for attaching the unmachined workpiece W to the machine tool 10.

Specifically, the workpiece removal operation postures may include, for example, a work start posture before entering the machine tool 10, a posture in which the hand 30 and camera 32 enter the machining area of the machine tool 10 and the camera 32 is positioned so that the first identification figure D1 provided on the support bar 15 is within the field of view of the camera 32, and the first identification figure is imaged by the camera 32 (first image capture posture) (see FIG. 4), a posture in which the hand 30 faces the machined workpiece W' held in the chuck 12 of the machine tool 10 (removal preparation posture), a posture in which the hand 30 is moved toward the chuck 12 and the machined workpiece W' held in the chuck 12 is grasped by the hand 30 (grasping posture), a posture in which the hand 30 is separated from the chuck 12 and the machined workpiece W' is removed from the chuck 12 (removal posture), and a posture in which the hand 30 and camera 32 have left the machine tool 10 (work completion posture). In addition, the first imaging orientation in which the camera 32 images the first identification figure D1 is also set to an orientation in which the first identification figure D1 is imaged obliquely, similar to the second imaging orientation described above, that is, an orientation in which the imaging optical axis of the camera 32 is in an oblique direction defined by the angles Ry and Rz with respect to the first identification figure D1.

In addition, the workpiece mounting operation posture may be, for example, a work start posture before entering the machine tool 10, a posture in which the hand 30 and camera 32 enter the machining area of the machine tool 10, the camera 32 is positioned so that the first identification figure D1 provided on the support bar 15 is within the field of view of the camera 32, and the camera 32 captures the first identification figure D1 (the same first imaging posture as above) (see FIG. 4), a posture in which the unmachined workpiece W held by the hand 30 faces the chuck 12 of the machine tool 10 (mounting preparation posture), a posture in which the hand 30 is moved toward the chuck 12 so that the unmachined workpiece W can be held by the chuck 12 (mounting posture), a posture in which the hand 30 is separated from the chuck 12 (separation posture), and a posture in which the hand 30 and camera 32 have left the machine tool 10 (work completion posture).

In the product stocker 21, the work start posture (storage start posture) when work is started in the product stocker 21, each work posture (storage posture) for storing the machined workpiece W' held by the hand 30 in the product stocker 21, and the posture when storage is completed (storage completion posture, which in this example is the same posture as the storage start posture) are set as storage operation postures.

The automatic driving control unit 47 is a functional unit that uses either the automatic driving program or the map generation program stored in the operation program storage unit 41, and operates the automatic guided vehicle 35, the robot 26, and the camera 32 according to the program. At that time, the data stored in the movement position storage unit 42 and the operation posture storage unit 43 are used as necessary.

The reference image storage unit 45 is a functional unit that stores an image of the first identification figure D1 captured by the camera 32 as a first reference image during a teaching operation, and an image of the second identification figure D2 captured by the camera 32 as a second reference image. Specifically, the first reference image is an image of the first identification figure D1 captured by the camera 32 when the automated guided vehicle 35 is in a work position set for the machine tool 10 and the robot 26 is in a first imaging posture. The second reference image is an image of the second identification figure D2 captured by the camera 32 when the robot 26 is in a second imaging posture.

When the robot 26 is automatically driven according to the automatic driving program stored in the operation program storage unit 41 under the control of the automatic driving control unit 47, the robot 26 is in a first image capturing posture, and the first identification figure D1 is captured by the camera 32, the correction amount calculation unit 50 calculates the position error amount of the camera 32 between the current posture of the robot 26 and the posture during the teaching operation based on the current image of the first identification figure D1 obtained during the automatic driving and the first reference image (image captured during the teaching operation) stored in the reference image storage unit 45, and calculates a first correction amount in the working posture based on the calculated position error amount (first correction amount calculation process).

When the robot 26 is in the second imaging posture and the second identification figure D2 is imaged by the camera 32, the correction amount calculation unit 50 calculates the position change amount (mounting fluctuation amount) of the camera 32 between the current posture of the robot 26 and the posture during the teaching operation based on the current image of the second identification figure D2 obtained during the automatic driving and the second reference image (image captured during the teaching operation) stored in the reference image storage unit 45, and calculates a second correction amount in the working posture based on the calculated position change amount (second correction amount calculation process).

Below, an example of the first correction amount calculation process and the second correction amount calculation process is described.

<First Correction Amount Calculation Process>
The correction amount calculation unit 50 calculates the amount of error (position error amount and rotation error amount) of the camera 32 between the current posture of the robot 26 and the posture during the teaching operation, and calculates the first correction amount based on this position error amount and rotation error amount.

Specifically, the correction amount calculation unit 50 executes the following processing based on the current image of the first identification figure D1 obtained during automatic driving and the first reference image (image captured during the teaching operation) stored in the reference image storage unit 45, etc., to estimate the amount of error of the camera 32 between the current posture of the robot 26 and the posture during the teaching operation, which is the amount of position error of the camera 32 in the mutually orthogonal x-axis and y-axis set in a plane parallel to the first identification figure, and the z-axis direction orthogonal to these x-axis and y-axis, and the amount of rotation error of the camera 32 around the x-axis, y-axis, and z-axis, and calculates the first correction amount for correcting each working posture of the robot 26 based on the estimated amount of position error and rotation error.

を取得する。尚、この座標変換行列

は、カメラ３２の内部パラメータ、識別図形から認識されるホモグラフィ行列、中心座標、コーナ座標及び第１識別図形の大きさなどから取得することができる。 (Preprocessing)
The correction amount calculation unit 50 first calculates, as a preprocessing step, a coordinate transformation matrix for transforming a camera coordinate system (camera_teach, the same hereinafter) that is a coordinate system corresponding to the camera 32 into a figure coordinate system (fig, the same hereinafter) that is a coordinate system for the first identification figure D1 based on the first reference image captured during the teaching operation.

In addition, this coordinate transformation matrix

can be obtained from the internal parameters of the camera 32, the homography matrix recognized from the identification figure, the center coordinates, the corner coordinates, and the size of the first identification figure.

The camera coordinate system is a three-dimensional coordinate system set for the planar imaging element group of the camera, and the origin is set, for example, at the center position of the imaging element group. The graphic coordinate system is a three-dimensional coordinate system set for the first identification figure D1, and the origin is set, for example, at the center position of the first identification figure D1. The robot coordinate system, which will be described later, is a three-dimensional coordinate system set for the control device 40 to control the robot 26, and the origin is set at an appropriate position.

と、前記カメラ座標系におけるカメラ位置であって、前記ティーチング操作における撮像時のカメラ位置

とに基づいて、前記図形座標系におけるティーチング操作時のカメラ位置

を以下の数式１によって算出する。
（数式１）

Next, the correction amount calculation unit 50 calculates the obtained coordinate transformation matrix

and a camera position in the camera coordinate system, the camera position at the time of imaging in the teaching operation.

Based on the above, the camera position during the teaching operation in the graphic coordinate system is calculated.

is calculated by the following formula 1.
(Formula 1)

を以下の数式２によって算出する。
（数式２）

続いて、補正量算出部５０は、ティーチング操作時のカメラ座標系からティーチング操作時のロボット座標系（ideal、以下同じ）へ変換するための座標変換行列

を以下の数式３によって算出する。
（数式３）

ここで、

の回転行列成分

を基に、ｘ軸、ｙ軸及びｚ軸回りの各回転角度

、

、

を算出する。
尚、

は、図形座標系からティーチング操作時のロボット座標系へ変換するための座標変換行列である。例えば、図形座標系からティーチング操作時のカメラ座標系へ変換するための座標変換行列

と、前記カメラ座標系からティーチング操作時のロボット座標系へ変換するための座標変換行列

に基づいて、以下の数式４によって取得することができる。
（数式４）

(Camera position calculation process during teaching operation)
Next, the correction amount calculation unit 50 calculates the camera position in the robot coordinate system during the teaching operation.

is calculated by the following formula 2.
(Formula 2)

Next, the correction amount calculation unit 50 calculates a coordinate transformation matrix for transforming from the camera coordinate system during the teaching operation to the robot coordinate system during the teaching operation (ideal, the same applies below).

is calculated by the following formula 3.
(Formula 3)

here,

Rotation matrix components of

Based on this, the rotation angles around the x-axis, y-axis, and z-axis

,

,

Calculate.
still,

is a coordinate transformation matrix for transforming from the graphic coordinate system to the robot coordinate system during teaching operation. For example, the coordinate transformation matrix for transforming from the graphic coordinate system to the camera coordinate system during teaching operation is

and a coordinate transformation matrix for transforming the camera coordinate system into the robot coordinate system during teaching operation.

Based on this, it can be obtained by the following Equation 4.
(Formula 4)

を取得した後、当該現在の画像に基づいて、図形座標系における現在のカメラ位置

を以下の数式５によって算出するとともに、前記ティーチング操作時のロボット座標系における現在のカメラ位置

を以下の数式６によって算出する。
（数式５）

（数式６）

続いて、補正量算出部５０は、現在のカメラ座標系からティーチング操作時のロボット座標系へ変換するための座標変換行列

を以下の数式７によって算出する。
（数式７）

ここで、

の回転行列成分

を基に、ｘ軸、ｙ軸及びｚ軸回りの各回転角度

、

、

を算出する。
(Camera position calculation process during autonomous driving)
Next, the correction amount calculation unit 50 calculates a coordinate transformation matrix for transforming the current camera coordinate system (camera_curr) into the graphic coordinate system in the same manner as described above, based on the image of the current first identification figure D1 obtained during automatic driving (actual operation).

After obtaining the current camera position in the figure coordinate system based on the current image,

is calculated by the following formula 5, and the current camera position in the robot coordinate system during the teaching operation is calculated by

is calculated by the following formula 6.
(Formula 5)

(Formula 6)

Next, the correction amount calculation unit 50 calculates a coordinate transformation matrix for transforming the current camera coordinate system into the robot coordinate system during the teaching operation.

is calculated by the following Equation 7.
(Formula 7)

here,

Rotation matrix components of

Based on this, the rotation angles around the x-axis, y-axis, and z-axis

,

,

Calculate.

、

、

と、ティーチング操作時の座標系における現在のカメラ角度

、

、

とに基づいて、これらの差分を算出することによって、ｘ軸、ｙ軸及びｚ軸回りの各回転誤差Δｒｘ、Δｒｙ及びΔｒｚを算出する。
但し、

(Error amount calculation process)
Next, the correction amount calculation unit 50 calculates the camera angle during the teaching operation in the coordinate system during the teaching operation calculated as described above.

,

,

and the current camera angle in the coordinate system during teaching operation.

,

,

By calculating the differences between these based on the above, the rotation errors Δrx, Δry, and Δrz around the x-axis, y-axis, and z-axis are calculated.
however,

、即ち、回転誤差量を以下の数式８によって算出するとともに、ティーチング操作時のロボット座標系から現在のロボット座標系への並進行列

、即ち、位置誤差量を以下の数式９により算出する。
（数式８）

（数式９）

Next, the correction amount calculation unit 50 calculates a rotation matrix between the robot coordinate system during the teaching operation and the current robot coordinate system based on the rotation errors Δrx, Δry, and Δrz calculated as described above.

That is, the amount of rotation error is calculated by the following formula 8, and the translation matrix from the robot coordinate system during the teaching operation to the current robot coordinate system is calculated by

That is, the position error amount is calculated by the following Equation 9.
(Formula 8)

(Formula 9)

を以下の数式１０によって算出する。
（数式１０）

＜第２補正量算出処理＞
上述したように、前記補正量算出部５０は、前記ロボット２６が第２撮像姿勢を取ったときに、前記カメラ３２によって撮像される前記第２識別図形Ｄ２の現在の画像と、前記基準画像記憶部４５に格納された第２基準画像（ティーチング操作時に撮像された画像）とに基づいて、ロボット２６の現在の姿勢とティーチング操作時の姿勢との間におけるカメラ３２の位置変化量（取付変動量）（位置誤差量及び回転誤差量）を算出するとともに、算出された位置変化量に基づいて、前記作業姿勢における第２補正量を算出する。 (Correction amount calculation process)
Next, the correction amount calculation unit 50 calculates a first correction amount for correcting the amount of error based on the amount of error calculated as described above.

is calculated by the following formula 10.
(Formula 10)

<Second Correction Amount Calculation Process>
As described above, when the robot 26 takes the second imaging posture, the correction amount calculation unit 50 calculates the position change amount (installation fluctuation amount) (position error amount and rotation error amount) of the camera 32 between the current posture of the robot 26 and the posture during the teaching operation based on the current image of the second identification figure D2 captured by the camera 32 and the second reference image (image captured during the teaching operation) stored in the reference image memory unit 45, and calculates a second correction amount for the working posture based on the calculated position change amount.

を取得する。図１１に、ロボット座標系（「robot」）、カメラ座標系（「registered_camera」）及び図形座標系（「setup_tag」）との関係を模式的に示している。尚、上述した第１補正量算出処理の場合と同様に、この座標変換行列

は、カメラ３２の内部パラメータ、識別図形から認識されるホモグラフィ行列、中心座標、コーナ座標及び第２識別図形の大きさなどから取得することができる。 (Preprocessing)
The correction amount calculation unit 50 first calculates, as a preprocessing, a coordinate transformation matrix for transforming a camera coordinate system (referred to as "registered_camera" to distinguish it from the first correction amount calculation process) which is a coordinate system corresponding to the camera 32 at the initial time, into a figure coordinate system (referred to as "setup_tag" to distinguish it from the first correction amount calculation process) which is a coordinate system for the second identification figure D2, based on the second reference image captured during the teaching operation.

11 shows a schematic diagram of the relationship between the robot coordinate system ("robot"), the camera coordinate system ("registered_camera"), and the graphic coordinate system ("setup_tag"). As in the case of the first correction amount calculation process described above, this coordinate transformation matrix

can be obtained from the internal parameters of the camera 32, the homography matrix recognized from the identification figure, the center coordinates, the corner coordinates, and the size of the second identification figure.

を取得し、取得された

と、

との差分値を算出する（ステップ１）。尚、図１２に、ロボット座標系（「robot」）、初期時のカメラ座標系（「registered_camera」）、現在のカメラ座標系（「actual_camera」）、図形座標系（「setup_tag」）と、実動作時におけるロボット座標系における現在のカメラ位置（「desired_camera」）と、物理的な意味での現在の実際のカメラ位置（「actual_camera」）と、誤差を打ち消すべくカメラ３２の位置が補正されるべき位置（制御されるべき位置）（「updated_camera」）との関係を模式的に示している。 (Position change amount and correction amount calculation process)
Next, the correction amount calculation unit 50 calculates a coordinate transformation matrix for transforming the current camera coordinate system (also referred to as “actual_camera” to distinguish it from the first correction amount calculation process) into the graphic coordinate system in the same manner as described above, based on the current image of the second identification figure D2 obtained during automatic driving (during actual operation).

Obtained and obtained

and,

12 shows a schematic diagram of the relationship between the robot coordinate system ("robot"), the initial camera coordinate system ("registered_camera"), the current camera coordinate system ("actual_camera"), the graphic coordinate system ("setup_tag"), the current camera position ("desired_camera") in the robot coordinate system during actual operation, the current actual camera position in the physical sense ("actual_camera"), and the position ("updated_camera") to which the camera 32 should be corrected (position to be controlled) to cancel out the error.

、上記の行列

、行列

及び行列

に基づいて、位置変化量に相当する行列、即ち、実動作時におけるロボット座標系における現在のカメラ位置（「desired_camera」）と、物理的な意味での現在の実際のカメラ位置（「actual_camera」）との間の位置関係を示す行列

を以下の数式１１に従って算出する（ステップ２）。
（数式１１）

尚、行列

は、ロボット座標系から、カメラ３２をロボット２６に取り付けた時の初期のカメラ座標系へ変換するための変換行列である。
また、行列

は、ロボット座標系から、現在のカメラ座標系へ変換するための変換行列である

の逆行列である。 Then, the correction amount calculation unit 50 calculates the inverse matrix of the obtained matrix.

, the above matrix

,queue

and matrix

Based on the above, a matrix corresponding to the amount of position change is obtained, that is, a matrix indicating the positional relationship between the current camera position ("desired_camera") in the robot coordinate system during actual operation and the current actual camera position ("actual_camera") in the physical sense.

is calculated according to the following formula 11 (step 2).
(Formula 11)

Furthermore, the matrix

is a transformation matrix for transforming from the robot coordinate system to the initial camera coordinate system when the camera 32 is attached to the robot 26.
Also, the matrix

is the transformation matrix for transforming from the robot coordinate system to the current camera coordinate system.

is the inverse matrix of

の逆行列である行列

と、上記の行列

とを用いて、算出された位置変化量を打ち消すために更新（補正）されたカメラ位置（「updated_camera」）を以下の数式１２に従って算出する（ステップ３）。
（数式１２）

Next, the correction amount calculation unit 50 calculates the obtained matrix

The matrix that is the inverse of

and the above matrix

Using these, an updated (corrected) camera position ("updated_camera") is calculated according to the following formula 12 in order to cancel out the calculated position change amount (step 3).
(Formula 12)

で定義される位置にカメラ３２が位置するようにロボット２６の第２撮像姿勢を変更する（ステップ４）。 Then, the automatic driving control unit 47 calculates the matrix

The second image capturing posture of the robot 26 is changed so that the camera 32 is positioned at the position defined by (step 4).

と、

との差分値が所定の閾値（基準値）を下回ったとき、この繰り返し処理を終了するとともに、補正量算出部５０は、終了直前に算出された位置変化量に相当する行列

を最終的な位置変化量とする。このようにすることで、ティーチング操作時（初期状態）と実動作時との間におけるロボット２６とカメラ３２との間に生じた前記位置変化量を正確に算出することができる。また、補正量算出部５０、得られた位置変化量である

から、その逆行列である

を第２補正量として算出する。 Thereafter, the correction amount calculation unit 50 and the automatic driving control unit 47 repeatedly execute steps 1 to 4, and

and,

When the difference value between the above and falls below a predetermined threshold value (reference value), the repetitive process is terminated, and the correction amount calculation unit 50 calculates a matrix

is set as the final position change amount. In this way, the position change amount that occurs between the robot 26 and the camera 32 during the teaching operation (initial state) and the actual operation can be accurately calculated. In addition, the correction amount calculation unit 50 calculates the obtained position change amount

Then, the inverse matrix

is calculated as the second correction amount.

The automatic operation control unit 47 corrects the position of the hand 30 of the robot 26 based on the first correction amount and the second correction amount calculated by the correction amount calculation unit 50 for the working postures when the robot 26 works on the machine tool 10, for example, the removal preparation posture, gripping posture, and removal posture in the work removal operation posture, and the attachment preparation posture, attachment posture, and separation posture in the work attachment operation posture.

を以下の数式１３に従って補正する。
（数式１３）

That is, the automatic driving control unit 47 calculates the position of the hand 30 in the operating posture of the robot 26 based on the first correction amount and the second correction amount calculated by the correction amount calculation unit 50 through the first correction amount calculation process and the second correction amount calculation process.

is corrected according to the following formula 13.
(Formula 13)

With the system 1 of this example having the above configuration, unmanned automatic production is carried out as follows.

That is, under the control of the automatic driving control unit 47 of the control device 40, the automatic driving program stored in the operation program storage unit 41 is executed, and the automated guided vehicle 35 and the robot 26 operate according to this automatic driving program.

For example, when the process of calculating the second correction amount is performed, the robot 26 is moved to the second imaging posture under the control of the automatic driving control unit 47, and the second identification figure D2 is imaged by the camera 32. Then, the correction amount calculation unit 50 calculates the position change amount (mounting fluctuation amount) of the camera 32 between the current posture of the robot 26 and the posture during the teaching operation according to the formula 11 based on the current image of the second identification figure D2 obtained during the automatic driving and the second reference image (image captured during the teaching operation) stored in the reference image storage unit 45, and calculates the second correction amount in the working posture based on the calculated position change amount (second correction amount calculation process). Note that this second correction amount calculation process may be performed periodically or irregularly, but is preferably performed before the robot 26 performs work.

When the robot-mounted moving device 25 performs work, the automated guided vehicle 35 moves to a work position set for the machine tool 10 under the control of the automatic operation control unit 47, and the robot 26 takes the work start posture for the work removal operation described above. At this time, the machine tool 10 has completed the specified processing, and the door cover is open so that the robot 26 can enter the processing area, and a command is received from the automatic operation control unit 47 to advance the support bar 15 of the tool presetter 13 into the processing area.

The robot 26 then transitions to the first imaging posture, and the camera 32 images the first identification figure D1 provided on the support bar 15. When the first identification figure D1 is thus imaged by the camera 32, the correction amount calculation unit 50 calculates the amount of error of the camera 32 according to the image of the first identification figure D1 and the first reference image stored in the reference image storage unit 45, in accordance with the above formulas 1 to 9, and calculates a first correction amount according to formula 10, based on the obtained error amount.

Then, the automatic operation control unit 47 corrects the position of the hand 30 in the subsequent workpiece removal operation postures, i.e., the above-mentioned removal preparation posture, gripping posture, removal posture, and work completion posture, according to Equation 13 based on the first correction amount and the second correction amount calculated by the correction amount calculation unit 50, and grips the machined workpiece W' gripped by the chuck 12 of the machine tool 10 with the hand 30 and removes it from the machine tool 10. After the robot 26 is caused to take the gripping posture, the automatic operation control unit 47 sends a chuck open command to the machine tool 10, thereby opening the chuck 12.

Next, the automatic operation control unit 47 moves the automated guided vehicle 35 to a work position set for the product stocker 21, and causes the robot 26 to sequentially assume a storage start posture when starting work in the product stocker 21, various storage postures for storing the machined workpiece held by the hand 30 in the product stocker 21, and a storage completion posture when storage is completed, thereby storing the machined workpiece held by the hand 30 in the product stocker 21.

Then, the automatic operation control unit 47 moves the automated guided vehicle 35 to a work position set for the material stocker 20, and causes the robot 26 to sequentially assume an unloading start posture when starting work at the material stocker 20, various unloading postures for grasping the pre-machined workpiece stored in the material stocker 20 with the hand 30 and unloading it from the material stocker 20, and an unloading completion posture when unloading is completed, and causes the hand 30 to grasp the pre-machined workpiece.

Next, the automatic operation control unit 47 again moves the automated guided vehicle 35 to the work position set for the machine tool 10, and causes the robot 26 to assume the work start posture for the above-mentioned workpiece attachment operation. Next, the robot 26 is moved to the imaging posture, and the first identification figure D1 provided on the support bar 15 is imaged by the camera 32. Then, when the first identification figure D1 is thus imaged by the camera 32, the correction amount calculation unit 50 calculates the position error amount of the camera 32 according to the above formulas 1 to 9 based on the image of the first identification figure D1 and the first reference image stored in the reference image storage unit 45, and calculates the first correction amount according to formula 10 based on the obtained position error amount.

Then, the automatic operation control unit 47 corrects the position of the hand 30 in the subsequent workpiece mounting operation postures of the robot 26, i.e., the above-mentioned mounting preparation posture, mounting posture, separation posture, and work completion posture, according to Equation 13 based on the first correction amount and the second correction amount calculated by the correction amount calculation unit 50, and causes the robot 26 to mount the unmachined workpiece W held by the hand 30 to the chuck 12 of the machine tool 10 and then to exit the machine. After this, the automatic operation control unit 47 transmits a machining start command to the machine tool 10 to cause the machine tool 10 to perform a machining operation. After causing the robot 26 to assume the mounting posture, the automatic operation control unit 47 transmits a chuck close command to the machine tool 10, which closes the chuck 12 and causes the chuck 12 to grip the unmachined workpiece W.

By repeating the above steps, unmanned automatic production is carried out continuously in the system 1 of this example.

In the present example, the system 1 uses the first identification figure D1 arranged within the machining area of the machine tool 10 where the robot 26 actually works to correct the working posture of the robot 26, so that the working posture can be accurately corrected, and thus the robot 26 can perform tasks with high precision, even when high operational accuracy is required.

In addition, in the system 1 of this example, a second correction amount for correcting the position change between the robot 26 and the camera 32 between the time of the teaching operation (initial state) and the time of actual operation is calculated by a second correction amount calculation process, and the working posture is corrected by the second correction process based on the calculated second correction amount. Therefore, even if a position change occurs between the robot 26 and the camera 32 due to the operation of the robot over time, or a position change occurs between the robot 26 and the camera 32 due to an external force acting on the camera, the working posture of the robot 26 during actual operation can be accurately corrected.

In this example, the robot is moved to the second imaging posture, and the position change amount is calculated as one cycle, and the cycle is repeated multiple times. In the second and subsequent cycles, the second imaging posture is corrected according to a correction amount determined based on the position change amount calculated in the previous cycle. When the difference between the current positional relationship between the second identification figure D2 and the camera 32 and the positional relationship between the second identification figure D2 and the camera 32 during the teaching operation becomes equal to or less than a predetermined reference value, the repetition of the cycle is terminated, and the position change amount calculated immediately before the end is calculated as the final position change amount. According to this embodiment, the position change amount between the robot 26 and the camera 32 that occurred between the teaching operation (initial state) and the actual operation can be accurately calculated, and the second correction amount for correcting this can be accurately calculated.

In this way, the robot 26 can perform tasks with high precision, allowing the system 1 to operate at a high availability rate without unnecessary interruptions. As a result, the system 1 can achieve unmanned operation with high reliability and high production efficiency.

In addition, in this example, when the first identification figure D1 and the second identification figure D2 are imaged by the camera 32, the first identification figure D1 and the second identification figure D2 are imaged obliquely, so that the accurate positional relationship between the camera 32 and the first identification figure D1 and the D2 identification figure D2 can be obtained from the image captured. That is, the first identification figure D1 and the second identification figure D2 have a matrix structure in which a plurality of square pixels are arranged two-dimensionally, but when such first identification figure D1 and the second identification figure D2 are imaged obliquely, the square pixels are distorted into a trapezoidal shape, and by analyzing this distortion state, the accurate positional relationship between the camera 32 and the first identification figure D1 and the D2 identification figure D2 can be obtained.

Incidentally, the angle Ry at which the first identification figure D1 and the second identification figure D2 are imaged obliquely is preferably within the range of 15° to 60°, and more preferably within the range of 30° to 60°. The angle Rz is preferably within the range of 15° to 75°, and more preferably within the range of 30° to 60°.

Figure 10 shows the results of positioning the robot 26 a predetermined number of times in an imaging posture in which the angles Ry and Rz of the imaging optical axis of the camera 32 are set to the respective set angles, capturing the second identification image D2 (similarly the first identification image D1), calculating the position of the camera 32 based on each of the obtained images, for example, using the above-mentioned formula 2, and estimating the maximum error (mm) that can occur 200 mm from the tip of the third arm 29 of the robot 26 based on the calculated variance in the position of the camera 32. The angles Ry and Rz were set to 0°, 15°, 30°, 45°, 60°, and 75°, respectively, and the number of repetitions of positioning at each angle was 80.

As can be seen from the results shown in FIG. 10, the angle Ry is selected to be 15° or more and 60° or less, and the angle Rz is selected to be 15° or more and 75° or less, as the preferred angle range for the maximum error being 0.1 mm or less. Furthermore, the angle Ry is selected to be 30° or more and 60° or less, and the angle Rz is selected to be 30° or more and 60° or less, as the more preferred angle range.

As described above, it is preferable that the first and second imaging postures of the robot 26 are postures in which the imaging optical axis of the camera 32 is tilted by angles Ry and Rz, but it is also possible to set the angle Rz to 0° and have only the angle Ry tilted, or conversely, to set the angle Ry to 0° and have only the angle Rz tilted.

Although one embodiment of the present invention has been described above, the specific aspects that the present invention can adopt are in no way limited to this.

For example, in the above example, the objects on which the robot-mounted moving device 25 works are the machine tool 10, the material stocker 20, and the product stocker 21, but the work objects are not limited to these, and the work objects can be any other known industrial machine, such as a cleaning device for cleaning the workpiece, or a measuring device for measuring the shape and dimensions of the workpiece. Furthermore, in terms of machine tools, the work objects can be any known machine tools, such as a vertical lathe, a horizontal lathe, or a vertical or horizontal machining center.

In the above example, the robot is moved to the second imaging posture, and then the position change amount is calculated, which is counted as one cycle, and multiple cycles are repeated to execute the process. However, the present invention is not limited to this configuration, and the position change amount may be calculated by performing the position change amount calculation process once.

In addition, in the above example, an unmanned guided vehicle was used as an example of a moving device, but this is not limited to this, and a manually moved cart may also be used as the moving device.

Alternatively, the moving device may be configured as shown in FIG. 13. The robot-mounted moving device 110 in this example includes a moving device 111 and a robot 115 provided on the moving device 111. The moving device 111 includes a guide rail 112 as a track horizontally arranged so as to straddle the machine tool 100 and the stockyard 105 adjacent to the machine tool 100, and a slider 113 that engages with the guide rail 112 and is provided so as to be movable along the guide rail 112 in the direction of the arrow indicated by the two-dot chain line. The robot 115 equipped with a camera 116 is fixed to the slider 113, and the second identification figure D2 is provided on the slider 113. In this example, the slider 113 of the moving device 111 moves in the direction indicated by the arrow, causing the robot 115 to move between the machine tool 100 and the stockyard 105, and the robot 115 transports the workpieces W1 and W2 between the machine tool 100 and the stockyard 105.

To reiterate, the above description of the embodiment is illustrative in all respects and is not restrictive. Those skilled in the art can make appropriate modifications and changes. The scope of the present invention is indicated by the claims, not the above embodiment. Furthermore, the scope of the present invention includes modifications from the embodiment within the scope of the claims and the equivalent range.

REFERENCE SIGNS LIST 1 System 10 Machine tool 20 Material stocker 21 Product stocker 25 Robot-mounted moving device 26 Robot 32 Camera 35 Automatic guided vehicle 40 Control device 41 Operation program memory unit 42 Movement position memory unit 43 Operation posture memory unit 44 Map information memory unit 45 Reference image memory unit 46 Manual driving memory unit 47 Reference image memory unit 46 Manual driving memory unit 47 Automatic driving memory unit 48 Map information memory unit 49 Position recognition unit 50 Correction amount calculation unit D1 First identification figure (target side identification figure)
D2 Second identification figure (mobile device side identification figure)

Claims (1)

a robot having a camera for capturing an image and an action unit for acting on a predetermined object, the robot performing a task on the object;
a moving device that carries the robot and that can move to a position where the action unit can act on the object;
an identification figure provided on the moving device, the identification figure having at least two parallel boundaries within the identification surface when viewed from a direction perpendicular to the identification surface;
A control device for controlling the movement of the robot,
The control device includes:
A robot-mounted moving device that analyzes the identification figure in an image captured by the camera from a direction different from the orthogonal direction, including the two boundaries, calculates (i) position information of the camera, or (ii) position information of an action part, and controls the movement of the action part acting on the object based on the calculated position information.

Images