JP7353948B2 - Robot system and robot system control method - Google Patents

Description

The present invention relates to a robot system and a method of controlling the robot system.

Against the backdrop of labor shortages, the automation of manual labor through the introduction of robots is progressing. Until now, there has been progress in automating routine tasks such as parts cutting and welding in factories. It is expected that this technology will further expand in the future and find applications in a variety of industrial fields such as construction sites, distribution warehouses, food, cosmetics, and medical products.

One example is the automation of picking work for goods in a distribution warehouse. The automated picking robot acquires a 3D image of the product shelf using a 3D camera, detects the presence or absence of the target product through image recognition, and uses the image recognition and pre-registered 3D model data of the product to compare the grasping coordinates. Calculate. A trajectory is calculated based on the obtained grasping coordinates, and picking is performed.

A challenge in automating picking operations is that the throughput of picking operations by robots is slow. For example, even with the most advanced picking robots for distribution warehouses, the throughput is only about half that of skilled human workers.

One of the reasons why the operating throughput of a robot is slow is that the operating speed of the robot itself is slow. Up to now, vertically articulated robots have often been used to grasp objects that are three-dimensionally stacked in arbitrary positions.

Vertical articulated robots are equipped with a large number of joints and can assume any posture in three-dimensional space, and have been widely adopted for three-dimensional bulk picking applications. On the other hand, vertical articulated robots drive the joints connected in series and the motors that control the joints one by one to reach the final coordinates, so the more joints there are, the more time it takes to control them. It takes.

In addition, in a vertical articulated robot, each joint is connected in series and interacts with each other, so it is necessary to calculate the control value for each joint motor, including the interaction of each joint. Generally, there are multiple solutions for these motor control calculations, and it is necessary to select the solution that provides the safest posture for the robot from among them. The operation throughput of the robot was slow because it took time to calculate these multiple solutions, evaluate the multiple solutions, and select the solution with the safest posture.

Instead of a vertically articulated robot, as in Patent Document 1, the number of robot arms is increased from one to multiple parallel-connected types, and instead the number of joints is reduced to one joint per arm to automate high-speed picking work using a link structure. technology is being developed.

In parallel link robots in which multiple-arm robots are linked and operated in parallel, the joints are arranged in parallel, so the joints can be controlled in parallel, and each joint motor can be controlled independently. Since it can be calculated without calculating the interaction between joints, the operating speed of a single robot can be increased several times compared to a vertically articulated robot.

Japanese Patent Application Publication No. 2019-058979

A problem with automating picking work using a link structure robot having multiple arms described in Patent Document 1 is that the operable postures are limited. A vertically articulated robot can move in three dimensional positions, X, Y, and Z, and in rotational positions RX, RY, and RZ about the three axes, respectively.

On the other hand, parallel link robots can only move in three directions, X, Y, and Z, and RZ, which is a rotational posture around the Z axis, due to mechanical constraints in which joint motors are connected in parallel. Therefore, it is not possible to grasp articles randomly stacked in bulk in a three-dimensional space, and the range of application is limited to box-shaped articles stacked flat.

In the method of Patent Document 1, the gripping coordinates for gripping the object are dynamically extracted, but the placement coordinates for releasing the object are fixed. For this reason, if the placement coordinates are set fixedly assuming that a relatively large object is placed diagonally, the grasp of the object may be released from above for a thin object placed horizontally. become. As a result, the object would hit the workbench on the placement side, and there was a risk that the object would be damaged.

SUMMARY OF THE INVENTION An object of the present invention is to provide a robot system and a control method for a parallel link robot that can grip objects arbitrarily stacked in bulk in a three-dimensional space without damaging the objects.

A robot system according to one aspect of the present invention is a robot system using a parallel link robot having an imaging unit and a control unit, the parallel link robot having a three-dimensional robot hand equipped with a rotatable joint mechanism. The object stacked in bulk is gripped and moved from the gripping side to the placement side, the gripping side grips the object at the gripping point, the placement side releases the object at the placement point, and the imaging unit is configured to the control unit estimates the posture and size of the object based on the three-dimensional image, and estimates the posture and size of the object based on the estimation results of the posture and size of the object. , characterized in that the trajectory of the object from the grasping coordinates of the grasping point to the arrangement coordinates of the arrangement point is dynamically controlled.

A control method for a robot system according to one aspect of the present invention uses a parallel link robot that grips objects stacked in bulk three-dimensionally and moves them from a gripping side to a placement side using a robot hand equipped with a rotatable joint mechanism. A control method for a robot system, wherein a three-dimensional image of the object stacked in bulk is acquired, the posture and size of the object are estimated based on the three-dimensional image, and on the gripping side, the object is controlled at the gripping point. An object is gripped, and the placement side releases the object at a placement point, and moves the object from the gripping coordinates of the gripping point to the placement coordinates of the placement point based on the estimation results of the posture and size of the object. It is characterized by dynamically controlling the trajectory.

According to one aspect of the present invention, a parallel link robot can grip objects arbitrarily stacked in bulk in a three-dimensional space without damaging the objects.

It is a diagram showing a robot system of related technology. 1 is a diagram showing a robot system of Example 1. FIG. It is a figure which shows recognition and control processing of a recognition control part. FIG. 3 is a diagram showing the angle of view of a three-dimensional camera and candidate points for object grasping coordinates. FIG. 2 is a diagram showing target coordinates and motion trajectories of a parallel link robot. FIG. 2 is a diagram showing intermediate coordinates and motion trajectories of a parallel link robot. 3 is a flowchart diagram showing a method of controlling the robot system according to the first embodiment. FIG. FIG. 3 is a diagram showing a robot system according to a second embodiment.

First, a related technology control method for a robot system will be described with reference to FIG.
FIG. 1 is a diagram illustrating the operation of a related art autonomous grasping and placement system for three-dimensional bulk objects using a parallel link robot based on fixed placement coordinates.
As shown in FIG. 1, the related technology picking automation system using a parallel link robot 101 includes a parallel link robot body 101, a robot hand 103 equipped with a joint mechanism 102, a three-dimensional camera unit 109, and a robot based on three-dimensional camera data. It has a grip coordinate estimator 110 that plans and controls motion.

In a robot such as a parallel link robot 101 whose movement direction is limited, it is possible to grasp and align objects 111 and 112 stacked in bulk three-dimensionally. For this purpose, a robot hand 103 equipped with a three-dimensionally rotatable joint mechanism 102 is introduced.

The parallel link robot 101 is attached to a pedestal 106 and cannot move autonomously. Therefore, the objects 111 and 112 are conveyed to the placement-side workbench 108 by the gripping-side conveyor 107.

However, in the recognition/control method of the related technology, although the gripping coordinates 104 are dynamically extracted in the gripping coordinate estimation unit 110, the arrangement coordinates 105 are fixed. Therefore, as shown in FIG. 1(a), if the placement coordinates 105 are set fixedly assuming a case where a relatively large object 112 is placed diagonally, for a thin object 111 placed horizontally, As shown in FIG. 1(b), the grip will be released from above. As a result, the object 111 would hit the arrangement-side workbench 108, and there was a risk that the object 11 would be damaged.

Therefore, in the embodiment, a robot system and a method for controlling the robot system are provided, in which a parallel link robot can grasp objects arbitrarily stacked in bulk in a three-dimensional space without damaging the objects.

For this reason, in the embodiment, in a parallel link robot equipped with an autonomous control function and a rotatable joint mechanism, the object grasping position and placement position are estimated based on a 3D image captured by a 3D camera. The robot motion is controlled based on the grip position determined.

Examples will be described below with reference to the drawings.

The robot system of Example 1 will be described with reference to FIG. 2.
As shown in FIG. 2, the picking work automation system using the parallel link robot 101 of the first embodiment includes a parallel link robot main body 101, a robot hand 103 equipped with a joint mechanism 102, and three-dimensional camera units 109 and 3 that constitute an imaging unit. It has a recognition control unit 204 that plans and controls robot motion based on dimensional camera data.

The joint mechanism 102 is configured, for example, as a motor-type joint mechanism that includes a built-in motor and actively operates according to instructions from the recognition control unit 204. Note that the joint mechanism 102 may include a flexible mechanism such as rubber or a ball made of metal or resin, and may be configured as a joint mechanism that passively deforms in response to external force.

The parallel link robot 101 is attached to a pedestal 106 and cannot move autonomously. Therefore, the object 209 is conveyed to the placement-side workbench 108 by the gripping-side conveyor 107.

The recognition control unit 204 includes a three-dimensional object posture estimation unit 205, a three-dimensional object size estimation unit 206, a grip point/placement point coordinate calculation unit 207, and an intermediate coordinate/motion speed calculation unit 208.

The three-dimensional object posture estimation unit 205 estimates the posture of the object based on the three-dimensional image obtained from the three-dimensional camera 109. The three-dimensional object size estimation unit 206 estimates the size of the object. The gripping point/placement point coordinate calculation unit 207 calculates gripping coordinates 201 of the gripping point of the object and placement coordinates 202 of the placement point based on the estimated posture and size of the object.

On the gripping side, the gripping point/arrangement point coordinate calculation unit 207 calculates, as the gripping coordinates 201 of the gripping point, a candidate point that is located on the uppermost surface and has the widest grippable area among a plurality of predefined object gripping candidate points. do. On the placement side, based on the posture and size of the object 209, placement coordinates 202 of placement points that will prevent the object 209 from being damaged are calculated.

The intermediate coordinate/operation speed calculation unit 208 functions as an intermediate coordinate calculation unit, and based on the posture and size of the object 209, calculates intermediate coordinates that are intermediate points connecting the gripping coordinates 201 of the gripping point and the arrangement coordinates 202 of the arrangement points. Coordinates 203 are dynamically controlled.

Further, the intermediate coordinate/motion speed calculation unit 208 functions as a motion speed calculation unit, and based on the estimated posture and size of the object 209, the movement speed for gripping the object 209 and moving it from the gripping side to the placement side. Dynamically control.

In this way, the recognition control unit 204 recognizes the object 209 to be gripped and estimates the object posture, thereby determining the gripping coordinates 201 of the robot hand (suction pad) 103 and the arrangement coordinates at which the suction is released and the objects are aligned. 202 and the robot trajectory 210 from the gripping coordinates 201 to the placement coordinates 202 and the operating speed are dynamically controlled.

As described above, the robot system of the first embodiment is a robot system using the parallel link robot 101 having the three-dimensional camera 109 (imaging section) and the recognition control section 204 (control section). The parallel link robot 101 uses a robot hand 103 equipped with a rotatable joint mechanism 102 to grasp objects 209 stacked three-dimensionally in bulk and move them from the grasping side to the placement side.

On the grasping side, the object 209 is grasped at the grasping coordinates 201 of the grasping point, and on the placement side, the object 209 is released at the arrangement coordinates 202 of the arrangement point.

The three-dimensional camera 109 (imaging unit) acquires a three-dimensional image of the objects 209 stacked in bulk. The recognition control unit 204 (control unit) estimates the posture and size of the object 209 based on the three-dimensional image, and determines the arrangement point from the gripping coordinates 201 of the gripping point based on the estimation results of the posture and size of the object 209. The trajectory 210 of the object 209 toward the placement coordinates 202 of is dynamically controlled.

Here, the recognition control unit 204 (control unit) dynamically controls the trajectory 210 of the object 209 by variably controlling the arrangement coordinates of the arrangement points based on the estimation results of the posture and size of the object 209. .

The recognition control unit 204 acquires three-dimensional image data from the three-dimensional camera 109 and transmits it to the three-dimensional object posture estimation unit 205 . The three-dimensional camera 109 includes a stereo camera that uses two color cameras to estimate depth from parallax, and a stereo camera that irradiates a signal of a specific wavelength such as infrared light and measures the round trip time for it to reflect off the object surface and return to the sensor. Examples include a time of flight camera that estimates depth or a monocular camera that estimates depth using a method such as deep learning.

The three-dimensional image data that is the output of the three-dimensional camera 109 consists of two pieces of data: one is a depth image that aggregates the depth between the object surface captured within the angle of view and the sensor, and the other is a depth image that is obtained using a specific wavelength. image data (usually an RGB image acquired in a wide visible range).

The parallel link robot 101 is attached to a pedestal 106 and cannot move autonomously. Therefore, the object 209 is conveyed to the placement-side workbench 108 by the gripping-side conveyor 107. Therefore, when determining the gripping coordinates, the timing at which the three-dimensional image data was photographed is acquired, and the target coordinates are determined after taking into account the object movement distance by the conveyor 107 within the elapsed time due to recognition and control processing.

For example, the recognition control unit 204 transmits a trigger signal to the three-dimensional camera 109 to specify the timing of photographing. Alternatively, the three-dimensional camera 109 constantly transmits three-dimensional image data to the recognition control unit 204, and the recognition control unit 204 selects and acquires three-dimensional image data photographed at the same timing as the predetermined timing. In this way, it is possible to estimate the time difference between the timing at which the image was captured and the end of the recognition process, and to estimate the coordinates after transportation.

In this way, in the robot system of the first embodiment, the three-dimensional camera 109 acquires three-dimensional data of the bulk object 209 and transmits the data to the recognition control unit 204. The recognition control unit 204 not only estimates the grasping coordinates 201, but also estimates the size of the object and calculates the arrangement coordinates 202, intermediate coordinates 203, and motion speed.

Then, based on the three-dimensional image data obtained from the three-dimensional camera 109, first, the three-dimensional position and orientation of the target object are estimated. At the same time, the three-dimensional object model is called from the storage area and the size of the three-dimensional object is estimated. Then, among the plurality of object gripping candidate points predefined in the three-dimensional object model, the candidate point located on the uppermost surface and having the widest grippable area is calculated as the gripping coordinates 201.

Next, from the size information of the three-dimensional object and the three-dimensional position and orientation information, the height of the arrangement coordinate 202 at which the object gently contacts the arrangement surface is calculated. Then, intermediate coordinates 203 connecting the grasped coordinates 201 and placement coordinates 202 are calculated, and from the size information and attribute information of the object, the maximum operating speed of the robot that does not destroy the grasped object is calculated and output.

With reference to FIG. 3, recognition and control processing by the recognition control unit 204 will be described.
Only the two-dimensional image of the three-dimensional image data is input from the three-dimensional camera 109, and object identification is performed by two-dimensional image analysis (step 301). Recognizes grasped objects in the acquired image and estimates the type of each object.

If a plurality of objects exist, the object that exists furthest downstream in the traveling direction of the belt conveyor 107 is selected as the object to be grasped. Based on the type information of the workpiece selected as a gripping target, a three-dimensional object model of the object is called from the storage area, and three-dimensional position and orientation estimation is performed (step 302). As a result, it is estimated in what position and orientation the object to be grasped exists on the belt conveyor 107.

In the gripping point calculation step 303, the gripping candidate point and the grippable area are compared among the gripping coordinates of the plurality of candidates, and the candidate point that is the easiest to grip and has a wide grippable area is calculated as the gripping coordinate 201 of the gripping point. do.

As shown in FIG. 4, a plurality of gripping candidate points 402 to 405 are calculated for the object to be gripped within the viewing angle 401, and the gripping points 402 and 403 of the most downstream object are selected as the final candidate points. Compare for selection.

As shown in FIG. 4, between the gripping point 402 and the gripping point 403, the grippable area of the gripping point 402 is wider and flatter, so the gripping coordinates 201 of the final gripping point are used as the coordinates of the gripping point 402. Output. The graspable area is entered in advance as information into the three-dimensional model of the object. Alternatively, it may be determined inductively by machine learning using two-dimensional images.

Furthermore, since the rotational orientation of the object has been determined in the three-dimensional position and orientation estimation step 302 for the gripping coordinates 201 of the gripping point, the rotational orientation of the Z axis (RZ) is also assigned and output to the parallel link robot 101.

In the object placement coordinate estimation step 304, the object identification information obtained in the object identification step 301, the grasping coordinates 201 of the object obtained in the three-dimensional position and orientation estimation step 302, and the grasping point calculation step 303 and the rotational orientation of the object are used. , the placement coordinates 202 are calculated.

As an example, as shown in FIG. 5, by using the object identification information, the size L1 (503) of the object 209, the size L2 (502) of the object, and the grasped coordinates (501: point A), If the object is a rigid body, a point C (507) of the object 209 that is closest to the conveyor can be calculated.

Since the grasp coordinate point A (501) is a point predetermined for the three-dimensional model of the object 209, the distance L3 (504) from the grasp of the object 209 can be determined. Normally, the center of gravity of the object 209 is set so that L3=L1/2. The coordinates of point B (506), which is the coordinate point for grasping the object 209, can be calculated as (X + L3 cos θ, Y, Z - L3 sin θ) by using L3 (504) and three-dimensional rotation angle θ (505). ).

The coordinates of the point C (507) of the object 209 that is closest to the conveyor surface can be similarly calculated as (X+L3cosθ−L2sinθ,Y,Z−L3sinθ−L2cosθ).

Therefore, the height H (508) of the object 209 to the conveyor surface is H=Z−L3sinθ−L2cosθ. Therefore, by using the calculated height H (508) with respect to the reference coordinates (X', Y', Z'), the layout coordinates (509) can be set to (X', Y ', Z'-H).

By doing this, the placement coordinates are dynamically controlled according to the posture and size of the object, so the object can be placed safely without damaging it. Note that the method for calculating the arrangement coordinates is not particularly limited to the above-mentioned method, and may be calculated in combination with a functional machine learning method such as deep learning.

In the relay point calculation step 305, the intermediate coordinates 203 are calculated using the gripping coordinates 201 and the arrangement coordinates 202 as input. As shown in FIG. 6, intermediate coordinates 203 are set to be midway between gripping coordinates 201 and placement coordinates 202.

As shown in FIG. 6(b), a low intermediate coordinate 602 is set for a low gripping coordinate 601 and a low placement coordinate 603. Furthermore, the intermediate coordinates 203 and 602 are set so that the parallel link robot 101 does not always block the view of the three-dimensional camera 109.

Furthermore, the range of the maximum and minimum values of the coordinates is limited in advance in order to avoid interference with obstacles. Furthermore, the intermediate coordinates 203 and 602 are not limited to one point, and may include a plurality of coordinate points.

In this way, in the relay point calculation step 305, the intermediate coordinates 203, 602, which are relay points connecting the grasping coordinates 201, 601 and the placement coordinates 202, 603, are dynamically calculated based on the estimated posture and size of the object. to control.

In the robot operation speed calculation step 306, the conveyance speed of the parallel link robot 101 is controlled. The object identification results and posture estimation results are input, and the conveyance is performed for pre-registered objects that are difficult to pick up, heavy objects, objects with steep postures, or points whose intermediate coordinates 203 are at a high position relative to the conveyor surface. Processing is performed to increase conveyance accuracy by slowing down the speed.

In this way, the robot operation speed calculation step 306 dynamically adjusts the movement speed at which the object is gripped and moved from the gripping conveyor 107 to the arrangement workbench 108 based on the estimated posture and size of the object. Control.

In the robot control step 307, the grasping coordinates 201, posture (RZ), arrangement coordinates 202, intermediate coordinates 203, 602, and transport speed calculated so far are received as input variables, and are applied to various variables in the robot control program defined in advance. By making substitutions, the robot control program is completed and processing is executed.

A method of controlling the robot system according to the first embodiment will be described with reference to FIG.
First, a three-dimensional camera 109 images a bulk object to obtain three-dimensional data (S1).
Next, two-dimensional image analysis such as CNN is performed (S2). Then, the type of object is output (S3). In this manner, the two-dimensional image analysis (S2) recognizes the type of object, outputs the object type, and stores it in a buffer (S3). Common image recognition techniques include methods such as Convolutional Neural Network (CNN) and template matching.

Next, the most downstream object is selected as the object to be grasped (S4). Then, a three-dimensional object model is obtained (S5). Then, three-dimensional data analysis (eg, ICP method) is performed (S6). In this way, the model data created in advance and stored in the storage area is called up (S5), and three-dimensional data analysis (S6) is performed.

The Iterative Closest Point (ICP) method, which repeatedly calculates the degree of correlation with 3D image data while moving and rotating the model data, and finds the position coordinates and rotational orientation with the highest correlation, and recently, grasping candidates from 3D data. The rotational posture of the object to be grasped is determined by directly calculating the coordinate points using deep learning.

Next, the attitude of the object is output (S7). Then, grip candidate points are output (S8). Next, the widest gripping point in the gripping area is selected (S9). Then, the coordinates 201 of the gripping point are output (S10). At this time, the gripping candidate points defined in advance on the object are also output (S8), the grippable areas are compared (S9), and the gripping point with the widest gripping area is output (S0). Then, the arrangement coordinates 202 of the arrangement points are calculated (S11) .
Next, the relay point 203 is calculated (S12). Then, the robot operation speed is calculated (S13). Then, the calculated coordinates and robot commands are output (S14). Next, the robot operates and grasps and places the object (S15).

Finally, it is determined whether the grasping and placement of the object has been completed (S16). As a result of the determination, if the grasping and placement of the object is completed, the process ends. On the other hand, as a result of the determination, if the gripping and placement of the object are not completed, the process returns to S1.

Referring to FIG. 8, a robot system according to a second embodiment will be described.
The robot system of the second embodiment is a system in which the recognition control unit 204 of the robot system of the first embodiment shown in FIG. 2 is separated into a recognition arithmetic processing unit 801 and a robot control controller 802. The rest of the configuration is the same as the configuration of the robot system of the first embodiment shown in FIG. 2, so the explanation thereof will be omitted.

When separated as a robot controller 802, the imaging timing of the three-dimensional camera 109 is transmitted from the parallel link robot 101 to the recognition arithmetic processing unit 801 via an interface (not shown). If the real-time nature of the interface is not guaranteed, it is necessary to determine the gripping coordinates 201 in consideration of variations in the arrival time of the trigger signal.

According to the above-mentioned embodiment, even the parallel link robot can grip and arrange the articles stacked in bulk three-dimensionally. As a result, it is possible to realize bulk picking using a parallel link robot, which has been difficult in the past. Furthermore, work throughput can be increased compared to automation using vertically articulated robots.

101 Parallel link robot 102 Rotary joint 103 Robot hand 107 Gripping side belt conveyor 108 Arrangement side workbench
109 3D camera
204 Recognition control unit 205 3D object posture estimation unit 206 3D object size estimation unit 207 Gripping point/placement point coordinate calculation unit 208 Intermediate coordinate/motion speed calculation unit 209 Object 801 Recognition calculation processing unit 802 Robot control controller

Claims (11)

A robot system using a parallel link robot having an imaging unit and a control unit,
The parallel link robot is
A robot hand equipped with a rotatable joint mechanism grasps objects stacked in bulk three-dimensionally and moves them from the grasping side to the placement side.
The gripping side grips the object at the gripping point, and the placement side releases the object at the placement point;
The imaging unit includes:
obtaining a three-dimensional image of the object stacked in bulk;
The control unit includes:
Estimating the posture and size of the object based on the three-dimensional image,
dynamically controlling the trajectory of the object from the gripping coordinates of the gripping point to the placement coordinates of the placement point based on the estimation results of the posture and size of the object ;
The imaging unit is composed of a three-dimensional camera,
The control unit includes:
a three-dimensional object posture estimation unit that estimates the posture of the object based on the three-dimensional image acquired from the three-dimensional camera;
a three-dimensional object size estimation unit that estimates the size of the object;
a grasping point/arrangement point coordinate calculation unit that calculates the grasping coordinates and the arrangement coordinates of the object based on the estimated posture and size of the object;
has
The grasping point/arrangement point coordinate calculation unit includes:
On the gripping side, among a plurality of predefined object gripping candidate points, a candidate point located on the uppermost surface and having the widest grippable area is calculated as the gripping coordinates;
The robot system is characterized in that, on the placement side, the placement coordinates such that the object is not damaged are calculated based on the estimated posture and size of the object. The control unit includes:
2. The robot system according to claim 1, wherein the trajectory of the object is dynamically controlled by variably controlling the placement coordinates based on the estimation results of the posture and size of the object. The control unit includes:
The object may further include an intermediate coordinate calculation unit that dynamically controls intermediate coordinates that are relay points connecting the grasping coordinates and the placement coordinates based on the estimated posture and size of the object. The robot system according to claim 1. The control unit includes:
The method further includes a motion speed calculation unit that dynamically controls a movement speed for grasping the object and moving it from the grasping side to the placement side based on the estimated posture and size of the object. The robot system according to claim 1. The joint mechanism is
The robot system according to claim 1, further comprising a motor-type joint mechanism that actively operates according to instructions from the control unit. The joint mechanism is
The robot system according to claim 1, comprising a soft joint mechanism that passively deforms in response to external force. The parallel link robot is attached to a pedestal,
The robot system according to claim 1, wherein the object is conveyed by a conveyor. A method for controlling a robot system using a parallel link robot that grips three-dimensionally stacked objects in bulk with a robot hand equipped with a rotatable joint mechanism and moves them from a gripping side to a placement side, the method comprising:
obtaining a three-dimensional image of the object stacked in bulk;
Estimating the posture and size of the object based on the three-dimensional image,
The gripping side grips the object at a gripping point;
On the placement side, release the object at the placement point;
dynamically controlling the trajectory of the object from the gripping coordinates of the gripping point to the placement coordinates of the placement point based on the estimation results of the posture and size of the object ;
On the gripping side, among a plurality of predefined object gripping candidate points, a candidate point located on the uppermost surface and having the widest grippable area is calculated as the gripping coordinates;
The method for controlling a robot system is characterized in that, on the placement side, the placement coordinates such that the object is not damaged are calculated based on the estimated posture and size of the object. The robot system according to claim 8 , wherein the trajectory of the object is dynamically controlled by variably controlling the arrangement coordinates based on the estimation results of the posture and the size of the object. Control method. The robot according to claim 8 , wherein the robot dynamically controls intermediate coordinates that are relay points connecting the grasping coordinates and the placement coordinates based on the estimated posture and size of the object. How to control the system. 9. A moving speed at which the object is gripped and moved from the gripping side to the placement side is dynamically controlled based on the estimated posture and size of the object. control method for robot systems.

Images