JP2021122879A - Image processing device and carrier device equipped with image processing device - Google Patents

Abstract

To provide a carrier device that carries an article while making article recognizing means grasp a position and a posture of a rectangular box-shape stacked article such as a cardboard box, and to provide article recognizing means that can reduce time required in processing for recognizing a position and a posture of an article and grasp the position and the posture of the article with high accuracy.SOLUTION: An image processing device 7 of a carrier device 1 comprises photographing means 10 that photographs a stacked article 30 in a rectangular box-shape to obtain image data and an image distance sensor 11 that measures distances to the article 30 with respect to a plurality of positions of the article 30 to obtain the distances as distance data. The image processing device 7 comprises: an image processing circuit that detects four corners 31 of a front surface of the article 30 from the image data and obtains distances to the four corners 31 of the front surface from the distance data; a position calculation circuit that calculates a position and a posture of the front surface of the article 30 on the basis of the distances to the four corners 31 of the front surface; and a control device 7b having a robot controller that controls a holding hand 4 and a robot arm 3.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image processing device, for example, an image processing device used for a transport device for transporting rectangular box-shaped articles stacked in a container (loading room of a truck), and a transport provided with the image processing device. Regarding the device.

For example, there is provided a transport device capable of holding and unloading rectangular box-shaped stacked articles such as cardboard boxes with a robot hand. By unloading articles with a transport device equipped with a robot hand, labor saving and work efficiency can be improved. In this transport device, the articles can be held more accurately by grasping the positions and postures of the stacked articles by the article recognition means with high accuracy. Patent Document 1 discloses a configuration in which a distance sensor for detecting the position and orientation of an article is arranged above the stacked articles. The invention described in Patent Document 1 is configured to acquire three-dimensional point group data relating to an article by moving or rotating a distance sensor to detect the distance to the article at a plurality of positions. Patent Document 2 discloses a transport device that detects the upper sides of a plurality of stacked articles and holds them from the articles stacked on the upper side based on the positions of the upper sides.

Japanese Patent No. 5492614 Japanese Patent No. 6172293

However, when recognizing the position and orientation of an article using three-dimensional point cloud data as in the invention described in Patent Document 1, the recognition process takes time due to the large number of data. In order to carry out the unloading work efficiently, it is desirable that the time for recognizing the article can be shortened as much as possible. Further, when only the upper side of the article is detected as in the invention described in Patent Document 2, it is difficult to grasp the accurate position and posture of the article. Therefore, there is a demand for an article recognition means capable of grasping an accurate position and posture of an article while shortening the recognition processing time.

An object of the present invention is to provide an article recognition means capable of shortening the recognition processing time and grasping the position and posture of an article with high accuracy.

According to one feature of the present disclosure, the image processing device of the transport device includes a photographing means and an image distance sensor. The photographing means acquires image data by photographing stacked rectangular box-shaped articles. The image distance sensor measures the distance to the article with respect to a plurality of positions of the article and acquires it as distance data. The image processing device is provided with an image processing circuit, a position calculation circuit, and a robot control circuit. The image processing circuit detects the four front corners on the front side in the shooting direction of the article from the image data, and acquires the distances to the four front corners from the distance data. The position calculation circuit calculates the position and orientation of the article based on the distances to the four front corners of the article. The robot control circuit controls a holding hand that holds the article and a robot arm that moves the holding hand based on the position and posture of the article.

Therefore, the position and posture of the article are grasped by using only the distances to the four front corners of the article from the distance data acquired by the image distance sensor. Therefore, the number of data required for the recognition process can be reduced. As a result, the time for the recognition process can be shortened, and the position and orientation of the rectangular surface of the article facing the image distance sensor can be grasped with high accuracy. Further, the number of data required for the recognition process is determined by the number of articles regardless of the size of the articles. Therefore, even when detecting the front four corners of articles of various sizes, the position and posture can be grasped in a short processing time.

According to another feature of the present disclosure, the image processing circuit detects the front center region on the front side in the shooting direction of the article from the image data, and acquires the distance from the distance data to the front center region. The position calculation circuit calculates the position and orientation of the article based on the distances to the front four corners of the article and the front center region.

Therefore, the distance data of the four front corners and the central front region of each article is acquired and recognized. As a result, the recognition process can be performed in a short time. Further, by detecting the distances to the four front corners of the article and the distances to the front center region, respectively, the position and orientation of the rectangular surface of the article facing the image distance sensor can be grasped with higher accuracy. For example, when the rectangular box-shaped article is a corrugated cardboard box or the like, the front center region may be located at a position slightly protruding from the front four corners, and the rectangular surface may not be flat. Even in such a case, the position and orientation of the rectangular surface can be grasped with higher accuracy while roughly grasping the shape of the rectangular surface of the article facing the image distance sensor.

According to another feature of the present disclosure, the photographing means and the image distance sensor simultaneously acquire image data and distance data of a plurality of stacked articles.

Therefore, the positions and postures of a plurality of articles can be grasped at the same time. As a result, the time required to recognize the position and posture of the article can be shortened. Further, by recognizing the positions and postures of a plurality of articles at one time, it becomes easy to select the order of holding the articles so as to improve the efficiency of the transport work. As a result, the working time can be further shortened.

According to another feature of the present disclosure, the transport device with the image processing device includes a holding hand, a robot arm, and a trolley. The dolly is movable with a robot arm mounted on it. The photographing means and the image distance sensor are mounted on the trolley.

Therefore, it is possible to grasp the position and posture of goods without installing a special device for goods stacked in various places such as containers (truck luggage compartments) and factory warehouses. can. Further, between the transfer work by the holding hand and the robot arm, the image data can be acquired by the photographing means mounted on the trolley and the distance data can be acquired by the image distance sensor. Therefore, the work time can be shortened and the work efficiency can be improved. Further, for example, when articles are stacked in a container or the like, the articles may be stacked near the ceiling of the container, and it may be difficult to recognize the position and posture of the articles from above the articles. Alternatively, it may be difficult to install the article recognition means separately from the transport device 1 in the container. According to the features of the present disclosure, the position and posture of the articles can be grasped regardless of the place where the articles are stacked.

It is an overall side view of the transport device which concerns on this embodiment. It is a front view of the article stacked in a container. It is a front view of each article recognized by the image processing circuit of an image processing apparatus. It is a side view of the holding hand which concerns on this invention. It is a side view when the holding hand which concerns on this invention adsorbs an article. It is a side view when the receiving part of the holding hand which concerns on this invention is extended. It is a block diagram of each configuration which concerns on article recognition in the image processing apparatus which concerns on this invention. It is a figure which shows the flowchart which recognizes the position and the posture of all the articles by the image processing apparatus which concerns on this invention. It is a figure which shows the flowchart which recognizes the position and the posture of the article to be picked by the image processing apparatus which concerns on this invention, and picks an article by the transport apparatus.

Next, this embodiment will be described with reference to FIGS. 1 to 9. As shown in FIG. 1, the transport device 1 includes an image processing device 7 for recognizing the position and orientation of the article 30. Further, the transport device 1 includes a carriage 2, a robot arm 3 mounted on the upper side of the carriage 2, and a holding hand (robot hand) 4 provided at the tip of the robot arm 3. The trolley 2 can self-propell on the floor surface of, for example, a container or a warehouse of a factory. In the following description, a transport device 1 for unloading the articles 30 stacked in the container (truck luggage compartment) 34 will be illustrated. The article 30 has a rectangular box shape, such as a cardboard box. The articles 30 are stacked in the vertical direction and are stacked in a large number in the depth direction. The up-down, front-back, left-right directions are defined when viewed from the transport device 1 side, and the front side of the article 30 is the front side and the depth side is the rear side.

As shown in FIG. 1, the carriage 2 is a crawler type traveling device. The bogie 2 includes an endless track 2a and drive wheels 2b. The endless track 2a operates by rotating the drive wheels 2b by driving the electric motor. The dolly 2 moves on the floor surface of the container 34 by the operation of the endless track 2a.

As shown in FIG. 1, the trolley 2 is equipped with an articulated robot arm 3 with 6-axis control. The robot arm 3 includes a first arm 3a extending from the upper surface of the carriage 2, a second arm 3b connected to the tip of the first arm 3a, and a third arm 3c connected to the tip of the second arm 3b. There is. The holding hand 4 is supported at the tip of the third arm 3c. The image processing device 7 includes a photographing unit 7a and a control device 7b. The photographing unit 7a and the control device 7b are mounted on the carriage 2. The photographing unit 7a is equipped with a photographing means (2D camera) 10 and an image distance sensor (TOF camera) 11. The photographing means 10 and the image distance sensor 11 are provided so as to face the depth direction (rear) so that the front side of the article 30 can be photographed.

As shown in FIG. 4, the holding hand 4 includes a suction portion 5 capable of sucking the article 30 and a receiving portion 6 capable of supporting the article 30 from below. The suction portion 5 is supported by a support portion 5a connected to the robot arm 3. The suction portion 5 is floatingly supported vertically with respect to the support portion 5a by a spring force. A plurality of suction pads 5b are provided at the rear portion of the suction portion 5 so as to face rearward. A suction hose is piped to the suction portion 5, although it is not particularly visible in the figure. One end of the suction hose is connected to the suction pad 5b, and the other end is connected to a suction machine provided outside the holding hand 4. By starting the suction machine and sucking air, a negative pressure is generated in the suction pad 5b, so that the front surface of the article 30 can be sucked by the suction pad 5b.

As shown in FIG. 4, the receiving portion 6 is provided at the lower part of the holding hand 4. The receiving portion 6 is supported by the supporting portion 5a. The receiving portion 6 is floatingly supported up and down by a spring force with respect to the supporting portion 5a independently of the suction portion 5. The pedestal 6b at the rear of the receiving portion 6 can be displaced in the front-rear direction in two stages by an electric motor and a moving pulley mechanism. The pedestal accommodating portion 6a at the front of the receiving portion 6 supports the pedestal 6b so as to be displaceable in the front-rear direction. As shown in FIG. 5, when the holding hand 4 approaches the front surface of the article 30, the cradle 6b retracts to the front side so as not to come into contact with the stacked lower article 30, and accommodates the cradle. It is in a state of being housed in the part 6a. As shown in FIG. 6, when the holding hand 4 moves the stacked articles 30 to the front side, the cradle 6b is operated so as to be displaced rearward. The movement of the cradle 6b in the front-rear direction is performed in synchronization with the movement of the article 30 that is attracted and held (the movement of the robot arm 3). By displacing the cradle 6b rearward, the article 30 attracted and held can be received from below. By receiving from below by the receiving portion 6, the article 30 is prevented from being deformed or detached from the suction pad 5b, and the robot arm 3 can be operated at high speed.

The photographing means 10 and the image distance sensor 11 shown in FIG. 1 have a rectangular shape set between the boundary line L1 and the boundary line L2 shown in FIG. 2 in the left-right direction and between the boundary line L3 and the boundary line L4 in the vertical direction. A plurality of articles 30 stacked in each shooting range are photographed. The photographing means 10 acquires the two-dimensional image data of the photographing range. The image distance sensor 11 is called a TOF (Time-оf-Flight) camera, and by measuring the reflection time of pulsed light irradiated to the shooting range, the shooting range is divided into pixel units at a plurality of positions. Acquire distance data to each position. In other words, the image distance sensor 11 acquires distance data to each position in the shooting range as surface (two-dimensional) data in one shooting. According to the TOF camera, distance data can be acquired regardless of the brightness of the work place.

As shown in FIG. 7, the control device 7b includes an image processing control unit 21, a programmable logic controller (PLC) 24, and a robot controller (robot control circuit) 25. The image processing control unit 21 includes an image processing circuit 22 and a position calculation circuit 23. The image processing circuit 22 has four front corners 31 (see FIG. 3) and a front center (front center region) 32 (FIG. 3) of the front surface (front in the photographing direction) of each article 30 based on the image data acquired by the photographing means 10. See). Further, the image processing circuit 22 acquires the detected distances between the front four corners 31 and the front center 32 from the distance data acquired by the image distance sensor 11 based on the image data acquired by the photographing means 10. The position calculation circuit 23 calculates the three-dimensional position and inclination (posture) of the front surface of the article 30 based on the distances between the front four corners 31 and the front center 32 acquired by the image processing circuit 22. The programmable logic controller 24 transmits an output signal to the robot controller 25 based on an input signal about the three-dimensional position and inclination of the front surface of the article 30 received from the image processing control unit 21 and a preset program. The robot controller 25 controls the operations of the robot arm 3 and the holding hand 4 based on the signal sent from the programmable logic controller 24.

Based on FIGS. 1 to 9, a series of steps for unloading the article 30 by detecting the three-dimensional position and inclination of the front surface of the article 30 by the image processing device 7 will be described. First, in step 01 shown in FIG. 8 (hereinafter abbreviated as ST) 01, the carriage 2 is moved to the front of the articles 30 stacked in the container 34. The dolly 2 is stopped, the robot arm 3 is operated, and the photographing means 10 and the image distance sensor 11 are moved to the photographing position where the article 30 is photographed (ST02). The photographing means 10 and the image distance sensor 11 that have moved to the photographing position have a plurality of photographing ranges between the boundary line L1 and the boundary line L2 in the left-right direction and the boundary line L3 and the boundary line L4 in the vertical direction shown in FIG. Article 30 is photographed at the same time (ST03).

Next, the image data is acquired by the photographing means 10, and the distance data is acquired by the image distance sensor 11. The image processing circuit 22 detects the gaps 33 of the plurality of articles 30 from the image data captured by the photographing means 10 (ST04). As shown in FIG. 2, as the gap 33, a vertical gap 33a extending in the vertical direction and a horizontal gap 33b extending in the horizontal direction are detected. The rectangular gap 33c is detected as a rectangular region surrounded by the gap 33a and the gap 33b, but the length in the vertical direction or the horizontal direction is smaller than the length set so as to be recognized as the article 30. , Is recognized as a gap 33. The vertical or horizontal length recognized as the article 30 by the image processing circuit 22 may be appropriately set to an upper limit, a lower limit, or both based on, for example, a typical size of the article 30, and in particular. It does not have to be set.

Next, the image processing circuit 22 removes the gap 33 detected in the image data, and as shown in FIG. 3, the rectangular contours of the plurality of articles 30 are simultaneously recognized (ST05). In the case illustrated in FIG. 3, 14 rectangular contours of articles 30a, 30b, 30c, 30d, 30e, 30f, 30g, 30h, 30i, 30j, 30k, 30l, 30m, and 30n are recognized. The image processing circuit 22 detects the front four corners 31 of each recognized article 30. Further, the image processing circuit 22 detects the front center 32 as intermediate positions in the left-right direction and the up-down direction of the front four corners 31 of the front surface (front in the photographing direction) of each article 30.

Next, the image processing circuit 22 compares the image data acquired by the photographing means 10 with the distance data acquired by the image distance sensor 11, and positions the positions corresponding to the front four corners 31 and the front center 32 of each article 30. Distance data is acquired (ST06). Based on the distance data between the front four corners 31 and the front center 32, the position calculation circuit 23 calculates the three-dimensional position and inclination of the front surface of each article 30 (ST07).

If it is difficult to fit the entire inside of the container 34 into the shooting range in one shooting, the shooting range is shifted and the shooting is divided into a plurality of times. When the entire shooting range in the container 34 has not been shot (ST08), the process returns to ST02, and the shooting means 10 and the image distance sensor 11 shoot from different shooting positions to acquire image data and distance data. When all the articles 30 in the container 34 have been photographed at all the photographing positions, information on all the articles 30 is generated (ST09). For example, the articles 30f and 30n within the photographing range shown in FIG. 3 actually extend to the right side of the photographing range (see FIG. 2). For articles 30 that are located at the boundary of the shooting range and are divided into two shooting ranges, such as articles 30f and 30n, the image data of the two shooting ranges are compared, and the articles 30 of the respective image data are displayed. The image processing circuit 22 determines whether the articles 30 are separate articles or one connected article 30. When it is determined that the article 30 is one connected article, the front four corners 31 and the front center 32 are newly calculated from the image data, and the distance data between the front four corners 31 and the front center 32 is newly acquired. Then, the position calculation circuit 23 recalculates the three-dimensional position and inclination of the front surface of the article 30. When the information of all the articles 30 is generated, the robot arm 3 is operated to move the photographing means 10 and the image distance sensor 11 to the standby position (ST10).

Next, the holding hand 4 unloads the articles 30 based on the information on the three-dimensional positions and inclinations of all the articles 30. Article 30 is unloaded in order from the upper side, for example, articles 30a, 30b, 30c, etc. shown in FIG. First, the robot arm 3 is operated to move the photographing means 10 and the image distance sensor 11 to the photographing position in front of the article 30 to be picked (held) (ST11 in FIG. 9).

Next, as shown in FIG. 9, the three-dimensional position and inclination of the front surface of the article 30 to be picked are recalculated. The procedure for recalculating is the same as ST03 to ST07 for calculating the three-dimensional positions and inclinations of the front surfaces of all the articles 30 stacked in the container 34. The photographing means 10 and the image distance sensor 11 that have moved to the photographing position photograph the periphery of the picking target article 30 as the photographing range (ST12). The image processing circuit 22 detects the gaps 33 of the plurality of articles 30 from the image data captured by the photographing means 10 (ST13). The image processing circuit 22 removes the gap 33 detected in the image data, and recognizes the rectangular contour of the article 30 (ST14). The image processing circuit 22 detects the front four corners 31 and the front center 32 of each of the recognized articles 30. By comparing the image data with the distance data, the distance data of the positions corresponding to the front four corners 31 and the front center 32 of the picking target article 30 and each article 30 around it is acquired (ST15). The position calculation circuit 23 calculates the three-dimensional position and inclination of the front surface of the article 30 to be picked and each article 30 around it based on the distance data between the front four corners 31 and the front center 32 (ST16). For example, when picking an article 30, the position of the surrounding article 30 that has been picked may shift. In order to grasp the three-dimensional position and inclination of the front surface of the article 30 with higher accuracy, each time the article 30 is picked, the three-dimensional position and inclination of the front surface of the article 30 to be picked and the surrounding articles 30 are calculated. ..

Next, the picking target article 30 is searched for from the recognized plurality of articles 30 (ST17). The position information of the article 30 to be picked is updated based on the three-dimensional position and inclination of the front surface of the article 30 calculated in ST16 (ST18). As shown in FIG. 4, the holding hand 4 is brought close to the picking target article 30 with the front center 32 as a target based on the position information of the front four corners 31 and the front center 32 on the front surface of the picking target article 30. As shown in FIG. 5, the suction pad 5b is applied to the front surface of the article 30 to be picked. The suction machine is activated to suck the suction pad 5b to the front surface of the article 30. As shown in FIG. 6, the holding hand 4 is moved forward by the robot arm 3 in a state where the article 30 is attracted to the suction pad 5b. As the article 30 moves forward with the holding hand 4, the receiving portion 6 is extended backward. The article 30 is placed on the receiving portion 6 and supported from below. Thus, the article 30 to be picked is picked (held) (ST19). According to the holding hand 4, for example, by attracting the article 30 (see FIG. 1) stacked near the ceiling of the container 34 to the suction pad 5b and picking it forward, it is easy without interfering with the ceiling of the container 34. The article 30 can be unloaded.

Next, the picked article 30 is transferred to a conveyor (not shown) for carrying the article 30 out of the container 34. The conveyor is provided separately from the transport device 1. While holding the article 30, the robot arm 3 moves the holding hand 4 upward on the conveyor (ST20). The height of the article 30 to be picked is measured on the conveyor based on the position information of the front four corners 31 on the front surface of the article 30 to be picked updated in ST18 (ST21). As a result, for example, it is possible to confirm whether or not the height information of the article 30 calculated by the position calculation circuit is accurate, and even if the height information is incorrect, it can be updated. Therefore, it is possible to prevent accidental failure of the holding hand 4 and the article 30. The holding hand 4 moved above the conveyor puts the lower surface of the article 30 on the conveyor while retracting the receiving portion 6, and stops the suction machine. As a result, the article 30 to be picked is transferred onto the conveyor (ST22). When the holding hand 4 has not finished picking all the articles 30 (ST23), in order to photograph a new article 30 to be picked and obtain image data and distance data, it returns to ST11 and the photographing means 10 and the image distance sensor. Move 11 to the shooting position. When all the articles 30 are picked and transferred to the conveyor (ST23), the holding hand 4 is moved to the standby position (ST24). With the above, the unloading work of the article 30 by the transport device 1 is completed (ST25).

According to the image processing device 7 of the transport device 1 of the present embodiment described above, as shown in FIG. 1, the image processing device 10 and the image distance sensor 11 are provided. The photographing means 10 photographs the stacked rectangular box-shaped articles 30 to acquire image data. The image distance sensor 11 measures the distance to the article 30 with respect to a plurality of positions of the article 30 and acquires it as distance data. As shown in FIG. 7, the image processing device 7 is provided with an image processing circuit 22, a position calculation circuit 23, and a robot controller 25. The image processing circuit 22 detects the front four corners 31 (see FIG. 3) on the front side in the shooting direction of the article 30 from the image data, and acquires the distance from the distance data to the front four corners 31. The position calculation circuit 23 calculates the three-dimensional position and inclination of the front surface of the article 30 based on the distances to the four front corners 31. The robot controller 25 controls a holding hand 4 for holding the article 30 and a robot arm 3 for moving the holding hand 4 based on the three-dimensional position and inclination of the article 30.

Therefore, the three-dimensional position and inclination of the front surface of the article 30 are grasped by using only the distances to the front four corners 31 of the article 30 from the distance data acquired by the image distance sensor 11. Therefore, the number of data required for the recognition process can be reduced. For example, when calculating the position and orientation of the front surface of an article using a camera capable of acquiring point cloud data, the point cloud data is acquired at predetermined intervals (for example, 5 mm intervals) in the vertical direction and the horizontal direction. Therefore, the larger the size of the front surface of the article 30, the larger the number of point cloud data. For example, when recognizing the front surface of an article 30 having a height and a left-right width of 300 mm at intervals of 5 mm, 3600 point cloud data is acquired for each article. That is, when there are 30 articles 30, it is necessary to process 108,000 data.

On the other hand, according to the image processing device 7, when there are 30 articles 30, a total of 120 pieces of data may be processed for each of the four front corners 31. As a result, the time for the recognition process can be shortened, and the three-dimensional position and inclination of the front surface of the rectangle of the article 30 facing the image distance sensor 11 can be grasped with high accuracy. Further, the number of data required for the recognition process is determined by the number of articles 30 regardless of the size of the front surface of the articles 30. Therefore, even when detecting the front four corners 31 of articles 30 of various sizes, the position and posture can be grasped in a short processing time.

Further, according to the image processing device 7, the image processing circuit 22 shown in FIG. 7 detects the front center 32 on the front side in the photographing direction of the article 30 shown in FIG. 3 from the image data, and extends from the distance data to the front center 32. Get the distance. The position calculation circuit 23 shown in FIG. 7 calculates the three-dimensional position and inclination of the front surface of the article 30 based on the distances to the front four corners 31 and the front center 32 of the front surface of the article 30 shown in FIG.

Therefore, distance data of a total of five locations, the front four corners 31 and the front center 32, is acquired and recognized for each article. As a result, the recognition process can be performed in a short time. Further, by detecting the distances to the front four corners 31 of the front surface of the article 30 and the distances to the front center 32, respectively, the three-dimensional position and inclination of the rectangular front surface of the article 30 facing the image distance sensor 11 can be more accurately determined. Can be grasped with. For example, when the rectangular box-shaped article 30 is a corrugated cardboard box or the like, the front center 32 may be located at a position slightly protruding forward from the front four corners 31, and the front may not be flat. Even in such a case, the three-dimensional position and inclination of the front surface of the article 30 can be grasped with higher accuracy while roughly grasping the shape of the front surface of the article 30 facing the image distance sensor 11.

Further, according to the image processing device 7, as shown in FIG. 1, the photographing means 10 and the image distance sensor 11 simultaneously acquire image data and distance data of a plurality of stacked articles 30.

Therefore, the three-dimensional positions and inclinations of the front surfaces of the plurality of articles 30 can be grasped at the same time. This makes it possible to shorten the time required to recognize the three-dimensional position and inclination of the front surface of the article 30. Further, by recognizing the three-dimensional positions and inclinations of the front surfaces of the plurality of articles 30 at a time, it becomes easy to select the order in which the holding hand 4 holds the articles 30 so as to improve the efficiency of the transport work. As a result, the working time can be further shortened.

Further, according to the transport device 1 provided with the image processing device 7, as shown in FIG. 1, the holding hand 4, the robot arm 3, and the carriage 2 are provided. The dolly 2 is movable with the robot arm 3 mounted on it. The photographing means 10 and the image distance sensor 11 are mounted on the carriage 2.

Therefore, it is necessary to grasp the three-dimensional position and inclination of the front surface of the article 30 without installing a special device in response to the article 30 stacked in various places such as a container 34 or a warehouse of a factory. Can be done. Further, the image data can be acquired by the photographing means 10 mounted on the carriage 2 and the distance data can be acquired by the image distance sensor 11 between the transfer operations by the holding hand 4 and the robot arm 3. Therefore, the work time can be shortened and the work efficiency can be improved. Further, for example, when the articles 30 are stacked in the container 34, the articles 30 may be stacked near the ceiling of the container 34, and it is difficult to recognize the position and posture of the articles 30 from above the articles 30. There is. Alternatively, it may be difficult to install the image processing device 7 separately from the transport device 1 in the container 34. According to the features of the present disclosure, the position and posture of the articles 30 can be grasped regardless of the place where the articles 30 are stacked.

Various changes can be made to the transport device 1 and the image processing device 7 of the present embodiment described above. The place where the photographing means 10 and the image distance sensor 11 are provided may be changed as appropriate. In the present embodiment, as described above, the photographing means 10 and the image distance sensor 11 are mounted on the carriage 2. Instead of this, for example, the photographing means 10 and the image distance sensor 11 may be installed in the container 34 separately from the transport device 1. For example, a photographing table that is immovable with respect to the robot arm 3 may be provided, and the photographing means 10 and the image distance sensor 11 may be provided on the photographing table. Due to these arrangements, it is difficult for the photographing means 10 and the image distance sensor 11 to move when the robot arm 3 operates. Therefore, the time for moving the photographing means 10 and the image distance sensor 11 to the photographing position can be shortened, and thereby the efficiency of the transport work can be improved.

For example, the photographing means 10 and the image distance sensor 11 may be provided on the rear side of the robot arm 3 mounted on the carriage 2 or below the receiving portion 6 of the holding hand 4. With this arrangement, the robot arm 3 moves the holding hand 4 in front of the picking target article 30, and after taking a picture with the photographing means 10 and the image distance sensor 11, the step immediately shifts to the step of holding the article 30 with the holding hand 4. can. Therefore, the efficiency of the transport work can be improved.

For example, the photographing means 10 and the image distance sensor 11 may be provided on the front side of the robot arm 3 or the holding hand 4. With this arrangement, when the article 30 held by the holding hand 4 is transferred to, for example, a conveyor arranged in front of the transport device 1, the photographing means 10 and the image distance sensor 11 are stacked behind the article 30. Can be turned to the side. Therefore, the next article 30 to be picked can be photographed by the photographing means 10 and the image distance sensor 11 while the article 30 to be picked is transferred to the conveyor or immediately after the transfer. Therefore, the work time of the unloading work can be shortened.

The image processing device 7 can be applied not only to the container 34 illustrated in the present embodiment but also to unloading the articles 30 stacked in a warehouse of a factory, a container for a ship, or the like. Further, the image processing device 7 can be applied not only to unloading but also to, for example, when loading or adjusting the position of each of the stacked articles 30. The transport device 1 and the image processing device 7 may be provided separately. Although an articulated robot with 6-axis control has been exemplified as the robot arm 3, an articulated robot with a simpler control axis may be used. Although the holding hand 4 that sucks and holds the front surface of the article 30 with the suction pad 5b has been illustrated, for example, the upper surface of the article 30 may be sucked and held by the suction pad. For example, the article 30 may be sandwiched and held in the left-right direction or the up-down direction by a pair of clamp mechanisms. Although the configuration in which the front center 32 is detected as an intermediate position in the left-right direction and the vertical direction of the front four corners 31 of the front surface of the article 30 is illustrated, instead of the front center 32, for example, in the vertical and horizontal directions in the middle portion of the front four corners 31 A configuration may be configured in which a front center region having a predetermined extent is detected and distance data of the front center region is acquired.

1 ... Transport device 2 ... Bogie, 2a ... Track, 2b ... Drive wheel 3 ... Robot arm, 3a ... 1st arm, 3b ... 2nd arm, 3c ... 3rd arm 4 ... Holding hand (robot hand)
5 ... Adsorption unit, 5a ... Support unit, 5b ... Adsorption pad 6 ... Receiving unit, 6a ... Cradle accommodating unit, 6b ... Cradle 7 ... Image processing device, 7a ... Imaging unit, 7b ... Control device 10 ... Imaging means ( 2D camera)
11 ... Image distance sensor (TOF camera)
21 ... Image processing control unit, 22 ... Image processing circuit, 23 ... Position calculation circuit 24 ... Programmable logic controller (PLC)
25 ... Robot controller (robot control circuit)
30, 30a to 30n ... Article 31 ... Front four corners, 32 ... Front center (front center area)
33 ... Gap 33a ... (Vertical) Gap, 33b ... (Horizontal) Gap, 33c ... (Rectangular) Gap 34 ... Containers L1, L2, L3, L4 ... (Indicating distance data detection range) Boundary line

Claims (4)

An image processing device for a transport device
A photography method for acquiring image data by photographing stacked rectangular box-shaped articles,
An image distance sensor that measures the distance to the article with respect to a plurality of positions of the article and acquires it as distance data.
An image processing circuit that detects the four front corners of the article on the front side in the shooting direction from the image data and acquires the distances from the distance data to the four front corners.
A position calculation circuit that calculates the position and posture of the article based on the distances to the four front corners,
An image processing device having a holding hand that holds the article and a robot control circuit that controls a robot arm that moves the holding hand based on the position and orientation of the article. The image processing apparatus according to claim 1.
The image processing circuit detects the front center region on the front side in the shooting direction of the article from the image data, and acquires the distance from the distance data to the front center region.
The position calculation circuit is an image processing device that calculates the position and orientation of the article based on the distances to the front four corners and the front center region. The image processing apparatus according to claim 1 or 2.
The photographing means and the image distance sensor are image processing devices that simultaneously acquire the image data and the distance data of a plurality of stacked articles. A transport device including the image processing device according to any one of claims 1 to 3.
A trolley on which the holding hand, the robot arm, and the robot arm are mounted and movable is provided.
The photographing means and the image distance sensor are transport devices mounted on the trolley.

Images