KR102719530B1 - Transpotation robot system and control method thereof - Google Patents

Abstract

A transport robot system and a control method thereof are disclosed. The transport robot system according to one aspect of the present invention includes a camera device, a driving module for driving wheels, and a processor for recognizing surrounding environment information based on image data acquired through the camera device, distinguishing a drivable area based on the surrounding environment information, generating a driving path to a worker's location using the drivable area, and controlling the driving module to drive along the driving path, thereby following the worker.

Description

{TRANSPOTATION ROBOT SYSTEM AND CONTROL METHOD THEREOF}

The present invention relates to a transport robot system and a control method thereof, and more specifically, to a transport robot system and a control method thereof that transports crops, logistics, etc., and enables the robot to easily follow a worker even when an obstacle is located between the robot and the worker.

In recent years, tasks that were previously performed by humans in various fields are gradually being replaced by those performed by robots or machines. Although robots are currently being used in many fields, the introduction of these robots is being delayed in some fields.

A representative example is the agricultural sector, where robotization is progressing slowly. Most of the work in the farming process is still performed by humans, and work machines are also operated manually by people.

Accordingly, in the agricultural field as well, research is continuously being conducted on robots that automatically perform necessary tasks in the fields.

However, despite these efforts, there is a problem in that the terrain in the field is not flat due to gravel or sand, so vibrations occur when the machine runs, which interferes with precise work.

Meanwhile, technology for robots that follow humans is disclosed in Korean Patent Publication No. 10-2011-0119438, etc.

In this type of tracking robot technology, ultrasonic waves, infrared rays, vision, lasers, etc. are used to make robots follow people.

However, conventional follower robots are very vulnerable to following when there is an obstacle between the person and the robot. For example, when a follower robot is used for transporting crops, logistics, etc., if there is an obstacle between the person and the robot, a problem may occur in which the robot cannot easily follow the person, i.e. the worker.

The background technology of the present invention is disclosed in Korean Patent Publication No. 10-2011-0119438 (published on November 2, 2011) titled ‘Human worker following mobile robot.’

The present invention has been made to improve the aforementioned problems, and an object of the present invention is to provide a transport robot system and a control method thereof that transports crops and logistics, and enables the robot to easily follow a worker even when an obstacle is located between the robot and the worker.

Another object of the present invention is to provide a transport robot system and a control method thereof that enable a worker to remotely control the transport robot system in various poses.

The problems to be solved by the present invention are not limited to the problem(s) mentioned above, and other problem(s) not mentioned will be clearly understood by those skilled in the art from the description below.

A transport robot system according to one aspect of the present invention includes a camera device, a driving module for driving wheels, and a processor for recognizing surrounding environment information based on image data acquired through the camera device, distinguishing a drivable area based on the surrounding environment information, generating a driving path to a worker's location using the drivable area, and controlling the driving module to drive along the driving path, thereby following the worker.

In the present invention, the processor recognizes surrounding environment information by applying an object recognition model to the image data, and the surrounding environment information may include at least one of a worker, a road surface, and an obstacle.

In the present invention, the processor can extract a road surface from the surrounding environment information, divide a drivable area from the extracted road surface, detect an obstacle on the drivable area, and generate a driving path to the worker's location by avoiding the obstacle.

The present invention further includes a work module for processing an obstacle, wherein the processor, if there is an area in the drivable area where obstacle avoidance is impossible, analyzes the obstacle existing in the area to determine whether the obstacle can be processed through the work module, and if the obstacle can be processed, processes the obstacle using the work module, and uses the area where the obstacle has been processed when generating a driving path, and the work module may include at least one of a first processing device for lifting and moving an obstacle, a second processing device for leveling a road surface, and a third processing device for cutting the obstacle.

In the present invention, the processor can estimate the position of the worker or obstacle by corresponding pixels of the image data to coordinates in a world coordinate system using a plurality of parameters associated with the photographing device.

The present invention further includes an actuator for adjusting the height of the housing of the robot, and the processor, when an unexpected obstacle appears while driving along the driving path, performs height and width analysis on the unexpected obstacle based on image data acquired through the photographing device, thereby adjusting the height of the housing to determine whether the unexpected obstacle can be passed, and if it can be passed, controls the actuator to adjust the height of the housing.

In the present invention, the processor can recognize the posture of the worker based on the image data, and, if the recognized posture indicates a hand signal command, control to perform an operation corresponding to the hand signal command.

The present invention further includes a loading unit for loading an article, and a weight sensor for measuring the weight of the article loaded on the loading unit, and the processor can notify the worker or a preset manager that the article has fallen on the road if the weight value measured by the weight sensor is less than the initial weight value by a preset value or more while moving along the driving path.

The present invention further includes a microphone for receiving a voice command, and the processor can control, if the voice command input through the microphone is the voice of the worker, to perform an operation corresponding to the voice command.

In the present invention, the processor can measure the intensity of a magnetic field generated from a remote control device carried by the worker, and control a movement speed to maintain a preset distance from the worker based on the intensity of the magnetic field.

A control method for a transport robot system according to another aspect of the present invention includes a step in which a processor recognizes surrounding environment information based on image data acquired through a photographing device, a step in which the processor divides a drivable area based on the surrounding environment information and generates a driving path to a worker's location using the drivable area, and a step in which the processor controls a driving module to drive along the driving path, thereby following the worker.

In the step of generating the driving path of the present invention, the processor can extract a road surface from the surrounding environment information, divide a driving area from the extracted road surface, detect an obstacle on the driving area, and generate a driving path to the worker's location by avoiding the obstacle.

In the present invention, in the step of following the worker, if an unexpected obstacle appears while driving along the driving path, the processor performs a height and width analysis of the unexpected obstacle based on image data acquired through the photographing device, and adjusts the height of the housing to determine whether the unexpected obstacle can be passed, and if it can be passed, controls the actuator to adjust the height of the housing.

The present invention may further include, after the step of following the worker, a step of controlling the processor to recognize the posture of the worker based on the image data and, if the recognized posture indicates a hand signal command, perform an operation corresponding to the hand signal command.

In addition, other methods for implementing the present invention, other systems and computer programs for executing the methods may be further provided.

A transport robot system and a control method thereof according to one embodiment of the present invention generate a driving path to a worker's location based on surrounding environment information recognized based on image data acquired through a photographing device, thereby transporting crops, logistics, etc., and enabling the robot to easily follow the worker even when an obstacle is located between the robot and the worker.

A transport robot system and a control method thereof according to one embodiment of the present invention can remotely control a transport robot system in various postures of a worker by setting various postures of a worker and hand signal commands for each posture.

Meanwhile, the effects of the present invention are not limited to the effects mentioned above, and various effects may be included within a range obvious to those skilled in the art from the contents described below.

FIG. 1 is a block diagram schematically illustrating a transport robot system according to one embodiment of the present invention.
FIG. 2 is a perspective view of a motion robot system according to one embodiment of the present invention.
FIG. 3 is an exemplary diagram illustrating a method for a transport robot system according to one embodiment of the present invention to follow a worker.
FIG. 4 is an exemplary diagram for explaining details of a receiving signal command according to one embodiment of the present invention.
FIG. 5 is a flowchart for explaining a control method of a transport robot system according to one embodiment of the present invention.

Hereinafter, a transport robot system and a control method thereof according to one embodiment of the present invention will be described with reference to the attached drawings. In this process, the thickness of lines and the size of components illustrated in the drawings may be exaggerated for the sake of clarity and convenience of explanation.

Furthermore, the implementations described herein may be implemented as, for example, a method or process, an apparatus, a software program, a data stream or a signal. Even if discussed in the context of only a single form of implementation (e.g., discussed only as a method), the implementation of the discussed features may also be implemented in other forms (e.g., as an apparatus or a program). The apparatus may be implemented as suitable hardware, software, firmware, and the like. The method may be implemented in a device such as a processor, which generally refers to a processing device including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. A processor also includes a communication device such as a computer, a cell phone, a personal digital assistant ("PDA"), and other devices that facilitate communication of information between end-users.

In addition, the terminology used herein is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" and the like are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof. The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the attached drawings. When describing with reference to the attached drawings, identical or corresponding components are assigned the same drawing numbers and redundant descriptions thereof are omitted.

FIG. 1 is a block diagram schematically showing a transport robot system according to one embodiment of the present invention, FIG. 2 is a perspective view of a locomotion robot system according to one embodiment of the present invention, FIG. 3 is an exemplary diagram explaining a method for a transport robot system according to one embodiment of the present invention to follow a worker, and FIG. 4 is an exemplary diagram explaining a posture for a hand signal command according to one embodiment of the present invention.

Referring to FIG. 1, a transport robot system (100) according to one embodiment of the present invention includes a housing (101), a communication module (110), a photographing device (120), a memory (130), a driving module (140), a processor (160), a loading unit (170), and a sensor (180).

The housing (101) may be formed in various shapes such as a rectangle, and a space may be provided inside in which a communication module (110), a memory (130), a driving module (140), a processor (160), and a sensor (180) may be mounted. At this time, the shape or material of the housing (101) may be modified in various ways. In addition, a photographing device (120) may be mounted on the upper part of the housing (101), and a wheel (150) may be mounted on the lower part of the housing (101).

The housing (101) can maintain a waterproof form so as to be able to respond to changes in the external environment, such as weather changes such as rain.

The communication module (110) may provide a communication interface necessary to provide a transmission/reception signal between the transport robot system (100) and a remote control device (not shown) in the form of packet data by linking with a communication network. In particular, the communication module (110) may receive a control command from the remote control device. Here, the communication network may be a medium that performs a role of connecting the transport robot system (100) and the remote control device. In addition, the communication module (110) may be a device including hardware and software necessary to transmit/receive signals such as a control signal or a data signal through a wired/wireless connection with another network device. In addition, the communication module (110) may be implemented in various forms such as a short-range communication module, a wireless communication module, a mobile communication module, and a wired communication module.

The memory (130) is a configuration that stores data related to the operation of the transport robot system (100). In particular, the memory (130) may store applications (programs or applets) that recognize surrounding environment information based on image data, divide a drivable area based on the surrounding environment information, generate a driving path to the worker's location using the drivable area, and drive along the driving path, and the stored information may be selectively selected by the processor (160) as needed. That is, the memory (130) stores various types of data generated during the execution of an operating system for driving the transport robot system (100) or an agricultural crop cultivation game application (program or applet). At this time, the memory (130) collectively refers to a nonvolatile storage device that maintains stored information even when power is not supplied and a volatile storage device that requires power to maintain stored information. In addition, the memory (130) may perform a function of temporarily or permanently storing data processed by the processor (160). Here, the memory (130) may include a magnetic storage media or a flash storage media in addition to a volatile storage device that requires power to maintain stored information, but the scope of the present invention is not limited thereto.

The photographing device (120) can obtain image data and transmit it to the processor (160). The photographing device (120) can be installed to have a field of view that can photograph the front.

These photographing devices (120) may include cameras, RGB cameras, deep learning cameras, depth cameras, vision cameras, stereo cameras, etc.

The sensor (180) includes a position sensor (182) and a weight sensor (184).

The position sensor (182) can collect position information of the transport robot system (100) and transmit it to the processor (160). The position sensor (182) can include GPS, Glonass, Galileo, etc.

The weight sensor (184) can measure the weight of an object loaded on the loading section (170) for transport by the transport robot system (100).

The sensor (180) according to the present invention may further include a distance measuring sensor (not shown). The distance measuring sensor enables the worker (10) to be recognized from behind the worker (10). The distance measuring sensor may be one of a laser scanner or an ultrasonic sensor, but is not limited thereto. The laser scanner can detect the distance of an object within 4 m within a range of 0° to 180°.

The drive module (140) moves the transport robot system (100) using power supplied from a power supply (not shown), and may be a motor drive driver that drives a wheel (150) provided in the transport robot system (100). The drive module (140) includes a motor that provides driving force to the wheel (150), and can drive the transport robot system (100) according to the driving force generated by the motor.

The wheel (150) is installed at the bottom of the housing (101) and can move the transport robot system (100). As shown in FIG. 3, the transport robot system (100) follows the worker (10) while driving on an outdoor road, and the wheel (150) can be configured to ensure smooth driving even in environments such as rain.

The loading section (170) can be loaded with items to be transported using the transport robot system (100). For example, crops, logistics, etc. can be loaded into the loading section (170).

The loading section (170) may be formed on the upper part of the housing (101) or may be formed inside the housing (101). In addition, the loading section (170) may be composed of a storage space (not shown) for storing items and a door (not shown) installed on one side of the storage space. If a storage space that opens and closes by a door is installed, there is an advantage in that theft of items or damage due to collision can be prevented during transport. In addition, the loading section (170) may further include a conveyor (not shown) that automatically loads and unloads items.

The processor (160) recognizes surrounding environment information based on image data acquired through the photographing device (120), divides a drivable area based on the surrounding environment information, generates a driving path to the worker's location using the drivable area, and controls the driving module (140) to drive along the driving path, thereby enabling the worker (10) to be followed.

Below, the operation of the processor (160) will be described in detail.

The processor (160) can recognize surrounding environment information by applying an object recognition model (driving environment recognition model) to image data. The object recognition model (driving environment recognition model) is a deep learning model for detecting driving environment information from image data, and is an image-based object detection model. Here, the surrounding environment information can include objects, road surfaces, etc., and the objects can include workers (10), obstacles, etc. For example, the processor (160) can detect objects from image data using an object recognition algorithm such as YOLO or Fast R-CNN.

The processor (160) can simultaneously perform a multi-task of detecting driving environment information in an integrated manner using an object recognition model. For example, the processor (160) can simultaneously perform a task of detecting a worker (10) from image data, a task of detecting a road surface, etc. by executing an object recognition model. When the worker (10) is detected, the processor (160) can perform labeling by assigning an ID to each of the worker (10) and other objects (obstacles) to distinguish them from the worker (10).

When the surrounding environment information is detected, the processor (160) can extract a road surface from the surrounding environment information, divide a drivable area from the extracted road surface, detect obstacles on the drivable area, and generate a driving path to the worker's location by avoiding the obstacles.

Specifically, the processor (160) can detect a drivable area by distinguishing a road surface such as a road, a ground, or a flat surface through image segmentation, and determine whether drivability is possible based on the presence or absence of an obstacle and the location of the obstacle through object recognition on the drivable area. That is, if there is no obstacle in the drivable area, the processor (160) can determine that the area is drivable and use it when generating a driving path.

If an obstacle exists in the drivable area, the processor (160) can identify the location of the obstacle. At this time, the processor (160) can estimate the location of the obstacle (object) by corresponding the pixels of the image data to the coordinates on the world coordinate system using a plurality of parameters associated with the photographing device (120).

That is, the processor (160) can calculate actual location information based on parameters related to the photographing device (120). Here, the actual location information can be a location related to real coordinates, and can be said to be a location based on coordinates linked to the World Coordinate System (WCS). This is basically calculated as a relative real distance from the photographing device (120). That is, if x=-2, y=1, it can indicate a location 1 meter in front of the photographing device (120) and 2 meters to the left. The photographing device-related parameters can include the height of the photographing device (120) from the ground, the vertical inclination (the inclination toward the ground), the horizontal inclination, the focal length of the photographing device (120), the tilt angle, the horizontal/vertical photographing range, etc. Through these parameters, the processor (160) can calculate the position in the world coordinate system of the shooting area (relative position from the camera), and further, can precisely specify the position of the worker (10) in the shooting image as a relative distance.

In other words, the processor (160) can confirm the coordinates of the target object (worker (10), obstacle, etc.) in the image data acquired through the photographing device (120). At this time, the processor (160) can use a simple image recognition technology for recognizing an object in an existing image to confirm the coordinates of the target object (e.g., worker (10), obstacles, etc.) in the image data captured by the photographing device (120). Then, the processor (160) can convert the coordinates of the object in the image into spatial coordinates for the actual shooting space of the captured image in which the angular ratio of the vertical shooting range is reflected based on parameters related to the photographing device.

For example, the processor (160) may calculate a vertical shooting angle that reflects the angular ratio of the vertical shooting range by using the y coordinate within the image of the shooting image based on the installation height, tilt angle, and vertical shooting range of the shooting device (120), and may calculate a y coordinate within the shooting space that matches the y coordinate within the image by using the calculated vertical shooting angle.

Image data acquired through a shooting device (120) is expressed as a digital image in pixel units, and the processor (160) can convert the location of each pixel unit in the image data by mapping it to the actual location of the actual shooting space, reflecting the fact that the image data and the corresponding actual shooting space have different shapes.

If the absolute position of the photographing device (120), i.e., the latitude and longitude position on a GIS (geographic information system), is known, the processor (160) can calculate the relative position of the photographing area from the photographing device (120) by a method of corresponding the photographing device coordinate system and the world coordinate system, and can calculate the absolute position of the photographing area by adding the calculated position to the absolute photographing device position.

Once the location of an obstacle in the drivable area is identified, the processor (160) can generate a driving path from the current location to the worker's location as a straight line or a path that avoids the obstacle.

Meanwhile, although avoidance of obstacles in the drivable area is impossible, the transport robot system (100) may process the obstacles and use them as a driving path.

To this end, the transport robot system (100) may further include a work module (not shown) for handling obstacles. The work module may be a variety of devices for performing transport-related tasks without limitation. The work module may include a first processing device (not shown) for lifting an obstacle and moving it to another location, a second processing device (not shown) for leveling a road surface, a third processing device (not shown) for cutting an obstacle, etc.

If a work module is provided and there is an area in the drivable area where avoidance of an obstacle is impossible, the processor (160) can perform an analysis on the obstacle and determine whether the obstacle can be processed through the work module. In the case of an obstacle that can be processed through the work module, the processor (160) can process the obstacle using the work module and use the area when generating a driving path. If the obstacle cannot be processed, the processor (160) can inform the worker (10) of information about the impossibility of movement due to the obstacle.

In addition, an unexpected obstacle may appear while the transport robot system (100) is driving along the driving path. Here, the unexpected obstacle may include a stone, a bird, a cat, a dog, etc. When an unexpected obstacle appears on the driving path, the transport robot system (100) may further include an actuator (not shown) that adjusts the height of the housing (101) (body) to avoid the unexpected obstacle. The actuator may expand or contract the housing (101) in the vertical direction under the control of the processor (160), thereby enabling the robot to pass through the obstacle.

When an unexpected obstacle appears while driving along a driving path, the processor (160) performs a height analysis and a width analysis on the unexpected obstacle based on image data acquired through the photographing device (120), and determines whether the unexpected obstacle can be passed by adjusting the height of the housing (101) (body), and if it can be passed, the height of the housing (101) (body) can be adjusted by controlling the actuator. By adjusting the height of the housing (101), the transport robot system (100) can overcome the unexpected obstacle and drive along the driving path.

Meanwhile, in the case of the transport robot system (100) configured as described above, the worker (10) may have difficulty in directly operating the transport robot system (100) because the worker (10) is a certain distance away from the transport robot system (100). In addition, the worker (10) may have difficulty in operating the transport robot system (100) using a switch or remote control because the worker (10) is holding gloves or an object. In addition, the worker (10) may want to control the transport robot system (100) from a distance with a specific hand signal command. In this case, the processor (160) may analyze the image data acquired through the photographing device (120) to recognize the posture of the worker (10), and if the recognized posture indicates a hand signal command, control to perform an operation corresponding to the hand signal command. At this time, various postures (poses) and hand signal commands for each posture (pose) are stored in the memory (130). For example, in the case of the posture shown in (a) of FIG. 4, it may be a hand signal command indicating "stop". The posture shown in (b) of Fig. 4 may be a hand signal command indicating “move.”

There may be multiple users in the image data acquired through the camera (120). In this case, the processor (160) must detect the worker (10) among the multiple users and recognize the posture of the detected worker (10). Accordingly, the processor (160) may acquire the face image of each user using a face recognition algorithm and detect the worker (10) by comparing the face image with the worker's face stored in the memory (130).

In addition, the transport robot system (100) can receive control commands not only by hand signal commands but also by voice. In this case, the transport robot system (100) can further include a microphone (not shown) for receiving voice commands. When a voice is input through the microphone, the processor (160) can determine whether the voice command input through the microphone is the voice of a preset worker (10). If the determination result is that it is the voice of the worker (10), the processor (160) can control to perform an operation corresponding to the voice command.

While moving along the driving path, if the weight value measured by the weight sensor (184) is less than the initial weight value by a preset value or more, the processor (160) can notify the worker (10) or a preset manager. That is, if the weight value measured by the weight sensor (184) is less than the initial weight value by a preset value or more, it can be determined that the item has fallen on the road surface. Accordingly, the processor (160) can notify the worker (10) or manager that the item has fallen on the road surface. At this time, the processor (160) can notify that the item has fallen on the road surface by using a sound, a text message, or the like.

When the worker (10) moves with a remote control device that remotely controls the transport robot system (100), the processor (160) can measure the intensity of the magnetic field generated by the remote control device and control the direction or speed of the transport robot system (100) so that the transport robot system (100) maintains a preset distance from the worker (10) based on the intensity of the magnetic field. For example, when the magnetic field intensity is compared with a preset reference magnetic field value and the error increases, the processor (160) can increase the moving speed of the transport robot system (100). When the magnetic field intensity becomes lower than the set allowable error with respect to the micro reference magnetic field value, the processor (160) can continue to move the transport robot in the direction in which it was moving just before.

By automatically allowing the transport robot system (100) to maintain a certain distance from the worker (10), the worker (10) can easily load and transport crops, etc. from the location where they were harvested onto the transport robot system (100).

FIG. 5 is a flowchart for explaining a control method of a transport robot system according to one embodiment of the present invention.

Referring to FIG. 5, the processor (160) acquires image data through the photographing device (120) (S502) and recognizes surrounding environment information based on the acquired image data (S504). At this time, the processor (160) can recognize surrounding environment information by applying an object recognition model (driving environment recognition model) to the image data. The surrounding environment information can include objects, road surfaces, etc., and the objects can include workers (10), obstacles, etc.

When step S504 is performed, the processor (160) divides a drivable area based on the surrounding environment information (S506) and generates a driving path to the position of the worker (10) using the drivable area (S508). That is, the processor (160) detects a drivable area by distinguishing a road surface such as a road, a floor, or a flat surface through image segmentation, and determines whether driving is possible based on the presence or absence of an obstacle and the location of the obstacle through object recognition on the drivable area. If there is no obstacle in the drivable area, the processor (160) determines the area as drivable and can use it when generating a driving path. If there is an obstacle in the drivable area, the processor (160) can identify the location of the obstacle. At this time, the processor (160) can estimate the location of the obstacle (object) by corresponding the pixels of the image data with the coordinates on the world coordinate system using a plurality of parameters associated with the photographing device (120). Once the location of an obstacle in the drivable area is identified, the processor (160) can generate a driving path from the current location to the worker's location as a straight line or a path that avoids the obstacle.

When step S508 is performed, the processor (160) controls driving along a driving path (S510). When a driving path is generated, the processor (160) controls the driving module (140) to drive along the driving path, thereby allowing the transport robot system (100) to follow the worker (10).

During the execution of step S510, if a hand signal command from the worker (10) is detected (S512), the processor (160) performs an operation corresponding to the hand signal command (S514). At this time, the processor (160) analyzes the image data acquired through the photographing device (120) to recognize the posture of the worker (10), and if the recognized posture indicates a hand signal command, it can control to perform an operation corresponding to the hand signal command.

As described above, the transport robot system and its control method according to one embodiment of the present invention generate a driving path to a worker's location based on surrounding environment information recognized based on image data acquired through a photographing device, thereby transporting crops, logistics, etc., and enabling the robot to easily follow the worker even when an obstacle is located between the robot and the worker.

A transport robot system and a control method thereof according to one embodiment of the present invention can remotely control a transport robot system in various postures of a worker by setting various postures of a worker and hand signal commands for each posture.

Although the present invention has been described with reference to the embodiments shown in the drawings, these are merely exemplary, and those skilled in the art will understand that various modifications and equivalent other embodiments are possible therefrom. Therefore, the true technical protection scope of the present invention should be defined by the following patent claims.

100 : Transport Robot System
101 : Housing
110 : Communication module
120 : Camera device
130 : Memory
140 : Drive module
150 : Wheel
160 : Processor
170 : Loading section
180 : Sensor

Claims (14)

filming device;
A driving module that drives the wheels; and
A processor is included that recognizes surrounding environment information based on image data acquired through the above-mentioned photographing device, divides a drivable area based on the surrounding environment information, generates a driving path to the worker's location using the drivable area, and controls the driving module to drive along the driving path, thereby following the worker.
A loading section for loading goods; and
Further comprising a weight sensor for measuring the weight of the goods loaded on the above loading section,
The above processor,
A transport robot system characterized in that, while moving along the above driving path, if the weight value measured by the weight sensor is less than the initial weight value by a preset value or more, it notifies the worker or a preset manager that the item has fallen on the road surface.
In the first paragraph,
The above processor,
By applying an object recognition model to the above image data, we recognize the surrounding environment information,
A transport robot system, characterized in that the above-mentioned surrounding environment information includes at least one of a worker, a road surface, and an obstacle.
In the first paragraph,
The above processor,
A transport robot system characterized by extracting a road surface from the above-mentioned surrounding environment information, dividing a drivable area from the extracted road surface, detecting obstacles on the drivable area, and generating a driving path to a worker's location by avoiding the obstacles.
In the third paragraph,
Including additional task modules for handling obstacles,
The above processor,
If there is an area in the above-mentioned drivable area where obstacle avoidance is impossible, an analysis is performed on the obstacle in the area, and whether the obstacle can be processed is determined through the above-mentioned work module. If the obstacle can be processed, the obstacle is processed using the above-mentioned work module, and the area where the obstacle is processed is used when generating a driving path.
The above working module is,
A transport robot system characterized by including at least one of a first processing device for lifting and moving an obstacle, a second processing device for leveling a road surface, and a third processing device for cutting the obstacle.
In the third paragraph,
The above processor,
A transport robot system characterized in that the position of the worker or obstacle is estimated by corresponding the pixels of the image data to coordinates in a world coordinate system using a plurality of parameters associated with the photographing device.
In the first paragraph,
Further comprising an actuator for adjusting the housing height of the robot;
The above processor,
A transport robot system characterized in that, when an unexpected obstacle appears while driving along the above driving path, a height and width analysis of the unexpected obstacle is performed based on image data acquired through the above photographing device, the height of the housing is adjusted to determine whether the unexpected obstacle can be passed, and if passage is possible, the actuator is controlled to adjust the height of the housing.
In the first paragraph,
The above processor,
A transport robot system characterized in that it recognizes the posture of the worker based on the image data, and if the recognized posture indicates a hand signal command, it controls to perform an action corresponding to the hand signal command.
delete In the first paragraph,
It further includes a microphone for receiving voice commands,
The above processor,
A transport robot system characterized in that, if a voice command input through the microphone is the voice of the worker, it is controlled to perform an action corresponding to the voice command.
In the first paragraph,
The above processor,
A transport robot system characterized in that it measures the intensity of a magnetic field generated from a remote control device carried by the worker and controls the movement speed to maintain a preset distance from the worker based on the intensity of the magnetic field.
A step in which a processor recognizes surrounding environment information based on image data acquired through a photographing device;
A step in which the processor divides a drivable area based on the surrounding environment information and generates a driving path to the worker's location using the drivable area; and
Including a step of controlling the driving module so that the processor follows the worker by driving along the driving path,
At the stage of following the above worker,
The above processor receives the weight value of the item loaded on the loading unit from the weight sensor, and while moving along the driving path, if the weight value of the item is less than the initial weight value by a preset value or more, the control method of the transport robot system is characterized in that it notifies the worker or a preset manager that the item has fallen on the road surface.
In Article 11,
In the step of generating the above driving route,
A control method for a transport robot system, characterized in that the processor extracts a road surface from the surrounding environment information, divides a drivable area from the extracted road surface, detects an obstacle on the drivable area, and generates a driving path to a worker's location by avoiding the obstacle.
In Article 11,
At the stage of following the above worker,
A control method for a transport robot system, characterized in that when an unexpected obstacle appears while driving along the driving path, the processor performs a height and width analysis of the unexpected obstacle based on image data acquired through the photographing device, determines whether the unexpected obstacle can be passed by adjusting the height of the housing, and if it can be passed, controls an actuator to adjust the height of the housing.
In Article 11,
After the step of following the above worker,
A control method for a transport robot system, characterized in that the processor further includes a step of recognizing the posture of the worker based on the image data, and, if the recognized posture indicates a hand signal command, controlling the processor to perform an operation corresponding to the hand signal command.