KR102070067B1 - Artificial intelligence Moving Robot and controlling method - Google Patents

Abstract

The artificial intelligence mobile robot learning the obstacles of the present invention and a control method thereof store a plurality of image data photographed by shooting an image while driving through an image acquisition unit, and the obstacle detection unit determines that the obstacle is located within a predetermined distance. The control unit starts a corresponding motion at a predetermined time point, and includes a controller configured to determine the corresponding motion according to the obstacle determined based on the image data acquired before the predetermined time point. The obstacle may be determined based on the captured image, and the corresponding operation may be immediately performed when the obstacle is located within a predetermined distance, and the obstacle may be determined through the image even when the same detection signal is input when detecting the obstacle with respect to a plurality of different obstacles. To perform different actions according to obstacles It can cope with obstacles to learning through obstacles using video effectively avoid obstacles and can perform the operation according to the type of obstacles.

Description

Artificial intelligence mobile robot and controlling method for learning obstacles {Artificial intelligence Moving Robot and controlling method}

The present invention relates to a mobile robot and a control method thereof, and more particularly, to a mobile robot and a method of controlling the same.

In general, a mobile robot is a device for automatically cleaning by inhaling foreign substances such as dust from the floor surface while driving by itself in the area to be cleaned without the user's operation.

The mobile robot detects the distance to obstacles such as furniture, office supplies, walls, etc. installed in the cleaning area, and performs an obstacle avoidance operation.

The mobile robot is a means for detecting an obstacle, and includes an obstacle detection means such as an infrared ray or a laser. The mobile robot avoids obstacles by detecting an obstacle within a predetermined distance based on a detection signal by a sensor such as an infrared ray or a laser.

If the path is changed immediately when an obstacle is detected, the area cannot be cleaned, so the path is changed after approaching or colliding with the obstacle as much as possible.

However, in the case of approaching and cleaning an obstacle to the cleaning, problems such as damage to the object due to the collision of the mobile robot may occur. For example, there may be a problem that the flower pot on the table may fall or an obstacle may be broken, and the position of the obstacle may be changed.

In addition, there is a problem that an error occurs in the position and the actual position determined by the mobile robot as a phenomenon such as a slip occurs in the process of collision with the obstacle.

Therefore, it is necessary to recognize the type of obstacle to operate the mobile robot according to the obstacle.

Republic of Korea Patent Publication 10-2016-01575

SUMMARY OF THE INVENTION An object of the present invention is to provide a mobile robot and a control method of photographing an image while driving and performing a motion corresponding to an obstacle through a pre-photographed image when an obstacle is detected.

According to an embodiment of the present invention, a mobile robot includes: a main body movable; An image acquisition unit for photographing an image of a driving direction; An obstacle detecting unit detecting an obstacle located in a driving direction; Stores a plurality of image data photographed through the image acquisition unit, and starts the corresponding motion at a predetermined time point determined by the obstacle detection unit to be located within a predetermined distance, the image obtained before the predetermined time point And a controller configured to determine the corresponding motion according to the obstacle determined based on data.

The control method of the mobile robot of the present invention includes the steps of: storing the image data by taking an image of the driving direction by the image acquisition unit while driving; Determining that the obstacle is located within a predetermined distance through the obstacle detecting unit; Determining a corresponding motion according to a predetermined time point at which the obstacle is determined to be located within a predetermined distance and the obstacle determined based on the previously acquired image data; Starting a corresponding motion for the obstacle at the predetermined time point; And operating according to the corresponding motion to avoid or pass through the obstacle.

The mobile robot and its control method of the present invention set a plurality of corresponding motions corresponding to the obstacles located in the driving direction based on a detection signal for the obstacles, and perform any one of the plurality of corresponding motions. Accordingly, different operations may be performed according to obstacles.

According to the present invention, when an image is taken while driving, and an obstacle is located within a predetermined distance, an obstacle is detected for a plurality of different obstacles as the selected image is analyzed by performing any one of a plurality of motions set for the obstacle. Even when the same detection signal is input at the same time, the obstacle may be determined based on the image so that different operations may be performed according to the obstacle, and thus various obstacles may be responded to.

In addition, the present invention by taking an image of the driving direction while driving, it is possible to determine the obstacle through the image taken before it at the time when the obstacle is located within a predetermined distance, the image in advance before the obstacle is located within the predetermined distance By determining the obstacle in advance, the corresponding operation can be performed immediately at the time when the obstacle is located within a predetermined distance, it is possible to effectively avoid the obstacle and to perform the operation according to the type of obstacle.

The present invention can track the obstacle based on the image to be photographed, and as soon as the operation is performed at the time when the obstacle is detected to prevent the damage of the obstacle due to the collision with the obstacle, to prevent the position error due to the collision It can be operated by detecting an environment change of the driving area.

1 is a perspective view showing a mobile robot of the mobile robot system according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating a horizontal angle of view of the mobile robot of FIG. 1.
3 is a front view of the mobile robot of FIG. 1.
4 is a diagram illustrating a bottom surface of the mobile robot of FIG. 1.
5 is a diagram schematically illustrating a system for recognizing a mobile robot and an obstacle according to an embodiment of the present invention.
6 is a block diagram illustrating main parts of a mobile robot according to an exemplary embodiment of the present invention.
7 and 8 are diagrams showing an example of an image photographed while driving and driving of the mobile robot of the present invention.
FIG. 9 is a diagram for describing a detection signal and an obstacle recognition according to an obstacle of a mobile robot according to an embodiment of the present invention.
FIG. 10 is a diagram referred to for describing an operation according to a type of an obstacle while driving of a mobile robot according to an embodiment of the present invention.
FIG. 11 is a diagram referred to for describing a signal flow between a mobile robot and a device for obstacle recognition according to an embodiment of the present invention.
12 is a flowchart referred to for explaining a process of performing an obstacle detection and a corresponding operation of a mobile robot according to an embodiment of the present invention.
FIG. 13 is a flowchart referred to for describing obstacle recognition and a control method according to an embodiment of the present invention. FIG.

Advantages and features of the present invention, and methods for achieving them will be apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be embodied in various different forms, and the present embodiments merely make the disclosure of the present invention complete, and those of ordinary skill in the art to which the present invention belongs. It is provided to fully inform the person having the scope of the invention, which is defined only by the scope of the claims. Like reference numerals refer to like elements throughout. The control configuration of the present invention may be composed of at least one processor.

1 is a perspective view of a mobile robot according to an embodiment of the present invention. 2 illustrates a horizontal angle of view of the mobile robot of FIG. 1. 3 is a front view of the mobile robot of FIG. 1. 4 illustrates a bottom surface of the mobile robot of FIG. 1.

1 to 4, the mobile robot 1 according to an embodiment of the present invention moves along the bottom of the cleaning area, the main body 10 and the main body 10 for sucking foreign matter such as dust on the floor It is disposed in the front of the includes a sensing means for detecting the obstacle.

The main body 10 has a casing 11 that forms an exterior and forms a space in which the components constituting the main body 10 are accommodated, and a suction unit disposed in the casing 11 to suck foreign substances such as dust or garbage. 34, the left wheel 36 (L) and the right wheel 36 (R) rotatably provided in the casing 11. As the left wheel 36 (L) and the right wheel 36 (R) rotate, the main body 10 moves along the bottom of the cleaning area, and foreign matter is sucked through the suction unit 34 in this process.

The suction unit 34 may include a suction fan (not shown) for generating a suction force and a suction port 10h through which air flow generated by the rotation of the suction fan is sucked. The suction unit 34 may include a filter (not shown) that collects foreign matters from the airflow sucked through the suction port 10h, and a foreign matter collecting container (not shown) in which foreign matters collected by the filter are accumulated.

In addition, the main body 10 may include a driving unit for driving the left wheel 36 (L) and the right wheel 36 (R). The driving unit may include at least one driving motor. The at least one drive motor may include a left wheel drive motor for rotating the left wheel 36 (L) and a right wheel drive motor for rotating the right wheel 36 (R).

The left wheel drive motor and the right wheel drive motor may be driven straight, backward or swiveling of the main body 10 by independently controlling the operation by the traveling control unit of the controller. For example, when the main body 10 travels straight, when the left wheel drive motor and the right wheel drive motor rotate in the same direction, but the left wheel drive motor and the right wheel drive motor rotate at different speeds or in opposite directions. The running direction of the main body 10 may be switched. At least one auxiliary wheel 37 may be further provided to stably support the main body 10.

A plurality of brushes 35 positioned at the front side of the bottom of the casing 11 and having a brush composed of a plurality of radially extending wings may be further provided. The dust is removed from the bottom of the cleaning area by the rotation of the plurality of brushes 35, and the dust separated from the bottom is sucked through the suction port 10h and collected in the collecting container.

The upper surface of the casing 11 may be provided with a control panel including an operation unit 160 for receiving various commands for controlling the mobile robot 1 from the user.

The sensing means includes a sensor unit 150 for detecting an obstacle using a plurality of sensors as shown in FIG. 1A, and an image acquisition unit 170 for capturing an image.

In addition, the sensing means, as shown in Figure 1 (b), may be disposed in the front of the main body 10, the obstacle detecting unit 100 for detecting the obstacle, and may include an image acquisition unit 170 for taking an image. have. The obstacle detecting unit 100 detects an obstacle through an image that is irradiated with an optical pattern and photographed. The obstacle detecting unit 100 may include a pattern acquisition unit 140 and may further include a sensor unit 150.

The image acquisition unit 170 may be provided to face the front direction, respectively, and may be provided to face the ceiling in some cases. In some cases, two image acquisition units 170 facing the front and the ceiling may be provided. When two image acquisition units are provided, the front and top portions of the main body may be respectively provided to capture an image of the front and the ceiling.

The obstacle detecting unit 100 may be disposed in front of the main body 10.

The obstacle detecting unit 100 is fixed to the front surface of the casing 11 and includes a first pattern irradiator 120, a second pattern irradiator 130, and a pattern acquisition unit 140. In this case, the pattern acquisition unit 140 may be disposed below the pattern irradiation unit as shown, or disposed between the first and second pattern irradiation units.

In FIG. 2, the displayed irradiation angle θh represents the horizontal irradiation angle of the first pattern light P1 irradiated from the first pattern irradiation unit 120, and both ends of the horizontal line form the first pattern irradiation unit 120. It represents an angle, but is preferably determined in the range of 130 ° to 140 °, but is not necessarily limited thereto. The dotted line shown in FIG. 2 is directed toward the front of the mobile robot 1, and the first pattern light P1 may be configured to be symmetrical with respect to the dotted line.

The main body 10 includes a rechargeable battery 38, and the charging terminal 33 of the battery 38 is connected to a commercial power source (for example, a power outlet in a home) or a separate charging stand connected to the commercial power source. The main body 10 is docked to the 400, and the charging terminal 33 is electrically connected to a commercial power source through contact with the terminal 410 of the charging stand, and the battery 38 may be charged. The electric components constituting the mobile robot 1 may be supplied with power from the battery 38, and thus, the mobile robot 1 is magnetically separated from the commercial power source while the battery 38 is charged. Driving is possible.

5 is a diagram schematically illustrating a system for recognizing a mobile robot and an obstacle according to an embodiment of the present invention.

The mobile robot 1 receives a cleaning command for a predetermined cleaning area of the travel area H to perform cleaning. The mobile robot 1 analyzes an image taken while driving to determine the type of obstacle.

The mobile robot 1 detects a plurality of obstacles 2 and 3 present in the driving area H to recognize the obstacles, and performs any one of approaching, passing, and avoiding according to the type of the obstacle. At this time, the mobile robot 1 may avoid the obstacle without approaching, or may perform a predetermined operation after approaching a predetermined distance, may avoid after approaching the obstacle, and may pass through the obstacle according to the shape of the obstacle. have.

The mobile robot 1 photographs the driving direction through the image acquisition unit 170 provided while driving and detects an obstacle located within a predetermined distance through the obstacle detecting unit 100. The mobile robot 1 may continuously photograph the driving direction through the image acquisition unit 170, and may photograph images at predetermined time intervals according to a setting.

The mobile robot 1 stores an image photographed while driving, and when an obstacle is detected through the obstacle detecting unit 100, selects a corresponding corresponding motion, and according to the obstacle determined through image analysis, in particular, It operates by determining the corresponding motion for obstacles according to the type.

Even if the same detection signal is input by the obstacle detecting unit 100 to the different obstacles, the mobile robot 1 analyzes the obstacles based on the pre-recorded image, and sets a corresponding motion according to the obstacles. Perform the action corresponding to the obstacle.

In addition, when the image is taken, the mobile robot 1 analyzes the image at predetermined time intervals to recognize the obstacle included in the image, and when the obstacle is detected by the obstacle detecting unit 100 within a predetermined distance, An operation for the corresponding motion may be performed based on the type of the determined obstacle.

For example, the mobile robot 1 avoids the dangerous obstacles according to the obstacles without approaching them, and in the case of the general obstacles, approaches the proximity or avoids the obstacles after the collision.

The mobile robot 1 may determine the type of the obstacle based on the data stored therein, and may be connected to the server 90 or the terminal 80 to determine the type of the obstacle. The mobile robot 1 may be connected to a separate communication device (not shown) to communicate with the server 90 through the network (N).

The mobile robot 1 filters each image data with respect to the image photographed while driving, and transmits the information to the server or the terminal at a predetermined time interval to request obstacle information, thereby providing information on the obstacle included in the image data. Receive from the terminal. The mobile robot 1 may immediately identify the obstacle through the image data when it is determined by the obstacle detecting unit 100 that the obstacle is located within a predetermined distance.

The terminal 80 includes an application for controlling the mobile robot 1, displays a map of the driving area to be cleaned by the mobile robot 1 through execution of the application, and sets an area to clean a specific area on the map. Can be specified. In addition, the position of the mobile robot is displayed based on the data received from the mobile robot, and information about the cleaning state is displayed.

The terminal is a device equipped with a communication module is connected to the network and the program for controlling the mobile robot, or the application for controlling the mobile robot, a device such as a computer, laptop, smartphone, PDA, tablet PC may be used. In addition, the terminal may also use a wearable device such as a smart watch.

The terminal 80 may be connected to the mobile robot through a network built in the home. The terminal and the mobile robot may be connected to each other using WIFI as well as short range wireless communication such as Bluetooth, infrared communication, and Zigbee. It is specified that the communication method between the terminal and the mobile robot is not limited thereto.

In addition, the terminal 80 may determine the type of obstacle in response to the image received from the mobile robot 1. The terminal 80 may determine the type of the obstacle based on the stored data, or may be connected to the server to determine the type of the obstacle. For example, when the mobile robot 1 cannot connect to the server through a network connection, the terminal 80 may receive data of the mobile robot and transmit it to the server.

In addition, the terminal 80 may display an image on a screen, input a type of obstacle through a user input, and transmit the type of obstacle to a mobile robot, and designate an operation for a specific obstacle through a menu of an application.

The server 90 analyzes the image data received from the mobile robot 1, which is connected through a predetermined network N, determines the type of obstacle, and transmits a response to the mobile robot 1. If there is a request from the terminal, the server 90 may determine and respond to the type of obstacle.

The server 90 includes a database (not shown) for obstacle recognition based on image data received from the plurality of mobile robots 1, and recognizes a feature of the obstacle extracted from the image data to determine the type of obstacle. To judge.

The server 90 accumulates and stores obstacle information, and analyzes the obstacle information to determine the type of obstacle. In addition, the server 90 may classify obstacles according to types to set the operation of the mobile robot 1 with respect to the obstacles.

The server 90 analyzes new obstacle information and updates previously stored obstacle information. In addition, the server 90 receives and stores the motion information of the mobile robot on the obstacle set or changed by the terminal 80, and sets the setting of the motion of the mobile robot on the obstacle by matching the previously stored obstacle information. Update

6 is a block diagram illustrating main parts of a mobile robot according to an exemplary embodiment of the present invention.

As shown in FIG. 6, the mobile robot 1 includes an obstacle detecting unit 100, an image acquisition unit 170, a cleaning unit 260, a traveling unit 250, a data unit 180, and an output unit 190. , An operation unit 160, a communication unit 280, and a control unit 110 for controlling the overall operation.

The manipulation unit 160 includes at least one button, a switch, and a touch input means, and receives an on / off or various commands necessary for the overall operation of the mobile robot 1 and inputs the same to the controller 110.

The output unit 190 includes a display such as an LED and an LCD, and displays an operation mode, reservation information, a battery state, an operation state, an error state, and the like of the mobile robot 1. In addition, the output unit 190 includes a speaker or a buzzer and outputs a predetermined sound effect, a warning sound, or a voice guidance corresponding to an operation mode, reservation information, a battery state, an operation state, and an error state.

The communication unit 280 communicates with the terminal 80 in a wireless communication method. In addition, the communication unit 280 may be connected to the Internet network through a home network to communicate with an external server 90.

The communication unit 280 transmits the generated map to the terminal 80, receives a cleaning command from the terminal, and transmits data on an operation state and a cleaning state of the mobile robot to the terminal. The communication unit 280 transmits and receives data including communication modules such as Wi-Fi and WiBro, as well as short-range wireless communication such as Zigbee and Bluetooth.

The communication unit 280 transmits information on the obstacle detected by the obstacle detecting unit 100 to the server 90 through the communication unit 280, and receives data about the obstacle from the server. In addition, the communication unit 280 receives from the terminal 80 information about the obstacle existing in the driving zone and the corresponding operation information, and transmits the operation data of the mobile robot to the terminal 80.

The driving unit 250 includes at least one driving motor to allow the mobile robot to travel according to a control command of the driving control unit 113. As described above, the driving unit 250 may include a left wheel driving motor for rotating the left wheel 36 (L) and a right wheel driving motor for rotating the right wheel 36 (R).

The cleaning unit 260 operates the brush to make it easy to inhale dust or foreign matter around the mobile robot, and operates the suction device to suck in dust or foreign matter. The cleaning unit 260 controls the operation of the suction fan provided in the suction unit 34 that sucks the foreign matter such as dust or garbage so that the dust is introduced into the foreign matter collecting container (dust box) through the suction port.

In addition, the cleaning unit 260 may further include a mop cleaning unit (not shown) installed in the rear of the bottom portion of the main body in contact with the bottom surface and mopping the bottom surface, and a water tank (not shown) for supplying water to the mop cleaning unit. .

The battery (not shown) supplies not only a drive motor, but also power necessary for the overall operation of the mobile robot 1. When the battery is discharged, the mobile robot 1 may travel to return to the charging station 400 for charging, and during this return driving, the mobile robot 1 may detect the location of the charging station by itself. The charging unit 400 may include a signal transmitter (not shown) for transmitting a predetermined return signal. The return signal may be an ultrasonic signal or an infrared signal, but is not necessarily limited thereto.

In the data unit 180, a detection signal input from the obstacle detecting unit 100 or the sensor unit 150 is stored, reference data for determining an obstacle is stored, and obstacle information on the detected obstacle is stored.

The data unit 180 stores obstacle data 181 for determining the type of obstacle, image data 182 for storing a captured image, and map data 183 for a region. The map data 183 includes obstacle information, so that the user can check the basic map of the movable area searched by the mobile robot, the cleaning map in which the area is separated from the base map, and the shape of the area of the cleaning map. The generated user map and a guide map in which the cleaning map and the user map are overlapped can be stored.

The obstacle data 181 is data for obstacle recognition and determination of types of obstacles, motion information about movements of the robot, for example, traveling speed, driving direction, avoidance, stop, etc. Data about the effect sound, warning sound, and voice guidance output through 173 are included. The image data 182 includes captured images and recognition information for recognizing obstacles received from a server.

In addition, the data unit 180 stores control data for controlling the operation of the mobile robot, data according to the cleaning mode of the mobile robot, and detection signals such as ultrasonic waves / lasers by the sensor unit 150.

The data unit 180 stores data that can be read by a microprocessor, and includes a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, and a CD. -May include ROM, magnetic tape, floppy disks, optical data storage devices.

The obstacle detecting unit 100 includes a first pattern irradiator 120, a second pattern irradiator 130, and a pattern acquirer 140. When the pattern irradiation unit is not provided as illustrated in FIG. 1A, an obstacle may be detected from an image captured by the image acquisition unit without a pattern to be irradiated.

In addition, the image acquisition unit may be provided in plurality. When the pattern irradiation unit is provided, an image acquisition unit for photographing an image including a pattern and a separate image acquisition unit for photographing a driving direction or an upper portion of the front may be further provided.

The obstacle detecting unit 100 includes a first pattern irradiator 120, a second pattern irradiator 130, and a pattern acquisition unit 140. In addition, the obstacle detecting unit 100 may include a sensor unit 150 composed of at least one sensor. In some cases, the obstacle detecting unit 100 may be configured as a sensor unit.

As described above, the obstacle detecting unit 100 is installed on the front surface of the main body 10 to irradiate the first and second patterns of light P1 and P2 to the front of the mobile robot and to radiate the light of the irradiated pattern. Shoot to obtain an image including the pattern. The obstacle detecting unit 100 inputs the acquired image to the controller 110 as an obstacle detecting signal.

The first and second pattern irradiation units 120 and 130 of the obstacle detecting unit 100 may include a light source and an optical pattern projection element (OPPE) that generates a predetermined pattern by transmitting light emitted from the light source. Can be. The light source may be a laser diode (LD), a light emitting diode (LED), or the like. Laser light is superior to other light sources in terms of monochromaticity, straightness, and connection characteristics, and accurate distance measurement is possible. In particular, infrared or visible light may vary in accuracy of distance measurement depending on factors such as color and material of the object. Since there is a problem that is largely generated, a laser diode is preferable as the light source. The pattern generator may include a lens and a diffractive optical element (DOE). Various patterns of light may be irradiated according to the configuration of the pattern generators provided in the pattern irradiation units 120 and 130.

The pattern acquisition unit 140 may acquire an image of the front of the main body 10. In particular, the pattern light P1 and P2 appear in an image acquired by the pattern acquisition unit 140 (hereinafter, referred to as an acquired image). Hereinafter, the image of the pattern light P1 and P2 shown in the acquired image is lighted. The pattern is referred to as a pattern, and since the pattern light P1 and P2 incident on the actual space are images formed on the image sensor, the same reference numerals as those of the pattern lights P1 and P2 are assigned to the first pattern light P1. ) And the images corresponding to the second pattern light P2 are referred to as a first light pattern P1 and a second light pattern P2, respectively.

When the pattern irradiator is not provided, the pattern acquisition unit 140 acquires an image of the front of the main body that does not include pattern light.

The pattern acquisition unit 140 may include a camera that converts an image of a subject into an electrical signal and then converts the image into a digital signal and stores the image in a memory device. The camera comprises at least one optical lens, an image sensor (e.g. CMOS image sensor) configured to include a plurality of photodiodes (e.g. pixels) formed by light passing through the optical lens, It may include a digital signal processor (DSP) that forms an image based on the signals output from the diodes. The digital signal processor may generate not only a still image but also a moving image composed of frames composed of the still image.

The image sensor is an apparatus for converting an optical image into an electrical signal, and is composed of a chip in which a plurality of photo diodes are integrated. For example, a pixel may be a photo diode. Charges are accumulated in each pixel by an image formed on the chip by light passing through the lens, and the charges accumulated in the pixels are converted into an electrical signal (for example, a voltage). As the image sensor, a charge coupled device (CCD), a complementary metal oxide semiconductor (CMOS), and the like are well known.

The image processor generates a digital image based on the analog signal output from the image sensor. The image processor includes an AD converter for converting an analog signal into a digital signal, a buffer memory for temporarily recording digital data in accordance with a digital signal output from the AD converter, and information recorded in the buffer memory. It may include a digital signal processor (DSP) for processing to form a digital image.

The obstacle detecting unit 100 analyzes the pattern through the acquired image to detect the obstacle according to the shape of the pattern, and the sensor unit 150 detects the sensor having an obstacle located at a sensing distance of each sensor.

The sensor unit 150 includes a plurality of sensors to detect an obstacle. The sensor unit 150 detects an obstacle in front of the main body 10, that is, a driving direction, by using at least one of laser, ultrasonic waves, and infrared rays. In addition, the sensor unit 150 may further include a cliff detection sensor for detecting the presence of a cliff on the floor in the driving zone. When the transmitted signal is reflected and incident, the sensor unit 150 inputs information on the presence or absence of an obstacle or a distance to the obstacle to the controller 110 as an obstacle detection signal.

The image acquisition unit 170 captures continuous images when the mobile robot operates. In addition, the image acquisition unit 170 may capture an image at a predetermined cycle. The image acquisition unit 170 captures an image even in a driving or cleaning state in which an obstacle is not detected by the obstacle detecting unit 100.

For example, when the image acquisition unit 170 moves in the same driving direction after the first shooting, the size of the obstacles recorded on the image once captured is changed while the driving direction is not changed. Since the image is taken, the image is periodically taken in predetermined time units. In addition, the image acquisition unit 170 captures a new image when the driving direction is changed.

The image acquisition unit 170 may set a shooting cycle according to the moving speed of the mobile robot. In addition, the image acquisition unit 170 may set the shooting period in consideration of the sensing distance by the sensor unit and the moving speed of the mobile robot.

The image acquisition unit 170 stores the image photographed while the main body is traveling as the image data 182 in the data unit 180.

Meanwhile, the obstacle detecting unit 100 detects an obstacle located in the driving direction while driving and inputs a detection signal to the controller. The obstacle detecting unit 100 inputs the information on the position or movement of the detected obstacle to the controller 110. The pattern acquisition unit 140 inputs an image including a pattern irradiated by the pattern irradiation unit as a detection signal to the controller, and the sensor unit 150 inputs a detection signal for an obstacle detected by the sensor provided to the controller. .

The controller 110 controls the driving unit 250 to allow the mobile robot to travel within the designated driving area of the driving area H.

The control unit 110 controls the driving unit 250 and the cleaning unit 260 while driving, so as to absorb dust or foreign matter around the mobile robot to perform cleaning for the driving zone. Accordingly, the cleaning unit 260 operates the brush to make it easy to suck the dust or foreign matter around the mobile robot, and operates the suction device to suck the dust or foreign matter. The cleaning unit is controlled to inhale foreign matter and carry out cleaning while driving.

The controller 110 checks the charging capacity of the battery and determines when to return to the charging station. When the charging capacity reaches a predetermined value, the controller 110 stops the operation being performed and starts searching for the charging station to return to the charging station. The controller 110 may output a notification about the charge capacity of the battery and a notification about the return of the charging stand.

The controller 110 processes the data input by the manipulation of the manipulation unit 160 to set the operation mode of the mobile robot, outputs the operation state through the output unit 190, and detects the operation state, error state, or obstacle. Follow the warning sound, sound effect, voice guidance through the speaker.

In addition, the controller 110 recognizes the obstacle with respect to the obstacle detected by the image acquisition unit 170 or the obstacle detecting unit 100, and sets one of a plurality of corresponding motions in response to the obstacle.

The controller 110 determines an obstacle from the image photographed by the image acquisition unit 170, and if the obstacle is detected by the obstacle detecting unit 100 within a predetermined distance, sets the corresponding motion for the obstacle. Do it.

The controller 110 may determine the obstacle included in the image data by analyzing the image photographed through the image acquisition unit 170, that is, the image data, before the obstacle is detected by the obstacle detecting unit 100. The controller 110 may determine the obstacle through the data provided by itself, and also determine the type of the obstacle by transmitting image data to the server or the terminal through the communication unit.

After the image is captured, the controller 110 detects that the obstacle is located within a predetermined distance by the obstacle detecting unit 100, and then detects the detection signal by a sensor unit that sets a corresponding motion according to the type of the obstacle. When the obstacle is detected, the main body performs a specified operation according to the type of the obstacle recognized in advance.

The controller 110 may perform an operation corresponding to the obstacle when the obstacle is detected according to the sensing distance of the obstacle detecting unit, and even if the obstacle is detected by the obstacle detecting unit, the obstacle may be located within a specified distance. In this case, an operation corresponding to the obstacle may be performed.

For example, when the distance from which the obstacle is initially detected from the detection signal by the pattern acquisition unit is different from the distance detected by the ultrasonic sensor, the time point at which the detection signal for the obstacle is input is different. Accordingly, the controller 110 is a plurality of means for detecting the obstacle, or when the distance to detect the obstacle, such as a laser sensor is more than a certain distance, to the obstacle detected by the obstacle detection unit 100 Based on the distance, any one of a plurality of corresponding motions is performed when the obstacle is located at a specified distance, for example, 30 cm.

The controller 110 selects a corresponding motion that can be performed from among a plurality of corresponding motions according to the shape of the detection signal based on the detection signal for the obstacle, analyzes the image data, and recognizes the obstacle included in the image accordingly. Select the corresponding motion to control the operation of the main unit.

For example, when the detection signal is an image included in the pattern, the controller may determine it as a different detection signal according to the shape of the pattern, and may distinguish an obstacle according to the detection signal, that is, the shape of the pattern.

When the obstacle 110 is recognized through the image, the controller 110 may perform a predetermined cycle with respect to the captured image before the detection signal is input by the obstacle detection unit, and may also determine when the obstacle detection signal is input. It may be. The controller 110 may receive data on the obstacle by transmitting data to the server 90 or the terminal 80 as necessary.

For example, when the same detection signal is input to different obstacles, thresholds, fans, and tables, the controller 110 selects a plurality of corresponding motions that are operable to the detection signals, and sets the threshold, the fan, and the table through the image data. The motion is controlled by setting the corresponding motion for each.

The controller 110 includes an obstacle recognition unit 111, a map generation unit 112, and a driving control unit 113.

The map generator 112 generates a map for the cleaning area based on the obstacle information while driving the cleaning area during the initial operation or when the map for the cleaning area is not stored. In addition, the map generator 112 updates the previously generated map based on the obstacle information acquired while driving.

The map generator 112 generates a base map based on the information obtained by the obstacle recognition unit 111 while driving, and generates a cleaning map by dividing an area from the base map. In addition, the map generation unit 112 organizes the region with respect to the cleaning map and sets the attributes of the region to generate the user map and the guide map. The base map is a map in which the shape of the cleaning area obtained through driving is displayed as an outline, and the cleaning map is a map in which areas are divided into the base map. The basic map and the cleaning map include information on the moving area of the mobile robot and obstacle information. The user map is a map that has a visual effect by simplifying the area of the cleaning map and arranging the outlines. The guide map is a map in which the clean map and the user map overlap. Since the cleaning map is displayed on the guide map, a cleaning command may be input based on an area in which the mobile robot can actually travel.

After generating the base map, the map generation unit 112 divides the cleaning area into a plurality of areas, includes a connection path connecting the plurality of areas, and generates a map including information on obstacles in each area. The map generation unit 112 generates a map in which regions are divided by dividing the small regions to distinguish the regions on the map, and setting the representative regions, and setting the separated small regions as separate detailed regions and merging them into the representative regions. .

The map generation unit 112 processes the shape of the region for each divided region. The map generator 112 sets attributes for the divided regions and processes the shape of the regions according to the attributes for each region.

The obstacle recognition unit 111 determines an obstacle based on data input from the image acquisition unit 170 or the obstacle detection unit 100, and the map generator 112 generates a map for the driving zone and detects the obstacle. Ensure that information about is included on the map. In addition, the driving controller 113 controls the driving unit 250 to change the moving direction or the driving path in response to the obstacle information so as to pass through the obstacle or to avoid the obstacle.

The driving controller 113 controls the driving unit 250 to independently control the operation of the left wheel driving motor and the right wheel driving motor so that the main body 10 travels straight or rotated. The traveling control unit 113 controls the driving unit 250 and the cleaning unit 260 according to the cleaning command so that the main body 10 inhales foreign substances while driving the cleaning area to perform cleaning.

The obstacle recognition unit 111 determines an obstacle by analyzing data input from the obstacle detection unit 100. The obstacle recognition unit 111 calculates the direction of the obstacle or the distance to the obstacle according to a detection signal of the obstacle detection unit, for example, an ultrasonic or laser signal, and analyzes the acquired image including the pattern to analyze the pattern. The obstacles are determined by extracting and analyzing the shape of the pattern. When the obstacle recognition unit 111 uses an ultrasonic wave or an infrared signal, the obstacle recognition unit 111 determines an obstacle based on the distance between the obstacle or the position of the obstacle, since the shape of the ultrasonic wave is different from the time when the ultrasonic wave is received.

In addition, the obstacle recognition unit 111 stores the image data photographing the obstacle by the image acquisition unit 170 and stores it in the data unit 180. Since the image acquisition unit 170 photographs the obstacle in front of the plurality of times, a plurality of image data are also stored. In addition, when the image acquisition unit 170 continuously photographs the image in the driving direction, the obstacle recognition unit 111 stores the input video as image data or divides the video by frame unit as image data. The obstacle recognition unit 111 analyzes the video in a frame unit, and removes unnecessary frames, that is, frames in which an object is not shaken or in focus, or empty frames (frames in which obstacles are not photographed). Save as.

The obstacle recognition unit 111 analyzes the plurality of image data to determine whether the object to be photographed, that is, the obstacle can be recognized. At this time, the obstacle recognition unit 111 analyzes the image data to determine whether or not the image data is recognizable. For example, the obstacle recognition unit 111 separates and discards a shaken image, an image that is not in focus, and an image that cannot distinguish an obstacle due to a dark state.

The obstacle recognition unit 111 analyzes the image data and extracts the characteristics of the obstacle, determines the obstacle based on the shape (shape), size, and color of the obstacle, and determines the location thereof. The obstacle recognition unit 111 analyzes the obstacle from the plurality of images photographed in advance, and analyzes the image photographed before the preset time on the basis of the point in time at which the obstacle is determined to be located at the designated distance, and determines the obstacle.

Although it is described that an obstacle is determined through the image data after the obstacle is determined by the detection signal, the present invention is not limited thereto, and the obstacle is determined by the detection signal after the obstacle is determined through the image. , Can be performed at the same time.

Since only a part of the obstacle may be photographed in a state in which the obstacle is approached within a specified distance, the obstacle is determined using an image photographed before a preset time, that is, captured at a distance greater than the specified distance and the entire shape of the obstacle is photographed. The obstacle recognition unit 111 may determine the type of a specific obstacle, and in some cases, may determine only a shape and a size.

The obstacle recognition unit 111 may determine the type of the obstacle by extracting the feature of the obstacle based on the previously stored obstacle data, excluding the background of the image from the image data. The obstacle data 181 is updated by new obstacle data received from the server. The mobile robot 1 may store obstacle data on the detected obstacle and receive data on the type of obstacle from the server for other data.

The obstacle recognition unit 111 detects a feature such as a point, a line, a surface, or the like with respect to predetermined pixels of the image, and detects an obstacle based on the detected feature.

The obstacle recognition unit 111 extracts the outline of the obstacle and recognizes the obstacle based on the shape to determine the type of the obstacle. The obstacle recognition unit 111 may determine the type of obstacle based on the shape and size of the obstacle. In addition, the obstacle recognition unit 111 may determine the type of obstacle based on the shape and movement of the obstacle.

The obstacle recognition unit 111 classifies a person, an animal, and an object based on the obstacle information. The obstacle recognition unit 111 may classify the types of obstacles into general obstacles, dangerous obstacles, biological obstacles, and floor obstacles, and determine the type of detailed obstacles for each classification.

In addition, the obstacle recognition unit 111 transmits recognizable image data to the server 90 through the communication unit 280 to determine the type of obstacle. The communication unit 280 transmits at least one image data to the server 90.

When the image data is received from the mobile robot 1, the server 90 analyzes the image data, extracts the outline or shape of the photographed object, and compares the data with the previously stored obstacles to determine the type of the obstacle. . The server 90 first searches for obstacles of a similar shape or color, and extracts and compares features from the corresponding image data to determine the type of obstacle.

After determining the type of obstacle, the server 90 transmits data about the obstacle to the mobile robot 1.

The obstacle recognition unit 111 stores the data on the obstacle received from the server through the communication unit as the obstacle data in the data unit 180. When the obstacle recognition unit 111 determines the type of the obstacle by the server, the obstacle recognition unit 111 performs an operation corresponding thereto. The driving controller 113 controls the driving unit to avoid, approach, or pass the obstacle in response to the type of the obstacle, and optionally outputs a predetermined sound effect, warning sound, and voice guidance through the speaker.

As described above, the obstacle recognition unit 111 determines whether the image data can be recognized and transmits the image data to the server according to the stored obstacle data, thereby recognizing the type of the obstacle in response to the server.

In addition, the obstacle recognition unit may store the obstacle data for obstacle recognition of the selected obstacle among the plurality of obstacles, and may recognize the obstacle based on the obstacle recognition data even if the image data is not transmitted to the server.

Since the controller 110 has a limited storage capacity of the data unit 180, the controller 110 may store information on some selected obstacles as obstacle recognition data. For example, the controller 110 may store the obstacle recognition data in the data unit for an obstacle selected through the terminal 80 or an obstacle having a large number of detections based on the number of detections.

Accordingly, the obstacle recognition unit 111 may store an obstacle in the cleaning area and an obstacle that is repeatedly detected in the data unit to immediately perform an operation corresponding to the detected obstacle.

When the driving controller 113 recognizes the type of obstacle from the image data, the driving controller 113 controls the driving unit 250 in response to the type of the obstacle so that the main body 10 performs a predetermined operation.

If it is determined that the obstacle is located within a predetermined distance according to the detection signal of the obstacle detecting unit 100, the driving controller 113 is determined based on the image data with respect to the corresponding motion executable according to the type or shape of the detection signal. An operation is performed by setting any one of a plurality of corresponding motions according to the type and shape of the obstacle.

In response to the obstacle recognized by the obstacle recognition unit 111, the driving controller 113 determines whether the vehicle is capable of driving or whether the vehicle is able to enter the vehicle, and moves to approach the obstacle, passes the obstacle, or avoids the obstacle. The driving path 250 is controlled by setting a cleaning path.

For example, the driving control unit 113 in response to the obstacle, the main body 10 to stop, decelerate, accelerate, reverse driving, u-turn, driving direction, so as not to approach the obstacle more than a certain distance, a certain time You can wait. In addition, the driving control unit 113 may output the specified sound according to the obstacle through the speaker, and may be output along with the specified operation.

When the obstacle is located within the designated distance through the obstacle detecting unit 100, the driving controller 113 sets the avoidance, approach, approach distance, and stop, deceleration, acceleration, reverse driving, u-turn and driving direction according to the detection signal. A plurality of corresponding motions such as a change are set, and one corresponding motion is set according to an obstacle determined from previously photographed image data to control the driving unit.

That is, the driving control unit 113 sets a plurality of corresponding motions according to the input detection signal, and in response to the obstacle determined from the image data of the time before the detection signal is input, the one of the plurality of corresponding motions is set. The driving unit is controlled to perform the corresponding motion. When the type of obstacle is determined in detail through the image data, the driving controller 113 sets the corresponding motion according to the type of the obstacle, and even if it is impossible to determine the correct type, the driving control unit 113 performs the corresponding motion according to the shape or size of the obstacle. Can be set. For example, the specific type is unknown, but if there is a space of a predetermined size or more from the bottom surface, that is, a space having a height and width through which the mobile robot can pass, the main body may be set to pass through an obstacle.

7 and 8 are diagrams showing an example of an image photographed while driving and driving of the mobile robot of the present invention.

As shown in FIG. 7, the mobile robot 1 detects an obstacle located in front of the main body 10 while driving.

When the mobile robot 1 travels toward the window O4 in the travel area H as illustrated in FIG. 7A, an image as illustrated in FIG. 7B is captured. The captured image includes a plurality of obstacles O01 to O06 positioned in the driving direction.

The image acquisition unit 170 may photograph images continuously or repeatedly photograph the images according to a predetermined period.

When the mobile robot 1 advances a certain distance, as shown in FIG. 8, as the obstacle approaches the obstacle, the obstacle is photographed in an enlarged form.

As the main body of the mobile robot 1 moves, the area of the image increases as the obstacle located in the traveling direction of the mobile robot increases, and the area occupied by the image decreases as the obstacle located opposite to the moving direction of the mobile robot moves. .

As the mobile robot moves as shown in FIG. 8A, the second and third obstacles O02 and O03 are approached among the plurality of obstacles O01 to O06 located in the area. Accordingly, the second and third obstacles O02 and O03 are photographed in the image photographed through the image acquisition unit 170 as shown in FIG.

The obstacle recognition unit 111 may store the image photographed at the position as shown in FIG. 7, and detect and recognize the plurality of obstacles O01 to O06 through the stored image data. As shown in FIG. 8, when the mobile robot 1 approaches the second and third obstacles O02 and O03, a detection signal is input by the obstacle detection unit 100, and the obstacle recognition unit 111 is an obstacle. It is determined that the position within the specified distance to determine the obstacle based on the pre-recorded image data. The obstacle recognition unit 111 may determine the obstacle before inputting the detection signal, and may determine the obstacle by transmitting image data to the server or the terminal.

When the driving direction of the mobile robot 1 is changed, when the obstacle is judged according to the detection signal, since the obstacle in the driving direction is not likely to be captured in the image data photographed before the driving direction is changed, the driving direction is changed. Obstacles are determined based on the following images. Therefore, when the obstacle is located within a predetermined distance according to the detection signal of the obstacle detecting unit 100, the mobile robot 1 determines the obstacle based on the image data photographed in the same driving direction.

The mobile robot 1 determines the type of obstacle by recognizing the obstacle from the captured image data. The image acquisition unit 170 stores image data photographed while driving, and the controller analyzes the image data according to a predetermined period to determine an obstacle or that the obstacle is located at a predetermined distance and detected by the obstacle detecting unit 100. If it is possible to determine the obstacle using the pre-recorded image data.

The controller determines the information on the type, shape, size, etc. of the obstacle through the image data as described above with respect to the second and third obstacles O02 and O03, and controls the driving unit to perform a corresponding motion in response thereto.

For example, when the detection signal for the second obstacle is input to the second obstacle O002, which is a table, the controller 110 selects two corresponding motions of avoiding and entering after approaching and analyzes the image data to analyze the image data. It operates by determining one of the corresponding motions according to whether it is possible to enter.

Even if the table is the same type, the size is different, the size of the space under the table is different, the size of the mobile robot may also be different, so whether or not the entry of the mobile robot is different, the control unit of the obstacle detection unit 100 Based on the detection signal, a plurality of corresponding motions, for example, a corresponding motion that can be avoided or entered or passed after approaching, can be set. Decide to perform by designating the corresponding motion of the motion.

Depending on the height of the table and the width of the legs of the table to determine whether to enter under the table to determine the driving direction. In addition, with respect to the same second obstacle O2 and the table, the corresponding motion may be changed according to the presence or absence of the chair as the third obstacle 03.

Accordingly, the mobile robot sets a plurality of corresponding motions that can be processed when the detection signal is input to the obstacle, and sets one of the corresponding motions according to the information of the obstacle determined through the image. Depending on the size and shape of the, the size of the main body of the mobile robot can be performed a different corresponding motion.

9 is a view for explaining the detection signal and the obstacle recognition according to the obstacle of the mobile robot according to an embodiment of the present invention, Figure 10 is a kind of obstacles while driving the mobile robot according to an embodiment of the present invention Is a diagram referred to for explaining the operation according to.

As illustrated in (a) of FIG. 9, when detecting that an obstacle is located within a predetermined distance, the obstacle detecting unit 100 inputs a detection signal thereto.

When the pattern irradiated from the pattern irradiator is photographed on the pattern acquisition unit, the pattern P1 irradiated to the obstacle is displayed on the image.

As the pattern is positioned above the reference line ref1, the obstacle recognition unit 111 may determine that an obstacle having a predetermined height in the driving direction is located.

Accordingly, in response to the detection signal, the traveling controller sets a plurality of corresponding motions that can be performed on the obstacle located ahead. For example, response motion such as avoidance, approach, entry, and passage can be set.

In addition, the obstacle recognition unit 111 determines the obstacle located in the driving direction by analyzing the image data photographed when the detection signal is input, that is, photographed before the detection signal is input.

When the pattern is irradiated to the lower end of the air conditioner as shown in (b) of FIG. 9, when the pattern is irradiated to the bottom of the fan and the pedestal portion of the fan as shown in (c) of FIG. 9, the threshold as shown in (d) of FIG. When the pattern is irradiated, and when the pattern is irradiated to the small box as shown in (e) of FIG. 9, the pattern as shown in FIG. 9 (a) described above may be photographed.

As the same pattern is detected even though the obstacles are different from each other, the obstacle recognition unit 111 determines the obstacle based on the pre-recorded image.

As shown in FIG. 10, the obstacle image located in the driving direction may be determined by analyzing the photographed image before the detection signal is input.

As shown in (a) of FIG. 10, the mobile robot 1 stores the captured image while driving and detects that the obstacle is located within a predetermined distance by the obstacle detecting unit 100. The 111 analyzes the photographed image to recognize obstacles photographed in the image data to determine information on obstacles such as type, size, shape (shape), and the like.

The obstacle recognition unit 111 determines the obstacle by analyzing the image data photographed before the detection signal is input, that is, when the obstacle is located farther than the specified distance.

For example, when the obstacle recognition unit 111 receives a detection signal at the second distance D02, only a part of the obstacle is photographed at the second distance as illustrated in FIG. 10C, so that the detection signal for the obstacle is input. Before the detection, the obstacle is determined by analyzing the image of FIG. 10 (b) taken at the first distance D01 based on a time point at which the detection signal is input. The obstacle recognition unit 1111 photographs based on the image photographed in the same driving direction.

When the driving echo of the main body is changed, the obstacle recognition unit 111 determines the obstacle from the image photographed after the driving direction is changed.

The driving controller determines that the obstacle is an electric fan as shown in FIG. 9C among the corresponding motions according to the detection signal, and performs the corresponding motion. For example, in the case of an electric fan, the mobile robot may be isolated from the lower part of the electric fan and the base of the electric fan, so that the moving part may be set by avoiding 10 cm or more of the electric fan. To control.

In addition, if it is determined through the image as a box as shown in FIG. If the box is light, the position can be changed by the mobile robot, so the corresponding motion can be set to approach and collide.

The controller 110 stores the information on the detected obstacle as obstacle data.

In the obstacle data, data about an obstacle that is frequently detected according to the number of times that the obstacle is detected is stored.

The driving controller 113 may further determine whether the obstacle is a dangerous obstacle according to the type of obstacle, and control the driving unit to perform an operation corresponding thereto. When the operation is designated according to the type of obstacle, the driving controller 113 may perform the designated operation. When the operation is not specified, the driving controller 113 may vary the degree of access to the obstacle according to whether the dangerous obstacle is recognized. In addition, the driving control unit 113 may output a predetermined sound effect or a warning sound according to the type of obstacle in the case of a dangerous obstacle, and output a voice guidance. The driving control unit 113 may set the object that may be damaged such as a flower pot or a vase, a pet, a chair leg that may be isolated, and the like as a dangerous obstacle.

In addition, the obstacle recognition unit 111 may perform a different corresponding motion according to the size or shape of the obstacle even when the same detection signal is input to the same obstacle.

For example, when a detection signal determined to be a leg of a chair is input, even when it is determined that the chair is an image, an operation of avoiding, approaching or entering may be performed according to the distance between the height of the chair and the leg of the chair. have. The obstacle recognition unit 111 analyzes the image after inputting the detection signal and calculates the distance between the chair legs and the height of the chair to determine whether to enter. As a result, the driving control unit performs one of the corresponding motions of avoiding, approaching and entering.

FIG. 11 is a diagram referred to for describing a signal flow between a mobile robot and a device for obstacle recognition according to an embodiment of the present invention.

As shown in Fig. 11, the mobile robot 1 performs cleaning while traveling in the travel area.

The mobile robot 1 captures an image while driving and stores the image as image data. When determining the obstacle through the image, the controller 110 analyzes the image based on the stored obstacle information to determine the obstacle (S1).

In addition, the mobile robot 1 may request to check the obstacle by transmitting the image data to the server 90 or the terminal 80 (S2).

The server 90 analyzes the image data to extract features of the obstacle, and determines the type of the obstacle based on the shape. The server 90 stores data on the accumulated obstacles in a database and utilizes the obstacles in determining the obstacles.

The terminal 80 may transmit data to the server when the mobile robot 1 cannot connect to the server. In addition, the terminal 80 may determine the type of the obstacle based on the received image data, and may determine the type of the obstacle through a user input.

In response to a request of the mobile robot, the server 90 or the terminal 80 transmits data on the type of obstacle to the mobile robot 1 (S3). In addition, the server 90 or the terminal 80 may transmit data on the avoidance operation corresponding to the type of obstacle to the mobile robot 1. The mobile robot 1 performs the avoiding operation according to the received data.

12 is a flowchart referred to for explaining a process of performing an obstacle detection and a corresponding operation of a mobile robot according to an embodiment of the present invention.

As shown in FIG. 12, the mobile robot 1 captures a plurality of images through the image acquisition unit 170 while driving. The controller 110 allows the image to be taken while driving even though the obstacle is not detected through the obstacle detecting unit.

Images may be taken as still images at predetermined time intervals, and may also be captured as moving images through continuous shooting. The image acquisition unit 170 stores the captured image as a plurality of image data (I01 to I03) in the data unit (S11).

The image processing unit provided in the image acquisition unit 170 filters the photographed image so that the image data is stored at predetermined time intervals, and when the video is captured, analyzes the frame unit and removes unnecessary frames, and then stores the image data.

The controller 110 stores the image data, and when an obstacle is detected by the obstacle detecting unit (T1), when the obstacle is located within a predetermined distance, the controller 110 removes the background by analyzing the photographed image data (S12) and extracts a feature. (S13). The obstacle recognition unit 111 determines the shape (shape), size, and type of the obstacle (S14).

The controller 110 controls the driving unit to set the corresponding motion in response to the obstacle determined by the detection signal and the image and operate accordingly.

In some cases, before the detection signal is input by the obstacle detection unit 100, the control unit 110 analyzes, filters, and extracts image data in a predetermined unit (S12 and S13) to determine the shape, size, or the like of the obstacle. The type can be determined (S14). When the detection signal is input by the obstacle detection unit 100 while the determination of the obstacle is completed (T2), the controller sets a corresponding motion for the obstacle based on the detection signal and the determination of the obstacle through the image to perform a specified operation. (S17).

Meanwhile, the controller 1110 may transmit a plurality of image data to the server (S15), and request an obstacle determination from the server. When data on the obstacle is received from the server, the obstacle information (S16) is stored, and the obstacle is determined accordingly (S14).

The controller 110 sets a corresponding motion based on the detection signal of the obstacle detecting unit 100 and the obstacle determination through the image (S17).

FIG. 13 is a flowchart referred to for describing obstacle recognition and a control method thereof according to an exemplary embodiment of the present invention. FIG.

As shown in FIG. 13, the mobile robot 1 travels in a traveling area within the traveling area H and performs cleaning on a designated area (S310).

When a moving or cleaning command is input, the mobile robot 1 captures an image through the image acquisition unit 170. The image acquisition unit 170 may continuously capture an image (video) or may capture an image at predetermined time intervals (S320). The acquired acquisition image is stored as image data.

The image acquisition unit 170 may set a shooting cycle according to the moving speed of the mobile robot, and may also set the distance based on the obstacle detectable distance of the obstacle detecting unit.

The image acquisition unit 170 determines whether the plurality of image data is recognizable or filtered, and selects and stores the image capable of image analysis as image data.

For example, the image acquisition unit 170 when the obstacle is not normally captured by the movement of the obstacle or the movement of the main body 10 during shooting, for example, when the image of the obstacle to be photographed shakes the obstacle cannot be recognized. In this case, the captured image is selected and stored as the image data, except when the recognition is impossible due to lack of focus.

The image acquisition unit 170 photographs an image while the mobile robot 1 is operating, and the obstacle detecting unit 100 detects an obstacle located in a driving direction by irradiating a pattern or by using an ultrasonic wave, an infrared ray, or a laser. Input the signal (S300).

The obstacle recognition unit 111 may determine the size and location of the obstacle through pattern analysis using the presence or absence of the obstacle located in the driving direction or the 3D sensor based on the detection signal (S340).

When the controller 110 determines an obstacle through a pattern obtained by examining a pattern, the controller 110 classifies and classifies obstacles to which the same sensing signal is input, and sets a plurality of corresponding motions that can be performed according to each sensing signal. (S350).

Since the controller 110 has a difference in the detectable distance according to the type of sensor of the obstacle detecting unit, the controller 110 determines whether the obstacle is located at the designated distance based on the detection signal (S360).

The controller 110 determines a plurality of corresponding motions that can be performed with respect to the obstacle located in the driving direction according to the shape of the detection signal, and when the obstacle is located within the designated distance, analyzes the image data of the previous photographed time point (S370). ), To determine the obstacle (S380).

The obstacle recognition unit 111 analyzes the image data and filters the image based on whether the obstacle can be recognized, removes the background from the filtered image data through image processing, and extracts the outline or feature point of the obstacle to form the shape of the obstacle. And extract features.

The obstacle recognition unit 111 may determine whether the image data is identified based on the brightness, the sharpness, and the moving speed of the main body 10. The obstacle recognition unit 111 determines the brightness of the image data based on the brightness values of the plurality of pixel values of the image data to distinguish between overexposure, underexposure, and normal exposure. In addition, the obstacle recognition unit 111 may determine that there is a shake in the captured image when the moving speed of the main body is greater than or equal to the set speed, and determine whether the image data can be identified by determining the sharpness of the image data.

The obstacle recognition unit 111 extracts the outline and features of the extracted obstacle, analyzes the shape, and determines the type and size of the obstacle based on the obstacle data.

On the other hand, when the obstacle recognition unit 111 does not have a separate obstacle data, or can not determine the type of the obstacle from the obstacle data may transmit the image data to the server 90 or the terminal may request the obstacle confirmation. Obstacle determination through the server may be immediately performed when the image is taken. The type of obstacle is judged by the self judgment or the response from the server.

The server 90 stores a plurality of image data with respect to one obstacle, extracts a feature of a specific obstacle and stores it in a database. The server 90 determines the type of obstacle by analyzing the image data received from the mobile robot 1 and comparing it with previously stored data. The server 90 transmits the type of obstacle and the obstacle data related thereto to the mobile robot 1.

The obstacle recognition unit 111 sets any one of a plurality of corresponding motions based on the data of the obstacle determined according to the shape and the feature of the obstacle (S390).

The driving controller controls the driving unit to perform an operation according to the corresponding motion (S400).

In the case of a dangerous obstacle, the driving control unit 113 is set to avoid the obstacle without approaching a certain distance or more and to travel. In addition, the driving controller 113 is set to perform the specified operation when the specified operation exists according to the type of obstacle.

The driving control unit 113 may be set to avoid after approaching an obstacle, or to immediately avoid when reaching a specified distance, and may set an access distance when approaching, and may be set to enter or pass through an obstacle. In addition, the mobile robot may output a predetermined warning sound.

Therefore, even if the same sensing signal is input, the mobile robot 1 may perform different operations according to the obstacles through image analysis, and even if the same sensing signal and the type of obstacles perform the corresponding motion according to the size or shape of the obstacle. Can be performed differently.

According to the present invention, when a predetermined distance is approached to an obstacle, the set operation may be performed immediately, and the obstacle may be easily determined by using a pre-recorded image, thereby more effectively avoiding the obstacle.

The above description is merely illustrative of the technical idea of the present invention, and those skilled in the art to which the present invention pertains may make various modifications and changes without departing from the essential characteristics of the present invention.

1: mobile robot 10: main body
100: obstacle detection unit 110: control unit
111: obstacle recognition unit 113: driving control unit
120, 130: pattern irradiation unit 140: pattern acquisition unit
150: sensor unit 170: image acquisition unit
180: data part
250: running part 260: cleaning part

Claims (22)

A movable body;
An image acquisition unit for photographing an image of a driving direction;
An obstacle detecting unit detecting an obstacle located in a driving direction;
Stores a plurality of image data photographed through the image acquisition unit, in accordance with a predetermined type of detection signal input from the obstacle detection unit, the obstacle within a predetermined distance based on the distance to the obstacle calculated through the detection signal Determine whether or not to be located, and start the corresponding motion at a predetermined time point at which it is determined that the obstacle is located within a predetermined distance, and according to the obstacle determined based on the image data acquired before the predetermined time point, A control unit for selecting one of the motions to determine the corresponding motion;
And the controller determines the obstacle based on image data photographed in the same travel direction as the travel direction of the main body at the predetermined time point. The method of claim 1,
The control unit is a mobile robot, characterized in that for setting a plurality of corresponding motion to perform according to the detection signal. delete The method of claim 2,
And the control unit controls the corresponding input signal to perform any one of the corresponding motions among the plurality of corresponding motions based on the image data. The method of claim 1,
The controller may determine the obstacle by analyzing the image data photographed before a predetermined time on the basis of the point of time when the obstacle is determined to be located at a predetermined distance by the obstacle detecting unit. delete The method of claim 1,
The controller analyzes the image data to determine the shape, size, or type of the obstacle. The method of claim 1,
The controller may determine the obstacle by analyzing the image data before the detection signal is input. The method of claim 1,
The controller may determine the obstacle by analyzing the image data when the detection signal is input. The method of claim 1,
The control unit is a mobile robot, characterized in that for transmitting the image data to the terminal or server to receive information about the obstacle included in the image data. The method of claim 1,
The image acquisition unit is a mobile robot, characterized in that for continuously shooting in the driving direction or to take an image according to a specified time to store the image data. The method of claim 1,
And the controller controls to perform the corresponding motion in which at least one of stop, deceleration, acceleration, reverse driving, standby, avoidance, short-range access prohibition, and voice guidance is performed. The method of claim 1,
The obstacle detecting unit includes at least one of an ultrasonic sensor, a laser sensor, an infrared sensor, and a 3D sensor. The method of claim 1,
The controller transmits the image data to a terminal or a server to request an obstacle check, and determines the type of the obstacle in response to the response data received from the server or the terminal. While driving, taking an image of a driving direction by an image acquisition unit and storing image data;
Inputting a detection signal for an obstacle through the obstacle detecting unit;
Determining whether the obstacle is located within a predetermined distance from the main body based on a distance from the obstacle calculated through the detection signal;
Determining, at a predetermined time point at which the obstacle is determined to be located within a predetermined distance, the obstacle is determined based on the image data photographed in the same traveling direction as the traveling direction of the main body at the predetermined time point among the image data previously acquired; ;;
Determining one of a plurality of corresponding motions as a corresponding motion according to the obstacle;
Starting the corresponding motion for the obstacle at the predetermined time point; And
Operating in accordance with the corresponding motion to avoid the obstacle or to pass through the control method of the mobile robot. delete The method of claim 15,
The setting of the corresponding motion may further include setting a plurality of corresponding motions according to the type of the detection signal input from the obstacle detecting unit, and selecting one of the plurality of corresponding motions based on the image data. Control method of the mobile robot. The method of claim 15,
And analyzing the image data to determine the obstacle after the image is photographed and before the detection signal for the obstacle is input from the obstacle detecting unit. The method of claim 15,
And detecting the obstacle by analyzing the image data when a detection signal for the obstacle is input from the obstacle detecting unit. delete The method of claim 18 or 19,
And analyzing the image data to determine the shape, size, or type of the obstacle. The method of claim 15,
And transmitting information about the obstacle included in the image data by transmitting the image data to a terminal or a server.

Images