JP7417944B2 - Autonomous vacuum cleaner, autonomous vacuum cleaner control method, and program - Google Patents

Description

The present invention relates to an autonomous vacuum cleaner that autonomously moves and cleans a predetermined space, a method for controlling the autonomous vacuum cleaner, and a program.

2. Description of the Related Art Conventionally, there are autonomous vacuum cleaners that autonomously run and clean a predetermined space such as inside a house.

For example, in Patent Document 1, an autonomous vacuum cleaner acquires an image by capturing the surroundings, analyzes the acquired image to recognize objects located in the surroundings, and based on the recognized object. A cleaning robot that moves and cleans is disclosed. For example, in Patent Document 2, an autonomous vacuum cleaner acquires an image by capturing the surroundings, analyzes the image to estimate the scene in which the device is located, and uses the estimated scene. A mobile device is disclosed that operates based on the invention.

Japanese Patent Application Publication No. 2019-121364 International Publication No. 2018/087971

The cleaning robot disclosed in Patent Document 1 mentioned above recognizes objects included in an image by analyzing the image. In object recognition, for example, an object is extracted from feature points included in an image, and it is determined whether the extracted object is a predetermined recognition target. If it is determined that the extracted object is a predetermined recognition target, the object is recognized as a recognition target.

Here, when the number of predetermined recognition targets is large, the accuracy of determining whether the object is a recognition target, that is, the recognition rate decreases. On the other hand, if the number of recognition targets is randomly reduced, even if an object is included in the image obtained from a camera, etc., the autonomous vacuum cleaner will avoid the object because it cannot be recognized as a recognition target. There is a possibility that problems such as running over an object and being unable to drive may occur.

The present invention provides an autonomous vacuum cleaner and the like that can improve the recognition rate of recognition targets.

An autonomous vacuum cleaner according to one aspect of the present invention is an autonomous vacuum cleaner that autonomously moves and cleans a predetermined space, and acquires a self-position of the autonomous vacuum cleaner in the predetermined space. A self-position acquisition unit and recognition information each indicating an image processing method for recognizing a recognition target included in an image, each of which recognizes one recognition information from a plurality of recognition information that differ in one or more recognition targets from each other. The determination unit that determines based on the self-position and the camera that photographs the predetermined space perform image analysis on the captured image using the one recognition information determined by the determination unit, thereby determining the captured image. A recognition unit that recognizes a recognition target included in the recognition unit, and a control unit that causes the autonomous vacuum cleaner to travel and clean based on the recognition result of the recognition unit.

Further, a method for controlling an autonomous vacuum cleaner according to one aspect of the present invention is a method for controlling an autonomous vacuum cleaner that autonomously moves and cleans a predetermined space, the autonomous vacuum cleaner cleaning in the predetermined space. A self-position acquisition step for acquiring the self-position of the vacuum cleaner, and a plurality of pieces of recognition information each indicating an image processing method for recognizing a recognition target included in an image, the plurality of pieces of recognition information each having one or more recognition targets different from each other. a determining step of determining one recognition information from recognition information based on the self-position; and a photographed image obtained from a camera photographing the predetermined space using the one recognition information determined in the determining step. a recognition step of recognizing a recognition target included in the photographed image by analyzing the image; and a control step of causing the autonomous vacuum cleaner to run and clean based on the recognition result in the recognition step. include.

Further, a program according to one aspect of the present invention is a program for causing a computer to execute the method for controlling an autonomous vacuum cleaner described above.

Note that the present invention may be realized as a non-temporary recording medium such as a computer-readable CD-ROM in which the above program is recorded. Further, the present invention may be realized as information, data, or a signal indicating the program. These programs, information, data, and signals may be distributed via a communication network such as the Internet.

According to an autonomous vacuum cleaner or the like according to one aspect of the present invention, the recognition rate for recognizing a recognition target can be improved.

FIG. 1 is a perspective view showing the appearance of an autonomous vacuum cleaner according to an embodiment. FIG. 2 is a bottom view showing the appearance of the autonomous vacuum cleaner according to the embodiment. FIG. 3 is a block diagram showing the functional configuration of the autonomous vacuum cleaner according to the embodiment. FIG. 4 is a diagram showing an example of area information. FIG. 5 is a diagram showing an example of correspondence information. FIG. 6 is a diagram illustrating an example of driving pattern information. FIG. 7 is a flowchart showing an overview of processing executed by the autonomous vacuum cleaner according to the embodiment. FIG. 8 is a flowchart showing details of area information registration processing. FIG. 9A is a diagram illustrating a first example of user operations in region information registration processing. FIG. 9B is a diagram illustrating a second example of user operations in the area information registration process. FIG. 9C is a diagram illustrating a third example of user operations in the area information registration process. FIG. 9D is a diagram illustrating a fourth example of user operations in the area information registration process. FIG. 9E is a diagram illustrating a fifth example of user operations in region information registration processing. FIG. 9F is a diagram illustrating a sixth example of user operations in the region information registration process. FIG. 10 is a flowchart showing details of the model information registration process. FIG. 11A is a diagram illustrating a first example of user operations in model information registration processing. FIG. 11B is a diagram illustrating a second example of user operations in model information registration processing. FIG. 11C is a diagram illustrating a third example of user operations in model information registration processing. FIG. 11D is a diagram illustrating a fourth example of user operations in model information registration processing. FIG. 12 is a flowchart showing details of the cleaning process. FIG. 13A is a diagram illustrating a first example of the cleaning operation of the autonomous vacuum cleaner according to the embodiment. FIG. 13B is a diagram illustrating a second example of the cleaning operation of the autonomous vacuum cleaner according to the embodiment. FIG. 13C is a diagram illustrating a third example of the cleaning operation of the autonomous vacuum cleaner according to the embodiment. FIG. 13D is a diagram illustrating a fourth example of the cleaning operation of the autonomous vacuum cleaner according to the embodiment. FIG. 14 is a diagram illustrating an example of an image displayed on a communication terminal while the autonomous vacuum cleaner according to the embodiment is cleaning.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Below, embodiments of an autonomous vacuum cleaner and the like according to the present invention will be described in detail with reference to the drawings. It should be noted that the embodiments described below each represent a preferred specific example of the present invention. Therefore, the numerical values, shapes, materials, components, arrangement and connection forms of the components, steps, order of steps, etc. shown in the following embodiments are merely examples, and do not limit the present invention.

The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present invention, and are not intended to limit the subject matter recited in the claims.

Furthermore, each figure is a schematic diagram and is not necessarily strictly illustrated. Furthermore, in each figure, substantially the same configurations are denoted by the same reference numerals, and overlapping explanations may be omitted or simplified.

Furthermore, in the following embodiments, expressions using "approximately", such as approximately triangular, are used. For example, "substantially triangular" means not only a complete triangle but also a substantially triangular shape, that is, a triangle with rounded corners, for example. The same applies to other expressions using "abbreviation".

In addition, in the following embodiments, when an autonomous vacuum cleaner that runs and cleans a floor in a predetermined space is viewed from vertically above, it is referred to as a top view, and when viewed from vertically below, it is referred to as a bottom view. May be stated.

(Embodiment)
[composition]
First, the configuration of an autonomous vacuum cleaner 100 according to an embodiment will be described with reference to FIGS. 1 to 6.

FIG. 1 is an external view showing the external appearance of an autonomous vacuum cleaner 100 according to an embodiment. FIG. 2 is a bottom view showing the appearance of the autonomous vacuum cleaner 100 according to the embodiment. FIG. 3 is a block diagram showing the functional configuration of the autonomous vacuum cleaner 100 according to the embodiment.

The autonomous vacuum cleaner 100 is an autonomous vacuum cleaner that autonomously moves and cleans a predetermined space such as a living room.

First, the autonomous vacuum cleaner 100 runs around while taking images of a predetermined space (more specifically, within the predetermined space) using the laser range finder 40, the camera 70, etc., thereby creating a map of the predetermined space. Map information 210 (data) indicating the map is generated.

Next, the autonomous vacuum cleaner 100 calculates a travel route to be traveled when cleaning a predetermined space based on the generated map information 210.

Next, the autonomous vacuum cleaner 100 cleans a predetermined space by traveling along the calculated travel route.

The autonomous vacuum cleaner 100 autonomously determines whether to avoid objects (obstacles) existing in a predetermined space by observing the state of the predetermined space using a camera 70 and a sensor such as a cliff sensor. If the object is present, the vehicle departs from the calculated travel route, that is, calculates a new travel route that avoids the object, and cleans the vehicle while avoiding the object.

The autonomous vacuum cleaner 100 uses, for example, SLAM (Simultaneous Localization and Mapping) to generate map information 210 of a predetermined space to be cleaned, and to locate the autonomous vacuum cleaner 100 in a map shown in the generated map information 210. and perform position estimation.

As shown in FIGS. 1 and 2, the autonomous vacuum cleaner 100 includes, for example, a main body 10, two wheels 20, two side brushes 30, a laser range finder 40, a main brush 50, It includes a camera 70 and a depth sensor 90.

The main body 10 is a housing in which each component of the autonomous vacuum cleaner 100 is arranged (more specifically, housed). In this embodiment, the main body portion 10 has a substantially triangular shape when viewed from above. Note that the shape of the main body section 10 when viewed from above is not particularly limited. The top view shape of the main body portion 10 may be, for example, a substantially rectangular shape or a substantially circular shape. As shown in FIG. 2, the main body 10 has a suction port 11 on the bottom surface.

The two wheels 20 are wheels for driving the autonomous vacuum cleaner 100.

The side brush 30 is a brush that is provided on the lower surface of the main body 10 and is used to clean the floor of a predetermined space (hereinafter also simply referred to as the floor). In this embodiment, the autonomous vacuum cleaner 100 includes two side brushes 30. The number of side brushes 30 provided in the autonomous vacuum cleaner 100 may be one, or may be three or more, and is not particularly limited.

The laser range finder 40 is a sensor for measuring the distance between the autonomous vacuum cleaner 100 and objects, walls, etc. in a predetermined space. The laser range finder 40 is provided, for example, at the top of the main body 10. The laser range finder 40 is, for example, a so-called LIDAR (Light Detection and Ranging).

The main brush 50 is arranged at the suction port 11, which is an opening provided on the lower surface of the main body 10, and is a brush for sucking dust from the floor.

The camera 70 is an imaging device that generates an image by photographing a predetermined space. Specifically, the camera 70 is arranged in the main body 10 and captures an image by photographing the surroundings of the autonomous vacuum cleaner 100 (more specifically, the direction in which the autonomous vacuum cleaner 100 is moving). generate. The camera 70 outputs the generated image to the recognition unit 115.

Note that although the camera 70 is arranged in the main body 10 in this embodiment, it is sufficient that it can photograph a predetermined space, and the camera 70 does not need to be arranged in the main body 10.

The depth sensor 90 generates an image (depth image) by photographing a predetermined space, and autonomously moves with an object that is a subject included in the generated depth image (more specifically, each pixel that constitutes the subject). This is a sensor that detects the distance to the vacuum cleaner 100. Depth sensor 90 is, for example, an infrared camera.

As shown in FIG. 3, the autonomous vacuum cleaner 100 includes the various sensors described above, such as the laser range finder 40, the camera 70, and the depth sensor 90, a self-position acquisition unit 112, and a correspondence information acquisition unit 113. , determination section 114, recognition section 115, shape detection section 116, coordinate detection section 117, plan generation section 120, control section 130, storage section 140, cleaning section 150, drive section 160, suction It includes a section 170, a notification section 180, a reception section 190, and a registration section 191.

The self-position acquisition unit 112 is a processing unit that acquires the self-position of the autonomous vacuum cleaner 100 in a predetermined space. More specifically, the self-location acquisition unit 112 acquires self-location information that is information indicating the coordinates of the self-position of the autonomous vacuum cleaner 100 on a map showing a predetermined space. In the present embodiment, the self-position acquisition unit 112 acquires the self-position by estimating the self-position in a predetermined space based on the detection result of the laser range finder 40. For example, the self-position acquisition unit 112 calculates distances to objects, including obstacles, walls, etc., located around the autonomous vacuum cleaner 100, which are input from the laser range finder 40, and which are stored in the storage unit 140. Based on a map (map information 210) indicating a predetermined space, the coordinates of the autonomous vacuum cleaner 100 on the map are calculated. The self-position acquisition unit 112 outputs the self-position to the determination unit 114 and the coordinate detection unit 117 in association with the time when the self-position was acquired.

Note that the self-location acquisition unit 112 may acquire self-location information from an external device such as a computer (not shown) by communicating with the external device. In this case, the self-location acquisition unit 112 may have a communication interface for communicating with an external device, and may acquire self-location information via the communication interface.

Further, in the present embodiment, map information 210 is stored in storage unit 140 in advance. If the map information 210 is not stored in advance in the storage unit 140, for example, the control unit 130 (described later) controls the laser range finder 40, camera 70, drive unit 160, etc. using the SLAM described above to obtain a predetermined information. By driving the vehicle while capturing an image of the space, map information 210 indicating a map of a predetermined space may be generated and stored in the storage unit 140.

The correspondence information acquisition unit 113 is a processing unit that acquires the correspondence information 220.

The correspondence information 220 is information in which recognition information in which one or more recognition targets are different is linked to a plurality of mutually different regions, each of which is a part of a predetermined space. The correspondence information 220 includes, for example, area information 221 indicating a plurality of different areas, each of which is a part of a predetermined space, and an image processing method for recognizing a recognition target included in the image. This information includes model information 222 that includes a plurality of pieces of recognition information (recognition model) in which one or more recognition targets are different from each other. Each of the plurality of regions is associated with one of the plurality of pieces of recognition information.

The recognition information is a processing method (more specifically, a program) for recognizing (identifying) the type of object included in the image generated by the camera 70. For example, different recognition information is a program that differs in one or more types (types) of objects to be recognized. A specific example of the correspondence information 220 will be described later.

In this embodiment, the correspondence information 220 is stored in the storage unit 140, and the correspondence information 220 is acquired from the storage unit 140.

Note that the correspondence information acquisition unit 113 may acquire the correspondence information 220 from an external device (for example, the communication terminal 300 shown in FIG. 11A) such as a computer (not shown) by communicating with the external device. In this case, the correspondence information acquisition unit 113 may have a communication interface for communicating with an external device, and may acquire the correspondence information 220 via the communication interface.

The correspondence information acquisition unit 113 determines an area in which the autonomous vacuum cleaner 100 is located in a predetermined space among the plurality of areas, based on the self-position acquired by the self-position acquisition unit 112 and the correspondence information 220. , and outputs information indicating the determined area (more specifically, a location ID (ID/Identification) to be described later) to the determining unit 114.

The determining unit 114 selects one recognition information from a plurality of pieces of recognition information, each of which indicates an image processing method for recognizing a recognition target included in an image, and which differs from each other in one or more recognition targets. This is a processing unit that makes decisions based on location. Specifically, the determining unit 114 selects a specific recognition target located around the autonomous vacuum cleaner 100 according to the area corresponding to the self-position of the autonomous vacuum cleaner 100 acquired by the self-position acquisition unit 112. Determine recognition information for recognizing. More specifically, the determining unit 114 selects specific places located around the autonomous vacuum cleaner 100 based on the self-location of the autonomous vacuum cleaner 100 and the location ID output from the correspondence information acquisition unit 113. Determine recognition information for recognizing only the recognition target.

For example, the determining unit 114 determines one or more recognition targets from among the plurality of recognition targets based on the self-position of the autonomous vacuum cleaner 100 and the correspondence information 220. In other words, the determination unit 114 determines one piece of recognition information from among the plurality of pieces of recognition information included in the model information 222, based on the self-position of the autonomous vacuum cleaner 100 and the correspondence information 220.

For example, when the determining unit 114 moves from one region to another among a plurality of regions, the determination unit 114 switches the recognition information corresponding to the one region to the recognition information corresponding to the other region. The determining unit 114 outputs information indicating the determined type of recognition information (for example, model ID) to the recognizing unit 115.

The recognition unit 115 analyzes the image obtained from the camera 70 photographing a predetermined space (photographed image) using one piece of recognition information determined by the determination unit 114, thereby identifying the recognition target included in the photographed image. This is a processing unit that recognizes. Specifically, the recognition unit 115 uses the recognition information determined by the determination unit 114, which is included in the photographed image obtained from the camera 70 photographing a predetermined space, to recognize only one or more recognition targets. Process. More specifically, the recognition unit 115 uses the recognition information determined by the determination unit 114 to recognize only one or more recognition targets, which is included in the captured image obtained from the camera 70 that photographs a predetermined space. By performing image processing, it is determined whether the captured image includes any of the one or more recognition targets.

For example, the recognition unit 115 uses the captured image obtained from the camera 70 and the recognition information determined by the determination unit to determine whether the recognition target is included in the image by comparing it with information such as the image of the recognition target. Recognize the recognition target by making a judgment. More specifically, the recognition unit 115 indicates the time when the captured image was generated from the camera 70, numerical values indicating R (Red), G (Green), and B (Blue) of the captured image, and the position in the captured image. The identification number (pixel number), etc., that is, the RGB value of each pixel, is acquired from the camera 70, and the image is analyzed based on the acquired information to identify the type of object and recognize the recognition target. .

For example, when the recognition unit 115 uses recognition information that recognizes "plastic bottle" and "remote control", when the captured image obtained from the camera 70 includes "plastic bottle" or "remote control", the recognition unit 115 respectively However, objects other than "plastic bottle" and "remote control" cannot be recognized even if they are included in the image. In other words, the recognition unit 115 recognizes only the recognition target according to the recognition information determined by the determination unit 114, and cannot recognize objects other than the recognition target even if they are included in the image.

The recognition unit 115 outputs the recognized type of recognition target to the coordinate detection unit 117 in association with time information indicating the time when the photographed image including the recognition target was photographed. More specifically, the recognition unit 115 calculates a bounding box (a circumscribed rectangular frame surrounding the recognition target) of the recognition target included in the captured image, and calculates the position of the calculated bounding box (more specifically, the pixel position). ) is output to the coordinate detection unit 117.

The shape detection unit 116 acquires the depth image generated by the depth sensor 90 and information indicating the time when the depth image was taken, and analyzes the depth image to identify the object included in the depth image and the autonomous driving type. This is a processing unit that detects (calculates) the distance (relative distance, more specifically, relative position) to the vacuum cleaner 100, and the width, height, etc. of the object.

For example, the storage unit 140 stores in advance position information indicating the installation position of the depth sensor 90 in the main body 10, size information indicating the size of the main body 10, and the like. The shape detection section 116 calculates the distance between the main body section 10 and the object included in the depth image (more specifically, the closest distance, which is the closest distance) based on this information. The shape detection unit 116 outputs the calculated distance to the object, the width, height, etc. of the object to the coordinate detection unit 117 in association with the shape information of the object and the time when the depth image was captured.

The coordinate detection unit 117 detects the relative distance between the autonomous vacuum cleaner 100 and the object detected by the shape detection unit 116 (more specifically, the relative position), and the autonomous vacuum cleaner acquired by the self-position acquisition unit 112. This is a processing unit that calculates (detects) the position (coordinates) of an object on the map from the position on the 100 maps. Further, the coordinate detection unit 117 links the recognition target recognized by the recognition unit 115 and the object detected by the shape detection unit 116, for example, by time or the like. Thereby, the coordinate detection unit 117 identifies the position, shape, and type of the object detected by the shape detection unit 116. For example, the coordinate detection unit 117 causes the storage unit 140 to store information indicating the coordinates of the detected object.

The coordinate detection unit 117 also collects time information indicating the time at which the recognition unit 115 recognized the recognition target (that is, the time at which the shape detection unit 116 detected the object), position information indicating the position of the detected object, and information about the object. type information indicating the type of .

The plan generation unit 120 is a processing unit that generates plan information indicating how to clean (cleaning mode) such as the travel route of the autonomous vacuum cleaner 100 and the timing of driving the main brush 50 and the like. For example, a cleaning plan (plan information) indicating how the autonomous vacuum cleaner 100 will travel and clean a predetermined space is generated.

For example, the plan generation unit 120 generates a travel route based on the map information 210 so that the autonomous vacuum cleaner 100 travels throughout a predetermined area. For example, the plan generating unit 120 determines a traveling method, which is a method of controlling the drive unit 160, such as the rotation speed of the wheel motor 161 of the autonomous vacuum cleaner 100 and the direction of the wheels 20, depending on the position of the predetermined space. do. For example, the plan generating unit 120 may also determine a method of controlling the cleaning unit 150 of the autonomous vacuum cleaner 100 (for example, the number of rotations of the brush motor 151) and a method of controlling the suction unit 170 according to the position of the predetermined space. (For example, the suction force, more specifically, the number of rotations of the suction motor 171), etc., is generated.

For example, when the plan generation unit 120 travels around the recognition target recognized by the recognition unit 115, the plan generation unit 120 generates a travel route along which the autonomous vacuum cleaner 100 travels based on the travel pattern information 230. For example, the driving pattern information 230 includes driving route and cleaning method information according to the type of recognition target, and the plan generation unit 120 generates plan information based on the driving pattern information 230, for example. Details of the driving pattern information 230 will be described later.

In this way, the plan generation unit 120 creates a cleaning mode (i.e., a cleaning mode) indicating how to run and clean the recognition target recognized by the recognition unit 115, that is, a running route, a running method, and a cleaning method. method), and for causing the control unit 130 to control the autonomous vacuum cleaner 100, more specifically, the cleaning unit 150, the drive unit 160, and the suction unit 170, so as to achieve the determined cleaning mode. Generate plan information indicating control details.

Note that for objects detected by various sensors but not recognized as recognition targets, the plan generation unit 120 generates plan information so as to avoid the objects in a predetermined driving manner. may be generated. For example, if the recognition unit 115 cannot recognize the recognition target based on the image obtained from the camera 70 while the autonomous vacuum cleaner 100 is running, the plan generation unit 120 may detect the autonomous vacuum cleaner 100 from the depth sensor 90. When an object that obstructs driving is detected, planning information that causes the object to be avoided in a predetermined driving mode may be regenerated, or the object may be autonomously driven without regenerating the planning information. The vacuum cleaner 100 may be allowed to ride on the vehicle and continue to run.

The control unit 130 causes the autonomous vacuum cleaner 100 to run and clean based on the recognition result of the recognition unit 115 (more specifically, the type of the recognition target when the recognition unit 115 recognizes the recognition target). This is a processing unit that performs Specifically, the control unit 130 controls the autonomous vacuum cleaner 100 based on the plan information generated by the plan generation unit 120. More specifically, the control unit 130 controls the autonomous vacuum cleaner 100 by controlling the cleaning unit 150, the drive unit 160, and the suction unit 170 based on the plan information generated by the plan generation unit 120. Have the robot run and clean a designated space.

For example, when the recognition unit 115 recognizes a recognition target while the autonomous vacuum cleaner 100 is running and cleaning, the control unit 130 uses a recognition target based on the type of the recognition target and the travel pattern information 230. , the autonomous vacuum cleaner 100 is caused to run and clean. In the present embodiment, for example, when the recognition unit 115 recognizes a recognition target while the autonomous vacuum cleaner 100 is running and cleaning, the plan generation unit 120 determines the type of the recognition target and the travel Based on the pattern information 230, plan information including information such as the driving route is regenerated. The control unit 130 causes the autonomous vacuum cleaner 100 to travel and clean based on the plan information regenerated by the plan generation unit 120.

The storage unit 140 is a storage device that stores various information, control programs executed by each processing unit, and the like. The storage unit 140 stores, for example, map information 210, correspondence information 220, and driving pattern information 230. The correspondence information 220 includes area information 221 and model information 222.

FIG. 4 is a diagram showing an example of the area information 221.

The area information 221 is information indicating a plurality of different areas, each of which is a part of a predetermined space (that is, a part of the area of the map shown in the map information 210).

The area information 221 includes, for example, location ID information 221A, coordinate information 221B, and area ID information 221C.

The location ID information 221A is information indicating the name of the area specified by the coordinate information 221B. In this embodiment, place ID information 221A includes a place name and a numerical value that is an ID corresponding to the place name.

The coordinate information 221B is coordinates indicating an arbitrarily specified area in a predetermined space (that is, the area of the map shown in the map information 210). In this embodiment, the map information 210 is a two-dimensional map in which positions (coordinates) are specified using XY coordinates. In addition, in this embodiment, the coordinate information 221B includes four positions (X-axis minimum value (Xmin), X-axis maximum value (Xmax), Y-axis minimum value (Ymin), and Y-axis maximum value). value (Ymax)). The area surrounded by the four positions is one of the predetermined spaces divided into a plurality of areas.

The area ID information 221C is an ID (numerical value) uniquely determined for each area indicated by the coordinate information 221B.

In the example shown in FIG. 4, for example, "Living" has a place ID of 0001, is an area surrounded by an X coordinate of 0 to 10000, and a Y coordinate of 0 to 8000, and has an area ID of is 0001.

The area information 221 may be stored in the storage unit 140 in advance, or may be acquired by the registration unit 191 via the reception unit 190 and stored in the storage unit 140. For example, the registration unit 191, which will be described later, obtains, via the reception unit 190, a place name included in the place ID information 221A and coordinate information 221B linked to the place name. The registration unit 191 assigns a unique ID (numerical value) to each of the acquired place name and the area associated with the place name, and stores them in the storage unit 140.

FIG. 5 is a diagram showing an example of the correspondence information 220. In the example shown in FIG. 5, of the area information 221, place ID information 221A and area ID information 221C are shown, and illustration of the coordinate information 221B shown in FIG. 4 is omitted.

The correspondence information 220 includes location ID information 221A, area ID information 221C, and model information 222.

Each place ID included in the place ID information 221A is linked with any of the area IDs included in the area ID information 221C and with any of the plurality of pieces of recognition information included in the model information 222. That is, each of the plurality of regions indicated by the region information 221 is associated with one of the plurality of recognition information included in the model information 222. Thereby, for example, the determination unit 114 can specify the area (area ID) in which the self-position (coordinates) acquired by the self-location acquisition unit 112 is located, and determine the recognition information corresponding to the specified area.

The model information 222 is information including a plurality of pieces of recognition information. A unique model ID (numerical value) is assigned to each piece of recognition information.

For example, a model ID of 0001 is an ID of recognition information whose location ID corresponds to "living room: 0001" and whose recognition targets are "plastic bottle" and "remote control." For example, when the recognition targets of the recognition information determined by the determining unit 114 are "plastic bottle" and "remote control" (that is, when the self-location is "living room" and the model ID is determined to be 0001) ), it is determined whether the photographed image obtained from the camera 70 includes either a "plastic bottle" or a "remote control".

In this way, the recognition unit 115 recognizes objects included in the captured image obtained from the camera 70 while switching the recognition target (that is, recognition information) according to the self-position of the autonomous vacuum cleaner 100.

Note that each of the plural pieces of recognition information may be an image processing method (program) for recognizing one recognition target, or may be a program for recognizing two or more recognition targets. Furthermore, the recognition targets of the plural pieces of recognition information do not have to completely match each other, and some of the recognition targets may overlap.

FIG. 6 is a diagram showing an example of the driving pattern information 230.

The travel pattern information 230 is a database that indicates how the autonomous vacuum cleaner 100 travels toward the recognition target.

The driving pattern information 230 includes type name information 230A, distance information 230B, and pattern information 230C. The type name information 230A, the distance information 230B, and the pattern information 230C are stored in association with each other.

The type name information 230A is information indicating the type of object. FIG. 6 illustrates "clothing" and "remote control" as the type name information 230A.

The distance information 230B is information indicating the distance between the autonomous vacuum cleaner 100 and the object indicated by the type name information 230A.

The pattern information 230C is information indicating how the autonomous vacuum cleaner 100 is run by the control unit 130 controlling the drive unit 160.

For example, when the recognition unit 115 recognizes “clothing” from the captured image obtained from the camera 70, the plan generation unit 120 generates plan information corresponding to “clothing” included in the type name information 230A.

For example, when the distance between "clothing" and the autonomous vacuum cleaner 100 is 1000 mm or more, the plan generation unit 120 generates plan information (traveling route) in which the autonomous vacuum cleaner 100 travels at the first speed until it reaches 1000 mm. Alternatively, for example, if the distance is 500 mm or more and less than 1000 mm, the plan generation unit 120 generates plan information for approaching the vehicle by decelerating the vehicle to a second speed that is slower than the first speed until 500 mm. For example, the plan generation unit 120 generates plan information such as stopping and avoiding "clothing" when the distance is less than 500 mm.

Note that if the type and position of the object can be identified before the autonomous vacuum cleaner 100 runs and starts cleaning, the plan generation unit 120 may generate the above-mentioned plan information in advance. good. Alternatively, if the recognition unit 115 recognizes an object included in the captured image obtained from the camera 70 while the autonomous vacuum cleaner 100 is running and cleaning, the plan generation unit 120 may perform laser measurement. Information indicating the distance between the object and the autonomous vacuum cleaner 100 is acquired from the distance meter 40, and from the acquired information and the travel pattern information 230, how to run the autonomous vacuum cleaner 100 (that is, how to run the autonomous vacuum cleaner 100) is determined. If the already generated plan information is different from the determined driving mode, the planning information is regenerated so as to match the determined driving mode.

The storage unit 140 is realized by, for example, an HDD (Hard Disk Drive), a flash memory, or the like. Further, the storage unit 140 stores, for example, control programs executed by various processing units such as the control unit 130.

The cleaning unit 150 is a mechanism for cleaning the floor surface. The cleaning unit 150 includes, for example, a brush motor 151.

The brush motor 151 is a motor that is connected to brushes such as the main brush 50 and drives (rotates) the brushes such as the main brush 50.

The drive unit 160 is a mechanism for driving the autonomous vacuum cleaner 100. The drive unit 160 includes, for example, a wheel motor 161.

The wheel motor 161 is a motor that is connected to the wheel 20 and rotates the wheel 20.

The autonomous vacuum cleaner 100 can move freely, such as going straight, going backwards, turning left, turning right, etc., by independently controlling the rotation of the two wheels 20 connected to the wheel motor 161. Note that the autonomous vacuum cleaner 100 may further include wheels (auxiliary wheels) that are not rotated by the wheel motor 161.

The suction unit 170 is a mechanism for suctioning dust on the floor surface of a predetermined space by suctioning the floor surface. The suction unit 170 includes, for example, a suction motor 171.

The suction motor 171 is, for example, a motor that is connected to a fan and sucks up dust on the floor by rotating the fan.

The notification unit 180 is a device for notifying the user of information through video, audio, and the like.

For example, when the recognition unit 115 recognizes a recognition target, the notification unit 180 notifies the type of the recognized recognition target and the position of the recognized recognition target. Specifically, for example, when the recognition unit 115 recognizes a recognition target, the control unit 130 includes type information indicating the type of the recognition target and position information indicating the area in which the recognition target is located among the plurality of areas. , to the notification unit 180.

For example, if the notification unit 180 is an audio device that includes an amplifier, a speaker, etc. that generates sound, the control unit 130 controls the notification unit 180 to generate sound at the timing when the recognition unit 115 recognizes the recognition target. Then, the user is notified of type information indicating the type of the recognition target and position information indicating the area in which the recognition target is located among the plurality of areas. Alternatively, if the notification unit 180 is a communication interface for communicating with the communication terminal 300, the control unit 130 may send type information indicating the type of the recognition target and a plurality of areas at the timing when the recognition unit 115 recognizes the recognition target. The location information indicating the area where the recognition target is located is transmitted to the communication terminal 300 owned by the user (see, for example, FIG. 9A) via the notification unit 180. For example, the communication terminal 300 displays an image indicating the received information on a display unit such as a display included in the communication terminal 300. Thereby, the control unit 130 notifies the user of the type of the recognized recognition target and the position of the recognized recognition target using the notification unit 180.

Further, for example, the notification unit 180 notifies that the determination unit 114 has switched the recognition model at the timing when the recognition information used by the recognition unit 115 has been switched.

For example, if the notification unit 180 is an audio device that includes an amplifier, a speaker, etc. that generates sound, the control unit 130 controls the notification unit 180 to generate sound at the timing when the determination unit 114 switches the recognition information. to notify the user that the recognition information has been switched. Alternatively, when the notification unit 180 is a communication interface for communicating with the communication terminal 300, the control unit 130 transmits information indicating that the recognition information has been switched to the notification unit at the timing when the determination unit 114 switches the recognition information. 180 to the communication terminal 300. For example, the communication terminal 300 causes the above-described display unit to display an image showing the received information.

The various information (various data) stored in the storage unit 140 may be stored in advance in the storage unit 140, or may be stored in an external device (for example, the communication terminal 300) that can communicate with the autonomous vacuum cleaner 100. It may be received from etc.

The reception unit 190 is a device for accepting user operations. For example, the reception unit 190 is a communication interface that receives information such as the correspondence information 220 by communicating with an operating terminal (for example, the communication terminal 300) operated by a user. For example, reception unit 190 includes an antenna and a wireless communication circuit. Note that the autonomous vacuum cleaner 100 and the communication terminal 300 may communicate by wire instead of wirelessly. In this case, the reception unit 190 may be a terminal to which a communication line is connected.

Further, the reception unit 190 only needs to be able to accept user operations, and may be a user interface such as a touch panel or buttons that accept user input.

Further, the receiving unit 190 receives individual information including individual area information and individual target information.

The individual area information is information indicating a part of a predetermined space.

Individual target information is information indicating a recognition target.

For example, by operating the communication terminal 300, the user acquires map information 210 indicating a map of a predetermined space from the autonomous vacuum cleaner 100, and selects a part of the area on the map indicated by the acquired map information 210. do. Further, the user selects information indicating the type of object that the autonomous vacuum cleaner 100 wants to recognize in the selected area, and information such as an image of the object. The user selects individual area information that is information indicating the selected area, information indicating the type of object that the autonomous vacuum cleaner 100 wants to recognize in the selected area, and individual target information that is information such as an image of the object. , is transmitted to the communication terminal 300 from the autonomous vacuum cleaner 100. Thereby, the receiving unit 190 receives the individual information.

The registration unit 191 is a processing unit that causes the storage unit 140 to store the information received by the reception unit 190. For example, the registration unit 191 acquires various information such as map information 210, correspondence information 220, and driving pattern information 230 from an external device via the reception unit 190, and stores the information in the storage unit 140.

For example, the registration unit 191 causes the storage unit 140 to store recognition information that can recognize the recognition target indicated by the individual target information in the area indicated by the individual area information received by the reception unit 190 in association with the area.

Specifically, for example, the registration unit 191 associates a place ID, which is identification information for specifying the type of place (area) such as a living room or a kitchen, with an area corresponding to the individual area information received by the reception unit 190. and stored in the storage unit 140. That is, the registration unit 191 performs registration processing by assigning a location ID to each of a plurality of areas.

For example, the place name and place ID are arbitrarily determined in advance. When the registration unit 191 acquires the place name and coordinate information 221B via the reception unit 190, it assigns a place ID corresponding to the place name to these pieces of information and stores them in the storage unit 140.

Further, the registration unit 191 assigns a model ID to the recognition information for recognizing the recognition target indicated by the individual target information received by the reception unit 190 and links it to the area corresponding to the individual area information and stores it in the storage unit 140. to be memorized.

When recognition information for recognizing the recognition target indicated by the individual target information is stored in the storage unit 140, the registration unit 191 selects the relevant recognition target from among the plurality of recognition information and adds it to the individual area information. It may be stored in the storage unit 140 in association with the corresponding area.

Alternatively, for example, if recognition information is linked in advance to each of a plurality of areas, the registration unit 191 may set the recognition information corresponding to the area corresponding to the individual area information so that the recognition target indicated by the individual target information can be recognized. Alternatively, a learning unit, which is a processing unit (not shown), may perform machine learning using images and the like included in the individual target information. Alternatively, for example, the registration unit 191 communicates with an external terminal that stores a plurality of pieces of recognition information via a communication interface (not shown) included in the autonomous vacuum cleaner 100, so that the individual target information is received from the external terminal. Recognition information that allows recognition of the indicated recognition target may be acquired.

Various processing units such as a self-position acquisition unit 112, a correspondence information acquisition unit 113, a determination unit 114, a recognition unit 115, a shape detection unit 116, a coordinate detection unit 117, a plan generation unit 120, a control unit 130, a registration unit 191, etc. For example, it is realized by a control program for executing the above-described processing and a CPU (Central Processing Unit) for executing the control program. The various processing units may be realized by one CPU, or may be realized by multiple CPUs.

Note that the autonomous vacuum cleaner 100 includes, for example, various sensors such as a laser range finder 40, a camera 70, and a depth sensor 90. A cliff sensor that measures the distance from the autonomous vacuum cleaner 100, a slip sensor that detects the movement of the autonomous vacuum cleaner 100, an ultrasonic sensor that detects the distance from the autonomous vacuum cleaner 100 to an arbitrary object, a contact sensor, etc. may be provided. . Furthermore, for example, the autonomous vacuum cleaner 100 may include a sensor for detecting odometry information used to calculate the direction in which the autonomous vacuum cleaner 100 moves.

[Processing procedure]
Next, an overview of the processing procedure of the autonomous vacuum cleaner 100 will be explained with reference to FIGS. 7 to 14.

<Summary>
FIG. 7 is a flowchart showing an overview of processing executed by the autonomous vacuum cleaner 100 according to the embodiment.

First, assume that the autonomous vacuum cleaner 100 is powered on by a user or the like (step S101). As a result, the autonomous vacuum cleaner 100 starts operating.

For example, the registration unit 191 determines whether the map information 210 is registered (stored) in the storage unit 140 (step S102).

If the registration unit 191 determines that the map information 210 is not registered in the storage unit 140 (No in step S102), for example, the control unit 130 may control various sensors, the drive unit 160, etc. using the SLAM described above. The map information 210 is generated and stored in the storage unit 140 (step S110).

When the registration unit 191 determines that the map information 210 is registered in the storage unit 140 (Yes in step S102), the registration unit 191 determines whether or not the area information 221 is registered (stored) in the storage unit 140 (step S103).

If the registration unit 191 determines that the area information 221 is not registered in the storage unit 140 (No in step S103), or after step S110, it executes the process of registering the area information 221 (step S130). The registration unit 191 causes the storage unit 140 to store the area information 221 by executing a registration process for the area information 221.

When the registration unit 191 determines that the area information 221 is registered in the storage unit 140 (Yes in step S103), the registration unit 191 determines whether the model information 222 is registered (stored) in the storage unit 140 (step S104).

If the registration unit 191 determines that the model information 222 is not registered in the storage unit 140 (No in step S104), or after step S130, the registration process of the model information 222 is executed (step S150). The registration unit 191 causes the storage unit 140 to store the model information 222 by executing a registration process for the model information 222.

If the registration unit 191 determines that the model information 222 is registered in the storage unit 140 (Yes in step S104), or after step S150, the self-position acquisition unit 112, the correspondence information acquisition unit 113, and the determination unit 114, the recognition unit 115, the plan generation unit 120, the control unit 130, and other processing units execute cleaning processing (step S170). In step S170, the autonomous vacuum cleaner 100 autonomously travels and cleans a predetermined space.

<Area information registration process>
FIG. 8 is a flowchart showing details of the region information 221 registration process (step S130 shown in FIG. 7). 9A to 9F are diagrams each showing an example of a user's operation in the region information 221 registration process. 9A to 9F each illustrate a communication terminal 300 operated by a user and an image displayed on the communication terminal 300.

The communication terminal 300 is a computer that can communicate with the autonomous vacuum cleaner 100, such as a personal computer, a smartphone, or a tablet terminal operated by a user. The communication terminal 300 includes, for example, a reception unit that receives user operations, a display unit that displays images acquired from the autonomous vacuum cleaner 100, a communication interface for communicating with the autonomous vacuum cleaner 100, and the like. In this embodiment, communication terminal 300 is a smartphone, and includes a touch panel display that accepts user operations and displays images acquired from autonomous vacuum cleaner 100.

For example, the registration unit 191 receives area information 221 via the reception unit 190 (step S131). For example, the registration unit 191 transmits the map information 210 to the communication terminal 300 via the reception unit 190. When communication terminal 300 receives map information 210, it displays map information 210 as shown in FIG. 9A. By flicking the screen of the communication terminal 300, the user specifies, for example, a part of the area of the map shown in the map information 210. The communication terminal 300 transmits information indicating the specified range (individual area information) to the autonomous vacuum cleaner 100. Thereby, the registration unit 191 receives area information 221 including individual area information via the reception unit 190.

Next, the registration unit 191 assigns an area ID to the area indicated in the received area information 221 (step S132). The area ID only needs to be unique for each area. For example, the registration unit 191 transmits information indicating the assigned area ID to the communication terminal 300 via the reception unit 190. Thereby, for example, as shown in FIG. 9C, the communication terminal 300 displays the area ID so as to be superimposed on the area specified by the user, based on the received information indicating the area ID.

By repeating steps S131 and S132 with different areas on the map designated, area IDs are assigned to multiple areas on the map, as shown in FIG. 9D.

Next, the registration unit 191 assigns a location ID (location ID information 221A) to each area (step S133). For example, the user taps one of the area IDs displayed on the communication terminal 300, as shown in FIG. 9D. Here, it is assumed that the user performs a tap operation on "Area ID: 0001". Next, the communication terminal 300 displays location candidates, as shown in FIG. 9E. For example, when the registration unit 191 transmits information indicating the area ID via the reception unit 190, the information on the location candidate is also transmitted. The user selects one location from the displayed location candidates. Thereby, the user selects a location (location name) for the area. The communication terminal 300 transmits information indicating a place name for the area selected by the user to the autonomous vacuum cleaner 100. Thereby, the registration unit 191 receives information indicating the place name for the area via the reception unit 190, and assigns a place ID to each area based on the received information.

<Model information registration process>
FIG. 10 is a flowchart showing details of the model information 222 registration process (step S150 shown in FIG. 7). FIGS. 11A to 11D are diagrams showing examples of user operations in the model information 222 registration process, respectively. 9A to 9F each illustrate a communication terminal 300 operated by a user and an image displayed on the communication terminal 300.

The model information 222 is stored in the storage unit 140 in advance, for example. For example, the registration unit 191 associates recognition information with the location ID information 221A included in the area information 221 and stores it in the storage unit 140. In the model information 222 registration process described below, for example, the recognition target indicated by the individual recognition information received from the user is further added to the recognition target recognized by the recognition information originally stored in the storage unit 140.

Note that FIG. 10 shows the process after step S133 described above, and it is assumed that the image shown in FIG. 9F is displayed on the communication terminal 300.

First, the registration unit 191 acquires individual area information via the reception unit 190 (step S151). For example, the user taps any area (for example, area ID) displayed on the communication terminal 300, as shown in FIG. 11A. Here, it is assumed that the user performs a tap operation on the area of "area ID: 0001". The communication terminal 300 transmits, for example, information indicating the area tapped by the user to the autonomous vacuum cleaner 100. Thereby, the registration unit 191 acquires the individual area information by receiving the individual area information indicating the area tapped by the user via the reception unit 190.

Next, the registration unit 191 acquires individual object information indicating a recognition object to be registered (added) to the selected area (step S152).

Next, the registration unit 191 acquires label information indicating the name (label) of the recognition target indicated by the acquired individual target information (step S153).

For example, when the area of "area ID: 0001" is tapped, the communication terminal 300 displays "recognition target" as shown in FIG. 11B, and allows the user to register (add) the recognition target. to be processed. For example, as shown in FIG. 11C, the user inputs a photo and a name (label) of an object to be registered as a recognition target into the communication terminal 300. A photograph of an object is an example of model information 222. FIG. 11C illustrates a case where the user inputs "plastic bottle" and "remote control" as recognition targets in the area of "area ID: 0001". The communication terminal 300 transmits the area selected by the user, and the photo and label information input by the user to the autonomous vacuum cleaner 100. As a result, the registration unit 191 receives, via the receiving unit 190, individual area information indicating the area selected by the user, individual target information indicating the recognition target that the user wants to recognize in the area indicated by the individual area information, and a label. Get information. For example, as shown in FIG. 11D, the user determines the recognition target to be recognized by the autonomous vacuum cleaner 100 in each area by performing the above-described operation in each area.

Next, the registration unit 191 associates the area indicated by the individual area information, the recognition information indicating the individual target information with which the recognition target can be recognized, and the label information obtained through the reception unit 190 and stores them in the storage unit 140. It is stored (step S154). As described above, when recognition information for recognizing the recognition target indicated by the individual target information is stored in the storage unit 140, the registration unit 191 selects the relevant recognition target from among the plurality of recognition information. Alternatively, a learning unit (not shown) may be used for machine learning using images included in the individual target information so that the recognition target indicated by the individual target information can be recognized. Recognition information that can recognize the recognition target indicated by the individual target information may be acquired from an external terminal.

Thereby, the autonomous vacuum cleaner 100 comes to recognize the recognition target selected by the user in the area selected by the user.

<Cleaning process>
FIG. 12 is a flowchart showing details of the cleaning process (step S170 shown in FIG. 7).

First, the autonomous vacuum cleaner 100 receives a cleaning start signal (step S171). For example, the autonomous vacuum cleaner 100 includes a receiving section (not shown). The user operates a console or the like to transmit a cleaning start signal to the autonomous vacuum cleaner 100 indicating that cleaning is to be started. The autonomous vacuum cleaner 100 receives the cleaning start signal at the receiving unit. The receiving unit is, for example, an optical sensor for receiving a signal, a communication interface such as a wireless communication circuit when the signal is transmitted from a communication device such as a smartphone, or the like. Note that the autonomous vacuum cleaner 100 may have an operation section such as a button attached to the main body section 10 instead of the reception section and used to obtain instructions from the user. The autonomous vacuum cleaner 100 may acquire a cleaning start signal when the operation unit is operated by the user.

Next, the self-position acquisition unit 112 acquires the self-position of the autonomous vacuum cleaner 100 (S172). For example, the self-position acquisition unit 112 calculates coordinates on a map of a predetermined space indicated by the map information 210 based on the laser range finder 40 and the map information 210.

Next, the determination unit 114 determines one piece of recognition information from among the plurality of pieces of recognition information that differ in one or more recognition targets based on the self-position of the autonomous vacuum cleaner 100 (step S173). In step S173, for example, first, the correspondence information acquisition unit 113 associates recognition information with one or more different recognition targets for each of a plurality of regions, each of which is a part of a predetermined space and which is different from each other. The corresponding information 220 is obtained. Then, the determination unit 114 determines one piece of recognition information based on the self-position of the autonomous vacuum cleaner 100 and the correspondence information 220.

Next, the plan generation unit 120 generates plan information (step S174). For example, the plan generation unit 120 calculates a travel route based on map information 210 indicating a map of a predetermined space stored in the storage unit 140, and the cleaning unit 150, drive unit 160, and By controlling the suction unit 170, plan information for making the autonomous vacuum cleaner 100 travel and clean a predetermined space is generated.

Next, the control unit 130 controls the cleaning unit 150, the drive unit 160, and the suction unit 170 based on the plan information generated by the plan generation unit 120, so that the autonomous vacuum cleaner 100 can move into a predetermined space. The robot starts running and cleaning (step S175).

The autonomous vacuum cleaner 100 runs in a predetermined space based on planning information while determining whether or not a recognition target is included in the image by analyzing the captured image obtained from the camera 70. clean.

Next, the plan generation unit 120 determines whether cleaning is completed (step S176). For example, the plan generation unit 120 calculates the locus of the route traveled by the autonomous vacuum cleaner 100 based on the acquisition result of the self-position acquisition unit 112, and the calculated locus matches the travel route indicated by the plan information. In other words, it is determined whether the entire travel route indicated by the plan information has been traveled.

When the plan generation unit 120 determines that the cleaning plan is completed (Yes in step S176), the control unit 130 drives the drive unit 160 to move the autonomous vacuum cleaner 100 to the initial position where it started traveling. Return and end the process.

On the other hand, if the plan generation unit 120 determines that the cleaning is not completed (No in step S176), the self-location acquisition unit 112 acquires the self-location of the autonomous vacuum cleaner 100 again (step S177).

Next, the plan generation unit 120 determines whether the room (area) that the autonomous vacuum cleaner 100 cleans has been changed based on the self-position of the autonomous vacuum cleaner 100 (step S178). Specifically, the plan generation unit 120 determines whether the area being cleaned by the autonomous vacuum cleaner 100 has been moved from one area to another area based on the self-location and correspondence information 220. .

Next, if the plan generation unit 120 determines that the room to be cleaned by the autonomous vacuum cleaner 100 has been changed (Yes in step S178), the determination unit 114 assigns the recognition target in the changed room to the recognition unit 115. Recognition information is redetermined so as to be recognized (step S179). That is, the determination unit 114 changes the recognition information used by the recognition unit 115 based on the self-location of the autonomous vacuum cleaner 100 and the correspondence information 220.

If the control unit 130 determines that the room to be cleaned by the autonomous vacuum cleaner 100 has not been changed (No in step S178), or after step S179, the control unit 130 photographs a predetermined space. It is determined whether the recognition target is recognized by the recognition unit 115 in the captured image obtained from the camera 70 (step S180). That is, the recognition unit 115 determines whether the captured image obtained from the camera 70 includes the recognition target.

If the recognition target is not recognized by the recognition unit 115 (No in step S180), the control unit 130 returns the process to step S176 and controls the cleaning unit 150, the drive unit 160, and the suction unit 170 to achieve autonomous driving. The vacuum cleaner 100 is caused to travel and clean a predetermined space (step S180), and the process returns to step S176.

On the other hand, when the recognition unit 115 determines that the recognition target is recognized (Yes in step S180), the control unit 130 controls the cleaning unit 150, the driving unit 160, and the suction unit 170 based on the recognition result. Then, the autonomous vacuum cleaner 100 is caused to travel and clean a predetermined space (step S181), and the process returns to step S176. For example, when the recognition unit 115 recognizes the recognition target, the plan generation unit 120 determines the recognition target based on the type of the recognition target, the distance between the autonomous vacuum cleaner 100 and the recognition target, and the travel pattern information 230. Regenerate planning information in the surroundings. The control unit 130 causes the autonomous vacuum cleaner 100 to travel and clean based on the regenerated plan information.

Further, for example, when the recognition unit 115 recognizes a recognition target, the control unit 130 may send a notification to The user is notified by the section 180. For example, when the notification unit 180 is an audio device, when a “plastic bottle” is recognized in the “living room”, the control unit 130 sends a voice message such as “I found a plastic bottle in the living room” to the notification unit 180. may also be output.

<Specific example of cleaning operation>
13A to 13D are diagrams showing a first example of the cleaning operation of the autonomous vacuum cleaner 100 according to the embodiment. Note that FIG. 13A, FIG. 13B, FIG. 13C, and FIG. 13D are diagrams showing a case where the autonomous traveling vacuum cleaner 100 performs processing such as traveling in this chronological order. Furthermore, FIGS. 13A to 13D are schematic diagrams of the predetermined space CA viewed from above.

As shown in FIG. 13A, for example, when the plan generation unit 120 receives a cleaning start signal, it generates a travel route PP indicated by a broken line arrow based on map information 210 indicating a map of a predetermined space CA. The plan generation unit 120 also calculates coordinates with the origin at the place where the vehicle starts traveling.

Note that if the autonomous vacuum cleaner 100 does not hold the map information 210, it generates the map information 210 by SLAM while running around in the predetermined space CA. In that case, the plan generation unit 120 may set, for example, the position of the charger 250 as the origin of the coordinates.

In this specific example, the autonomous vacuum cleaner 100 holds map information 210 in advance (that is, stored in the storage unit 140), and is connected to the charger 250. Assume that the position is the origin.

Further, for example, the plan generation unit 120 sets the direction of movement as the positive X-axis direction, and sets the direction perpendicular to the X-axis as the Y-axis direction. In this case, before the autonomous vacuum cleaner 100 starts traveling, its own position is (X, Y, direction of travel) = (0 m, 0 m, 0°).

Note that X in (X, Y, direction of movement) includes, for example, the distance from the origin in the X-axis direction. Further, for example, Y in (X, Y, direction of movement) includes the distance from the origin in the Y-axis direction. Further, for example, the direction of movement (X, Y, direction of movement) includes the angle at which the autonomous vacuum cleaner 100 moves with respect to the positive direction of the X-axis.

The autonomous vacuum cleaner 100 determines recognition information, and travels and cleans the area CA1 based on the plan information and the recognition result of the recognition unit 115.

Note that, for example, the control unit 130 notifies the user of information indicating the area CA1 that the autonomous vacuum cleaner 100 is cleaning, and type information indicating the type of the object to be recognized, to the user using the notification unit 180. You may.

FIG. 14 is a diagram showing an example of an image displayed on the communication terminal 300 while the autonomous vacuum cleaner 100 according to the embodiment is cleaning.

For example, if the notification unit 180 is a communication interface for communicating with the communication terminal 300, the control unit 130 recognizes that the autonomous vacuum cleaner 100 recognizes a “plastic bottle” and a “remote control” and recognizes a “living room”. When cleaning is in progress, information such as an image and label indicating the current recognition target and information indicating the cleaning location are transmitted to the communication terminal 300 via the notification unit 180. When receiving information indicating the current recognition target and cleaning location, the communication terminal 300 displays an image indicating the information, as shown in FIG. 14, for example. Thereby, the control unit 130 uses the notification unit 180 to notify the user of information indicating the area CA1 that the autonomous vacuum cleaner 100 is cleaning and type information indicating the type of the object to be recognized. .

Next, as shown in FIG. 13B, it is assumed that the autonomous vacuum cleaner 100 travels from the state shown in FIG. 13A along the travel route PP passing through the trajectory T. For example, the autonomous vacuum cleaner 100 continues to travel while photographing the imaging range V with the camera 70 and determining whether or not an object (for example, a recognition target) exists around the autonomous vacuum cleaner 100 .

Here, it is assumed that the recognition unit 115 recognizes the object 400 as a recognition target on the travel route PP indicated by the plan information. In this case, the plan generation unit 120 regenerates the plan information based on the type of recognition target, the distance between the object 400 and the autonomous vacuum cleaner 100, and the travel pattern information 230. For example, the plan generation unit 120 regenerates plan information including a travel route PP1 that avoids the object 400. The control unit 130 controls the autonomous vacuum cleaner 100 to travel based on the regenerated plan information.

Next, as shown in FIG. 13C, it is assumed that the autonomous vacuum cleaner 100 has finished cleaning the area CA1. In this case, for example, the plan generation unit 120 generates plan information including the driving route of the area CA2. Further, for example, when changing the area that the autonomous vacuum cleaner 100 cleans, the control unit 130 notifies the user of the change using the notification unit 180. For example, if the notification unit 180 is an audio device and the area to be cleaned is changed to “bedroom”, the control unit 130 may send a message to the notification unit 180 such as “Moved to the bedroom. Switch recognition target”. Output audio.

Next, the autonomous vacuum cleaner 100 redetermines the recognition target, regenerates plan information including the travel route PP2 of the area CA2, etc., and based on the regenerated plan information and the recognition result of the recognition unit 115, Travel and clean area CA2.

Note that in the example shown in FIGS. 13A to 13D, the plan generation unit 120, for area CA1 and area CA2, which are part of the area of the predetermined space CA, Although a route (that is, plan information in area CA2) is generated, the present invention is not limited to this. The plan generation unit 120 may generate plan information including the travel route of the entire predetermined space CA before the autonomous vacuum cleaner 100 starts cleaning the predetermined space CA.

[Effects etc.]
As described above, the autonomous vacuum cleaner 100 according to the embodiment is an autonomous vacuum cleaner that autonomously travels and cleans a predetermined space. The autonomous vacuum cleaner 100 includes a self-position acquisition unit 112 that acquires the self-position of the autonomous vacuum cleaner 100 in a predetermined space, and a recognition unit that indicates an image processing method for recognizing a recognition target included in an image. A photographed image obtained from a determining unit 114 that determines one piece of recognition information based on the self-position from among a plurality of pieces of recognition information that each have one or more different recognition targets, and a camera 70 that photographs a predetermined space. By analyzing the image using the one recognition information determined by the determination unit 114, the recognition unit 115 recognizes the recognition target included in the captured image, and based on the recognition result of the recognition unit 115, autonomous driving is performed. It includes a control unit 130 that causes the vacuum cleaner 100 to travel and clean.

As described above, if the predetermined area is a place such as a house that includes multiple rooms of different types such as "living room", "bedroom", "kitchen", etc., the autonomous vacuum cleaner 100 Objects that are likely to have fallen on the floor where the vehicle runs are different. For example, there is a possibility that cooking utensils have fallen on the floor in the "kitchen", but it is highly unlikely that cooking utensils have fallen on the floor in the "living room". Here, if the number of objects that can be recognized by the recognition unit 115 is increased too much, there is a problem that the recognition rate decreases. For example, even objects of the same type may have different designs, shapes, etc., so if recognition targets are set according to these multiple designs, shapes, etc., the number of recognition targets in the recognition unit 115 becomes enormous. In other words, when supporting multiple classes, the recognition rate per class decreases. In other words, when the recognition unit 115 increases the number of objects that can be recognized, the possibility that the recognition unit 115 incorrectly determines the type of object recognized increases. Therefore, the autonomous vacuum cleaner 100 changes the recognition target as appropriate based on its own position. According to this, for example, in an area where the autonomous vacuum cleaner 100 is cleaning, an object that is unlikely to have fallen in the area can be excluded from the recognition target. Therefore, since the autonomous vacuum cleaner 100 can appropriately reduce unnecessary recognition targets, it is possible to improve the recognition rate of recognition targets.

For example, the autonomous vacuum cleaner 100 further includes recognition information in which one or more different recognition targets are associated with each of a plurality of different regions, each of which is a part of a predetermined space. The correspondence information acquisition unit 113 is provided to acquire the correspondence information 220 that has been obtained. In this case, the determining unit 114 determines the one piece of recognition information based on the self-location and correspondence information 220.

According to this, the determining unit 114 can determine an appropriate recognition target for each type of a plurality of areas included in a predetermined area.

For example, the autonomous vacuum cleaner 100 further includes a reception unit for receiving individual information including individual area information indicating a part of a predetermined space and individual object information indicating a recognition target. 190, and a registration unit 191 that causes the storage unit 140 to store recognition information that can recognize the recognition target indicated by the individual target information in the area indicated by the individual area information in association with the area.

According to this, in the autonomous vacuum cleaner 100, the registration unit 191 can store a recognition target desired by the user. Therefore, the determining unit 114 can determine the recognition target desired by the user for each type of a plurality of areas included in the predetermined area.

For example, the autonomous vacuum cleaner 100 further includes a notification unit for notifying the type of the recognized recognition target and the position of the recognized recognition target when the recognition unit 115 recognizes the recognition target. 180.

According to this, the user can determine whether or not the autonomous vacuum cleaner 100 can appropriately recognize the object that the user wants the autonomous vacuum cleaner 100 to recognize.

Furthermore, the method for controlling the autonomous vacuum cleaner 100 according to the embodiment is a method for controlling an autonomous vacuum cleaner that autonomously travels and cleans a predetermined space. The control method for the autonomous vacuum cleaner 100 includes a self-position acquisition step (step S172) for acquiring the self-position of the autonomous vacuum cleaner 100 in a predetermined space, and a step for recognizing a recognition target included in each image. A determining step (step S173) of determining one recognition information based on the self-position from among a plurality of recognition information indicating an image processing method and having one or more recognition targets different from each other; a recognition step (step S180) in which a recognition target included in the photographed image is recognized by analyzing the photographed image obtained from the camera 70 that photographs the space using one piece of recognition information determined in the determination step; The method includes a control step (step S181) of causing the autonomous vacuum cleaner 100 to travel and clean based on the recognition result in the recognition step.

According to this, effects similar to those of the autonomous vacuum cleaner 100 can be achieved.

Note that the present invention may be realized as a program that causes a computer to execute the steps included in the method for controlling the autonomous vacuum cleaner 100 according to the embodiment described above.

According to this, the method for controlling the autonomous vacuum cleaner 100 according to the present embodiment can be easily executed by a computer.

Further, the present invention may be realized as a non-temporary recording medium such as a computer readable CD-ROM in which the above program is recorded. Further, the present invention may be realized as information, data, or a signal indicating the program. These programs, information, data, and signals may be distributed via a communication network such as the Internet.

(Other embodiments)
Although the autonomous vacuum cleaner and the like according to the present invention have been described above based on the above embodiments and modifications, the present invention is not limited to the above embodiments and modifications.

Further, for example, in the above embodiment, it has been explained that the processing units such as the plan generation unit and the control unit included in the autonomous vacuum cleaner are realized by the CPU and the control program, respectively. For example, each of the components of the processing unit may be composed of one or more electronic circuits. Each of the one or more electronic circuits may be a general-purpose circuit or a dedicated circuit. The one or more electronic circuits may include, for example, a semiconductor device, an IC (Integrated Circuit), an LSI (Large Scale Integration), or the like. An IC or LSI may be integrated into one chip or into multiple chips. Here, it is called an IC or LSI, but the name changes depending on the degree of integration, and may be called a system LSI, VLSI (Very Large Scale Integration), or ULSI (Ultra Large Scale Integration). Furthermore, an FPGA (Field Programmable Gate Array) that is programmed after the LSI is manufactured can also be used for the same purpose.

Further, general or specific aspects of the invention may be implemented in a system, apparatus, method, integrated circuit, or computer program product. Alternatively, the computer program may be realized by a computer-readable non-temporary recording medium such as an optical disk, an HDD (Hard Disk Drive), or a semiconductor memory in which the computer program is stored. Further, the present invention may be realized by any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

In addition, forms obtained by applying various modifications to the embodiments and modifications that those skilled in the art can think of, or by arbitrarily combining the components and functions of the embodiments without departing from the spirit of the present invention. The present invention also includes such forms.

INDUSTRIAL APPLICABILITY The present invention can be widely used in autonomous vacuum cleaners that clean while autonomously moving.

10 Main body part 11 Suction port 20 Wheel 30 Side brush 40 Laser range finder 50 Main brush 70 Camera 90 Depth sensor 100 Autonomous vacuum cleaner 112 Self-position acquisition part 113 Correspondence information acquisition part 114 Determination part 115 Recognition part 116 Shape detection part 117 Coordinate detection unit 120 Plan generation unit 130 Control unit 140 Storage unit 150 Cleaning unit 151 Brush motor 160 Drive unit 161 Wheel motor 170 Suction unit 171 Suction motor 180 Notification unit 190 Reception unit 191 Registration unit 210 Map information 220 Correspondence information 221 Area information 221A Place ID information 221B Coordinate information 221C Area ID information 222 Model information 230 Travel pattern information 230A Type name information 230B Distance information 230C Pattern information 250 Charger 300 Communication terminal 400 Object CA Predetermined space CA1, CA2 Area PP, PP1, PP2 Travel Route T Trajectory V Shooting range

Claims (6)

An autonomous vacuum cleaner that autonomously moves and cleans a predetermined space,
a self-position acquisition unit that acquires the self-position of the autonomous vacuum cleaner in the predetermined space;
Each piece of recognition information indicates an image processing method for recognizing a recognition target included in an image, and one recognition information is selected from a plurality of pieces of recognition information that differ from each other in one or more recognition targets based on the self-position. a deciding section that decides;
a recognition unit that recognizes a recognition target included in the photographed image by analyzing the photographed image obtained from the camera that photographs the predetermined space using the one recognition information determined by the determination unit;
An autonomous vacuum cleaner, comprising: a control unit that causes the autonomous vacuum cleaner to travel and clean based on the recognition result of the recognition unit. Furthermore, a correspondence information acquisition unit that acquires correspondence information in which recognition information in which one or more recognition targets are different from each other is linked for each of a plurality of mutually different regions, each of which is a part of the predetermined space. Prepare,
The autonomous vacuum cleaner according to claim 1, wherein the determining unit determines the one recognition information based on the self-position and the correspondence information. Further, a reception unit for receiving individual information including individual area information indicating a part of the predetermined space and individual target information indicating a recognition target;
Autonomous driving according to claim 1 or 2, further comprising: a registration unit that stores recognition information capable of recognizing a recognition target indicated by the individual target information in a storage unit in a region indicated by the individual region information in association with the region. type vacuum cleaner. The autonomous driving according to claim 2 or 3, further comprising a notification unit for notifying the type of the recognized recognition target and the position of the recognized recognition target when the recognition unit recognizes the recognition target. type vacuum cleaner. A method of controlling an autonomous vacuum cleaner that autonomously moves and cleans a predetermined space,
a self-position acquisition step of acquiring a self-position of the autonomous vacuum cleaner in the predetermined space;
A plurality of pieces of recognition information, each of which indicates an image processing method for recognizing a recognition target included in an image, and one piece of recognition information from among the plurality of pieces of recognition information that differ from each other in one or more recognition targets is placed at the self-position. a decision step of deciding based on;
a recognition step of recognizing a recognition target included in the photographed image by analyzing the photographed image obtained from the camera photographing the predetermined space using the one recognition information determined in the determining step;
A method for controlling an autonomous vacuum cleaner, including a control step of causing the autonomous vacuum cleaner to run and clean based on the recognition result in the recognition step. A program for causing a computer to execute the method for controlling an autonomous vacuum cleaner according to claim 5.

Images