JP2017167625A - Autonomous mobile device, autonomous mobile system, autonomous mobile method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To move to a destination while preventing communication from being disconnected.SOLUTION: An autonomous mobile device 100 comprises: a map creation unit 11 which creates a map using information on a plurality of images captured by an imaging unit; a position estimation unit 12 which estimates a self-machine position using the information on a plurality of images captured by the imaging unit; a communication quality measurement unit 13 which measures communication quality for communicating with an external communication device; a communication quality map creation unit 14 which creates a communication quality map using the self-machine position estimated by the position estimation unit and the communication quality measured by the communication quality measurement unit; and a route search unit 15 which searches for a route to a destination using the communication quality map created by the communication quality map creation unit.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an autonomous mobile device, an autonomous mobile system, an autonomous mobile method, and a program.

  Autonomous mobile devices that move autonomously according to their use have become widespread. For example, an autonomous mobile device that moves autonomously for indoor cleaning is known. In general, such an autonomous mobile device needs to create a map of the real space and estimate its own position in the real space. As a technique for creating a map of the real space, for example, a SLAM (Multiple Localization And Mapping) method is known. The basic principle of SLAM technology using a monocular camera is described in Non-Patent Document 1 and Non-Patent Document 2, and by tracking the same feature points from multiple frames of moving images taken by the camera, A process of estimating the three-dimensional position of the own device (camera position) and the three-dimensional position of the feature points (which together constitute map information) is performed.

  If computation processing such as image recognition and voice recognition of this autonomous mobile device is processed by an external computing device connected to the network, the network communication function is required instead of saving computation power and storage capacity on the autonomous mobile device side. There are many merits such as the omission of the cooling mechanism and the reduction in the weight of the battery etc., the enlargement of the database and the real-time sharing, and the ease of updating.

  As a prior art of a mobile robot that performs such wireless communication, Patent Document 1 discloses that even when moving to a place where wireless communication is disconnected, the mobile robot can move autonomously to a place where wireless communication can be restored. A possible mobile robot is disclosed.

JP 2008-87102 A

Andrew J. Davison, “Real-Time Simulaneous Localization and Mapping with a Single Camera”, Proceedings of the 9th IEEE International Conference on Computer, Computer Viper. 1403-1410 Richard Hartley, Andrew Zisserman, “Multiple View Geometry in Computer Vision”, Second Edition, Cambridge. University Press, March 2004, chapter 9

  However, in the technique disclosed in Patent Document 1, when communication is disconnected, the original destination is ignored and the communication is made to a place where communication is possible. Therefore, it is possible to move to the destination so that communication is not disconnected. There wasn't.

  The present invention has been made to solve the above-described problem, and an object thereof is to provide an autonomous mobile device and the like that can move to a destination so that communication is not disconnected.

In order to achieve the above object, the autonomous mobile device of the present invention provides:
A map creation unit that creates a map using information of a plurality of images taken by the imaging unit;
A position estimation unit that estimates the position of the device using information of a plurality of images captured by the imaging unit;
A communication quality measurement unit for measuring communication quality when communicating with an external communication device;
A communication quality map creating unit that creates a communication quality map using the own device position estimated by the position estimating unit and the communication quality measured by the communication quality measuring unit;
A route search unit that searches for a route to a destination based on the communication quality map created by the communication quality map creation unit;
It is characterized by providing.

  According to the present invention, it is possible to move to a destination so that communication is not disconnected.

It is a figure which shows the external appearance of the autonomous mobile device which concerns on embodiment. It is a figure which shows the structure of the autonomous mobile apparatus which concerns on embodiment. (A) It is a figure which shows the structure via a network among the system structures which concern on embodiment. (B) It is a figure which shows the structure by direct communication among the system structures which concern on embodiment. It is a figure which shows the flowchart of the whole autonomous movement control process which concerns on embodiment. It is a figure which shows the flowchart of the process of the own position estimation thread | sled in the autonomous movement control process which concerns on embodiment. It is a figure which shows the flowchart of the process of the map creation thread in the autonomous movement control process which concerns on embodiment. It is a figure which shows the specific example of map and route cost map S (x, y). It is a figure which shows the specific example of communication quality cost map C (x, y). It is a figure which shows the flowchart of the process of the loop closing thread | sled in the autonomous movement control process which concerns on embodiment. It is a figure which shows the flowchart of the route plan to the destination which concerns on 1st Embodiment. (A) It is a figure explaining the specific example of the route plan to the destination which concerns on 1st Embodiment by map and route cost map S (x, y). (B) It is a figure explaining the specific example of the route plan to the destination which concerns on 1st Embodiment by the cost map T (x, y) for route searches. It is a figure which shows the flowchart of the route plan to the destination which concerns on 2nd Embodiment. (A) It is a figure explaining the specific example of the route plan to the destination based on 2nd Embodiment by the cost map T (x, y) for route searches. (B) It is a figure explaining the specific example of the route plan to the destination which concerns on 2nd Embodiment by the communication quality cost map C (x, y). (A) It is a figure explaining the other specific example of the route plan to the destination which concerns on 2nd Embodiment by the cost map T (x, y) for route searches. (B) It is a figure explaining with the cost map T (x, y) for a route search updated in the other specific example of the route plan to the destination which concerns on 2nd Embodiment.

  Hereinafter, an autonomous mobile device according to an embodiment of the present invention will be described with reference to FIG. The autonomous mobile device 100 moves autonomously according to the application. Examples of this use include security monitoring, indoor cleaning, pets, and toys.

  The autonomous mobile device 100 includes an imaging unit 41 and a drive unit 42 in appearance.

  The imaging unit 41 includes a monocular imaging device (camera). The imaging unit 41 acquires an image (frame) at 30 fps, for example. The autonomous mobile device 100 performs autonomous movement while recognizing its own position and surrounding environment in real time based on images sequentially acquired by the imaging unit 41.

  The drive unit 42 is an independent two-wheel drive type, and is a moving means including wheels and a motor. The autonomous mobile device 100 translates back and forth (translation) by driving the two wheels in the same direction, and rotates (changes direction) on the spot by driving the two wheels in the reverse direction. A swivel movement (translation + rotation (direction change) movement) can be performed by the changed drive. Each wheel is also equipped with a rotary encoder, which measures the rotational speed of the wheel with the rotary encoder and uses the geometric relationship such as the wheel diameter and the distance between the wheels to translate and move. The amount can be calculated.

For example, if the wheel diameter is D and the rotation speed is R (measured by a rotary encoder), the translational movement amount at the ground contact portion of the wheel is π · D · R. Further, if the wheel diameter is D, the distance between the wheels is I, the rotation speed of the right wheel is R _R , and the rotation speed of the left wheel is _RL , the rotation amount of the direction change is 360 (when the right rotation is positive). ° × D × (R _L −R _R ) / (2 × I). By sequentially adding the translational movement amount and the rotation amount, the drive unit 42 functions as a so-called odometry, and measures its own position (position and orientation with reference to the position and orientation at the start of movement). Can do.

  In addition, you may make it provide a crawler instead of a wheel, and you may make it move by providing a plurality (for example, two) leg | foot and walking with a leg | foot. In these cases as well, the position and orientation of the subject aircraft can be measured based on the movements of the two crawlers and the movements of the feet as in the case of the wheels.

  As shown in FIG. 2, the autonomous mobile device 100 includes a control unit 10, a storage unit 20, a sensor unit 30, an input unit 43, a communication unit 44, and a power supply 45 in addition to the imaging unit 41 and the drive unit 42.

  The control unit 10 is configured by a CPU (Central Processing Unit) or the like, and by executing a program stored in the storage unit 20, each unit described later (a map creation unit 11, a position estimation unit 12, a communication quality measurement unit 13, The functions of the communication quality map creation unit 14 and the route search unit 15) are realized.

  The storage unit 20 includes a ROM (Read Only Memory), a RAM (Random Access Memory), and the like, and includes an image storage unit 21, a map storage unit 22, a communication quality map storage unit 23, and a route search map storage unit 24. The ROM stores a program executed by the CPU of the control unit 10 (for example, a program related to communication quality map creation and autonomous movement control processing described later) and data necessary for executing the program in advance. The RAM stores data that is created or changed during program execution.

  The image storage unit 21 stores an image captured by the imaging unit 41. However, in order to save the storage capacity, it is not necessary to store all captured images, and it is also possible to store the feature amount of the image instead of the image itself.

  The map storage unit 22 stores a map (information on walls and obstacles) created by the map creation unit 11 based on information from the SLAM method and distance sensor 31 described later.

  The communication quality map storage unit 23 stores a communication quality map created by a communication quality map creation unit 14 described later.

  The route search map storage unit 24 stores a route search map that is created from information in the map storage unit 22 and the communication quality map storage unit 23 and is used to search for a route that takes communication quality into consideration.

  A distance sensor 31 is provided as the sensor unit 30. The distance sensor 31 is a sensor that measures the distance to surrounding walls and obstacles, and is, for example, a sensor that uses infrared rays or ultrasonic waves. In addition, you may make it measure the distance to a wall or an obstruction using the imaging part 41, without mounting the independent distance sensor 31. FIG. Moreover, you may provide the bumper sensor (not shown) which detects having collided with the other object.

  As the input unit 43, an operation button for operating the autonomous mobile device 100 is provided. The operation buttons include, for example, a power button, a mode switching button (switching between the cleaning mode and the pet mode), an initialization button (re-creating the map), and the like. As the input unit 43, a microphone (not shown) that inputs sound and a voice recognition unit that recognizes a voice of an operation instruction to the autonomous mobile device 100 may be provided.

  The communication unit 44 is a wireless module for performing wireless communication with the external communication device 200. Generally, it is a wireless LAN (Local Area Network) module using radio waves, but may be one using infrared rays or visible light. In communication with the external communication device 200, communication may be performed via a network as illustrated in FIG. 3A, or direct communication may be performed as illustrated in FIG. 3B. As shown in FIG. 3, not only the autonomous mobile device 100 but also the communication device 200 includes a communication unit 44.

  The power source 45 is a power source for operating the autonomous mobile device 100, and is generally a built-in rechargeable battery, but may be a solar cell or a system that is wirelessly powered from the floor. Also good. When the power supply 45 is a rechargeable battery, the autonomous mobile device 100 is charged by being docked at a charging station (home base).

  Next, functions of the control unit 10 will be described. The control unit 10 includes a map creation unit 11, a position estimation unit 12, a communication quality measurement unit 13, a communication quality map creation unit 14, and a route search unit 15, and creates a communication quality map, which will be described later, and movement control of the autonomous mobile device 100. Etc. The control unit 10 supports a multi-thread function, and can proceed with a plurality of threads (different processing flows) in parallel.

  The map creation unit 11 stores the feature points (Map points) estimated using the SLAM method based on the image information stored in the image storage unit 21 and the position and orientation information of the own device at the time of image capture. The position, the position of the wall or obstacle obtained based on the position and orientation information of the own machine when the wall or obstacle is detected by the distance sensor 31, and the like are stored in the map storage unit 22 as map information. .

  The position estimation unit 12 estimates the position and orientation of its own device as visual odometry based on the SLAM method described later.

  The communication quality measurement unit 13 measures the communication quality when the communication unit 44 communicates with the external communication device 200.

  The communication quality map creating unit 14 measures the communication quality when communicating with the external communication device 200 by the communication quality measuring unit 13 at each point around the autonomous mobile device 100, and the communication quality map (communication at each point). A map with recorded quality).

  The route search unit 15 searches for a route based on the map information stored in the route search map storage unit 24.

  The main flow of the autonomous movement control process of the autonomous mobile device 100 will be described with reference to FIG. First, the control unit 10 sets “unset”, which is an initial value, to a variable (destination) indicating a destination (step S101). Next, the control unit 10 activates its own position estimation thread (step S102). Then, the map creation thread is activated (step S103). Next, the loop closing thread is activated (step S104). Generation of map information and visual odometry (information of the own position estimated using the map and the image) is started based on the SLAM method by the operation of the own position estimation thread and the map creation thread. Thereafter, the control unit 10 determines whether or not the operation is finished (step S105). If the operation is finished (step S105; Yes), the operation is finished. If the operation is not finished (step S105; No), the distance sensor 31 and visual The map is updated using odometry (step S106).

  Next, the control unit 10 determines whether or not there is an instruction to move to the destination (step S107). If there is a movement instruction (step S107; Yes), the instructed destination is set as a variable (destination) indicating the destination (step S110), and a route plan to the destination is performed (step S111). Details of the route plan to the destination will be described later. Then, the control unit 10 issues an operation instruction to the drive unit 42 in order to move along the route set by the route plan (step S112), and returns to step S105.

  If there is no instruction to move to the destination (step S107; No), the control unit 10 determines whether the destination is not set (step S108). If the destination is not set (step S108; Yes), free autonomous movement is performed (step S109), and the process returns to step S105. Free autonomous movement may be movement that moves randomly, or movement that moves around the room to create a map.

  When the destination is set (step S108; No), the route is set by the route plan, so the control unit 10 issues an operation instruction to the drive unit 42 in order to move along the route ( Step S112) and return to Step S105. A variable indicating the destination (destination) is stored in the storage unit 20 so that the value can be referred to across the threads.

  By this main flow process, it is possible to move to a destination that has received a movement instruction via an efficient movement route while autonomously moving while updating the map. As a typical example, when the autonomous mobile device 100 is initially turned on in a state where it is placed at the charging station, it relies on the distance sensor 31 to move through each room of the house, and the distance sensor 31 moves a wall or the like. The position of the obstacle can be specified, and the map information including the obstacle position can be created. When a map is created to some extent, it is possible to know an area that has no map information but is considered to be movable, and it is possible to encourage the creation of a wider map by moving autonomously to that area. It becomes like this. And if the information of the map of the almost movable whole area is created, the efficient movement operation | movement using the information of a map will be attained. For example, it is possible to return to the charging station through the shortest route from any position in the room or to efficiently clean the room.

  Next, the own position estimation thread activated in step S102 of the main flow (FIG. 4) of the autonomous mobile device 100 will be described with reference to FIG. This thread is a process in which the position estimation unit 12 first performs an initialization process, and then continues its own position estimation (estimating its own position by visual odometry using an image acquired by the imaging unit 41).

  The position estimation unit 12 first sets “initializing” to the estimated state variable (step S201). The estimated state variable indicates the state of the own position estimation process at that time, and takes three values of “initializing”, “tracking success”, and “tracking failure”. Next, it is determined whether or not the operation is finished (step S202). If the operation ends (step S202; Yes), the operation ends. If the operation does not end (step S202; No), it is determined whether or not the estimated state variable is “initializing” (step S203). If initialization is not in progress (step S203; No), the apparatus position estimation process from step S221 is performed. If initialization is in progress (step S203; Yes), the process proceeds to step S204 to perform initialization process. First, the initialization process will be described.

  In the initialization process, the position estimation unit 12 first sets −1 to the frame counter N (step S204), and the imaging unit 41 acquires an image (step S205). The image can be acquired at 30 fps, for example (the acquired image is also called a frame). Next, the position estimation unit 12 acquires 2D feature points included in the image acquired by the imaging unit 41 (step S206).

  A 2D feature point is a characteristic part in an image, such as an edge part in the image, and can be acquired by using an algorithm such as SIFT (Scale-Invariant Future Transform) or SURF (Speed-Up Robust Features). it can.

  If the number of acquired 2D feature points is small, calculation by the Two-view Structure from Motion method, which will be described later, is impossible. Therefore, the position estimation unit 12 compares the number of 2D feature points acquired with a reference value (for example, 10). If it is less than the reference value (step S207; No), the process returns to step S205, and the acquisition of the image and the acquisition of the 2D feature points are repeated until the number of 2D feature points equal to or greater than the reference value is obtained. Note that map information has not yet been created at this point. However, in the typical example described above, for example, since the distance sensor 31 has been relied on to start moving all rooms in the house, this initialization process If image acquisition and 2D feature point acquisition are repeated, image acquisition is repeated while moving. Therefore, it can be expected that various images can be acquired, and an image with a large number of 2D feature points can be acquired.

  If the number of 2D feature points acquired is equal to or greater than the reference value (step S207; Yes), the position estimation unit 12 increments the frame counter N (step S208). Then, it is determined whether or not the frame counter N is 0 (step S209). If the frame counter N is 0 (step S209; Yes), it means that only one image has been acquired yet, so the process returns to step S205 to acquire the second image. Although not described in the flowchart of FIG. 5, the position of the own machine when the first image is acquired and the position of the own machine when the second image are acquired are somewhat distant. The accuracy of the posture estimated in the subsequent processing is improved. Therefore, when returning from step S209 to step S205, the process waits until the translation distance by odometry becomes a predetermined distance (for example, 1 m) or more due to free autonomous movement in step S109 of the main flow (FIG. 4), and then step S205. You may make it return to.

  If the frame counter N is not 0 (step S209; No), it means that two images have been acquired. Therefore, the position estimation unit 12 corresponds the 2D feature points between these two images (the same point in the actual environment is The image that is present in each image and can be dealt with is acquired (step S210). If the number of corresponding feature points is less than 5, the position estimation unit 12 determines whether or not the number of corresponding feature points is less than 5 because the posture between two images to be described later cannot be estimated (step S211). . If it is less than 5 (step S211: Yes), the process returns to step S204 in order to reacquire the initial image. If the number of corresponding feature points is 5 or more (step S211; No), using the Two-view Structure from Motion method described later, the posture between the two images (the difference between the positions where the respective images are acquired ( Translation vector t) and orientation difference (rotation matrix R)) can be estimated (step S212).

  Specifically, the estimation of the posture by the Two-view Structure from Motion method is performed by obtaining a basic matrix E from corresponding feature points and decomposing the basic matrix E into a translation vector t and a rotation matrix R ( For details, see Non-Patent Document 2.) Note that the value of each element of the translation vector t obtained here (having three elements of X, Y, and Z with the position where the first image is acquired as the origin, assuming that it moves in a three-dimensional space) is Since it is different from the value in the real environment (the Two-view Structure from Motion method cannot obtain the value in the real environment itself, but obtains the value in a space similar to the real environment). These are regarded as values on the SLAM space, and will be described below using coordinates (SLAM coordinates) on the SLAM space.

  When the posture (translation vector t and rotation matrix R) between the two images is obtained, the values are based on the first image (the position where the first image was acquired is the origin of SLAM coordinates, the translation vector is the 0 vector, the rotation matrix) Is the unit matrix I.) The orientation of the second image (the position (translation vector t) and orientation (rotation matrix R) of the own machine when the second image is acquired)) . Here, when the postures of the two images (the position (translation vector t) and direction (rotation matrix R) of the own device at the time of photographing the image (frame) are also referred to as frame postures) are obtained. Based on the following concept, the map creation unit 11 obtains the 3D position in the SLAM coordinates of 2D feature points (corresponding feature points) that can be matched between two images (step S213).

If the coordinates (frame coordinates: known) of the 2D feature point in the image are (u, v) and the 3D position (unknown) in the SLAM coordinate of the 2D feature point is (X, Y, Z), these are the same. These relationships when expressed in the next coordinates are expressed by the following formula (1) using the perspective projection matrix P. Here, the symbol “˜” represents “equal except for a non-zero constant multiple” (that is, equal or is a constant (non-zero) multiple), and the “′” symbol represents “transpose”. .
(U v 1) ′ to P (XY Y 1) ′ (1)

In the above equation (1), P is a 3 × 4 matrix, and from the 3 × 3 matrix A indicating the internal parameters of the camera and the external parameters R and t indicating the posture (frame posture) of the image, the following equation It is represented by (2). Here, (R | t) represents a matrix in which the translation column vector t is arranged to the right of the rotation matrix R.
P = A (R | t) (2)

  In the above equation (2), R and t are obtained as the frame postures as described above. Further, since the internal parameter A of the camera is determined by the focal length and the image sensor size, it is a constant if the image capturing unit 41 is determined.

One of the 2D feature points corresponding to the two images is the frame coordinates (u ₁ , v ₁ ) of the first image and the frame coordinates (u ₂ , v ₂ ) of the second image. ), The following equations (3) and (4) can be obtained. Here, I is a unit matrix, 0 is a zero vector, and (L | r) is a matrix in which a column vector r is arranged to the right of the matrix L.
(U ₁ v ₁ 1) ′ to A (I | 0) (XY Z 1) ′ (3)
(U ₂ v ₂ 1) ′ to A (R | t) (XY Z 1) ′ (4)

In the above formulas (3) and (4), because there are formulas for u ₁ , v ₁ , u ₂ , and v _2, there are four formulas, but there are three unknowns, X, Y, and Z. , X, Y, Z can be obtained, and this is the 3D position of the 2D feature point in the SLAM coordinates. Since the number of equations is larger than the number of unknowns, for example, X, Y, Z obtained from u ₁ , v ₁ , u ₂ and X, Y, Z obtained from u ₁ , v ₁ , v ₂ are used. May be different. In such a case, it becomes a simultaneous linear equation with an excess condition, and there is generally no solution, but the map creation unit 11 finds the most probable X, Y, and Z using the least square method.

  When the 3D position (X, Y, Z) of the 2D feature point in the SLAM coordinate is obtained, it is used as a Map point, and the map creation unit 11 is also called a Map point database (Map point DB (Database), which is stored in the map storage unit 22. Registered) (step S214). As elements to be registered in the Map point database, at least “X, Y, Z which is a 3D position of a 2D feature point in SLAM coordinates” and “a feature amount of the 2D feature point” (for example, a feature amount obtained by SIFT or the like) )is necessary.

  Then, the map creation unit 11 determines whether all 2D feature points (corresponding feature points) corresponding to each other between the two images are registered in the Map point database (step S215), and all the registrations are still possible. If not (Step S215; No), the process returns to Step S213, and if all of them are registered (Step S215; Yes), the process proceeds to Step S216.

  Then, the position estimation unit 12 initializes a counter NKF of a key frame (pointing to an image to be processed in a subsequent thread) to 0 (step S216), and uses the second image as a key frame to generate a frame database (frame (Also referred to as DB and stored in the image storage unit 21) (step S217).

  Elements to be registered in the frame database are “key frame number” (value of the key frame counter NKF at the time of registration), “posture” (position (translation vector t) in the SLAM coordinates of the own device at the time of image capture) Direction (rotation matrix R)), “all extracted 2D feature points”, “a point where 3D position is known as a Map point among all 2D feature points”, “feature of key frame itself”, In addition to these, the “posture in the actual environment measured by odometry” (the position and orientation of the own machine determined based on the movement distance by the drive unit 42 in the actual environment) may be registered.

  Among the above, “features of the key frame itself” is data for improving the processing for obtaining the image similarity between the key frames, and it is usually preferable to use a histogram of 2D feature points in the image. The image itself may be a “characteristic of the key frame itself”. In addition, the “posture in the real environment measured by odometry” can be expressed by a translation vector t and a rotation matrix R. However, since the autonomous mobile device 100 normally moves on a two-dimensional plane, two-dimensional data Alternatively, it may be expressed as two-dimensional coordinates (X, Y) and direction φ with reference to the position (origin) and direction at the start of movement.

  Next, in order to inform the map creation thread that the key frame has been generated, the position estimation unit 12 stores the key frame queue of the map creation thread (the queue has a first-in first-out data structure). The key frame counter NKF is set (step S218).

  Since the initialization processing of the own position estimation thread is completed as described above, the position estimation unit 12 sets “successful tracking” to the estimated state variable (step S219), and obtains the scale correspondence between the SLAM coordinates and the real environment coordinates. Therefore, the scale S is obtained by dividing the translation distance (obtained by the coordinates in the real environment) by odometry by the translation distance d in the SLAM coordinates estimated by the above processing (step S220).

  And it progresses to step S221 which is a process in the case of having completed initialization via step S202 and step S203.

  Processing in the case of initialization is described. This process is a normal process of the own machine position estimation thread, and the position estimation unit 12 sequentially estimates the current position and orientation of the own machine (translation vector t and rotation matrix R in SLAM coordinates). It is processing.

  First, the position estimation unit 12 takes an image with the imaging unit 41 (step S221), and increments the frame counter N (step S222). And the feature point acquisition part 11 acquires the 2D feature point contained in the image image | photographed with the imaging part 41 (step S223).

  Next, the position estimation unit 12 determines whether or not the estimated state variable is “tracking success” (step S224). If it is “tracking success” (step S224; Yes), the position estimation unit 12 converts the information of the previous key frame (image having the key frame number NKF) registered in the frame database into the information of the image. Among the included 2D feature points, 2D feature points whose 3D positions are known (which are Map points registered in the Map point database) are acquired, and the feature points acquired in step S223 therein The number of 2D feature points (corresponding feature points) that can be matched with each other (number of feature points corresponding) is acquired (step S225).

  If the value of the estimated state variable is not “tracking success” (step S224; No), the position estimating unit 12 has two or more previous key frames registered in the frame database (an image with a key frame number less than NKF). The key frame having the maximum number of correspondence between the Map point and the feature point acquired in step S223 and the feature point correspondence number are acquired (step S226).

  Then, the position estimation unit 12 determines whether or not the feature point correspondence number is larger than a threshold (for example, 30. Hereinafter referred to as “reference corresponding feature point number”) (step S227). In S227; No), since the accuracy of the posture estimated by the SLAM method is deteriorated, “tracking failure” is set in the estimated state variable (step S230), and the process returns to step S202 without estimating the position.

  When the feature point correspondence number is larger than the reference correspondence feature point number (step S227; Yes), the key frame in the vicinity of the key frame for which the feature point correspondence number is obtained in step S225 or step S226 (the key frame for which the feature point correspondence number is obtained) And the number of feature points (corresponding feature points) acquired in step S223 that can correspond to the Map points included in key frames where there is an overlap of feature points over a predetermined ratio (for example, 30% or more) ) Is acquired by the position estimation unit 12 (step S228).

  Then, the position estimation unit 12 determines whether or not the feature point correspondence number is larger than a threshold value (for example, 50, hereinafter referred to as “second reference correspondence feature number”) (step S229), and the second reference correspondence feature. If it is below the score (step S229; No), “tracking failure” is set in the estimated state variable (step S230), and the process returns to step S202 without estimating the position. If the feature point correspondence number is larger than the second reference correspondence feature point number (step S229; Yes), the process proceeds to step S231.

In step S231, the position estimation unit 12 sets tracking success to the estimated state variable (step S231). The position estimation unit 12 acquires the 3D position (X _i , Y _i , Z _i ) of each corresponding feature point acquired in step S228 from the Map point database, and the corresponding feature point included in the image acquired in step S221. Is used to estimate the current attitude of the aircraft (the position and orientation of the aircraft represented by the translation vector t and the rotation matrix R) (step S232).

A supplementary explanation will be given on the method for estimating the attitude of the aircraft. The frame coordinates of the corresponding feature points included in the image acquired in step S221 are (u _i , v _i ), and the 3D positions of the corresponding feature points are (X _i , Y _i , Z _i ) (i is 1 to the number of corresponding feature points), and the value (ux _i ,) of the 3D position (X _i , Y _i , Z _i ) of each corresponding feature point projected onto the frame coordinate system by the following equation (5) vx _i ) and the frame coordinates (u _i , v _i ) should ideally match.
(Ux _i vx _i 1) ′ to A (R | t) (X _i Y _i Z _i 1) ′ (5)

However, since (X _i , Y _i , Z _i ) and (u _i , v _i ) actually contain errors, (ux _i , vx _i ) and (u _i , v _i ) and Rarely match. And the unknowns are only R and t (three-dimensional in the three-dimensional space, and 3 + 3 = 6 is the number of unknowns), but there are twice the number of corresponding feature points (for one corresponding feature point) Since there are equations for u and v of the frame coordinates), it becomes a simultaneous linear equation with an excess condition, and is obtained by the least square method as described above. Specifically, the position estimation unit 12 obtains an attitude (translation vector t and rotation matrix R) that minimizes the cost function E1 of the following equation (6). This is the attitude of the own machine in the SLAM coordinates obtained by the SLAM method (the position and orientation of the own machine represented by the translation vector t and the rotation matrix R). In this way, the position estimation unit 12 estimates the attitude of the own device (step S232).

  Since the current attitude (translation vector t and rotation matrix R) in the SLAM coordinates is obtained, the position estimation unit 12 obtains VO (visual odometry) by multiplying this by the scale S (step S233). The VO can be used as the position and orientation of the aircraft in the real environment.

  Next, the position estimating unit 12 captures a predetermined distance (for example, 1 m, for example, “reference” below) from the position of the own device when the immediately preceding key frame registered in the frame DB (an image having a key frame number of NKF) is captured. It is determined whether or not it is moving (step S234). If it is moving (step S234; Yes), the key frame counter NKF is incremented (step S235), and the current frame is set as a key frame. Registration in the frame DB (step S236). If the movement is less than the reference translational distance (step S234; No), the process returns to step S202.

  Here, the movement distance of the aircraft compared to the reference translation distance is the translation distance from the previous key frame to the current frame (the absolute value of the difference vector between the translation vectors of both frames (the square root of the sum of the squares of the elements)). It may be obtained from odometry or may be obtained from the VO described above. As described above, the contents to be registered in the frame DB are “key frame number”, “posture”, “all extracted 2D feature points”, “a point where 3D position is known as a Map point among all 2D feature points” "," Characteristics of the key frame itself ".

  Then, the position estimation unit 12 sets the key frame counter NKF in the key frame queue of the map creation thread to inform the map creation thread that a new key frame has occurred (step S237). Then, the process returns to step S202. Note that the key frame counter NKF, scale S, Map point DB, frame DB, estimated state variable, and VO (visual odometry) are stored in the storage unit 20 so that values can be referred across threads.

  Next, the map creation thread activated in step S103 of the main flow (FIG. 4) of the autonomous mobile device 100 will be described with reference to FIG. In this thread, the map creating unit 11 calculates Map point information obtained by calculating the 3D position of the corresponding feature point in the key frame, information on the distance to the wall or obstacle acquired using the distance sensor 31, The map information is created from and stored in the map storage unit 22. Then, the communication quality map creation unit 14 creates a communication quality map using the communication quality information acquired from the communication quality measurement unit 13 and stores it in the communication quality map storage unit 23.

  First, the map creation unit 11 determines whether the operation is finished (step S301). If the operation ends (step S301; Yes), the operation ends. If the operation does not end (step S301; No), it is determined whether the key frame queue is empty (step S302). If the key frame queue is empty (step S302; Yes), the process returns to step S301. If the key frame queue is not empty (step S302; No), data is extracted from the key frame queue and is processed by the MKF (key frame processed by the map creation thread). (A variable representing a number) (step S303). The map creation unit 11 determines whether MKF is greater than 0 (step S304). If MKF is 0 (step S304; No), the map creation unit 11 returns to step S301 and waits for data to enter the key frame queue. When MKF is 1 or more (step S304; Yes), the process proceeds to the following process.

  The map creation unit 11 refers to the frame DB, determines the 2D feature point of the previous key frame (key frame number MKF-1) and the 2D feature point of the current key frame (key frame number MKF). 2D feature points (corresponding feature points) that can be handled in step S305 are extracted. Since the posture (translation vector t and rotation matrix R) of each key frame is also registered in the frame DB, the 3D position of the corresponding feature point is processed in the same manner as in the process at the initialization of the own position estimation thread. Can be calculated. The map creation unit 11 registers the corresponding feature point for which the 3D position can be calculated as a Map point in the Map point DB (step S306). The map creation unit 11 registers the 3D position with respect to the 2D feature points for which the current 3D position can be calculated for other key frames in the frame DB (step S307).

  If the corresponding feature point extracted by the map creation unit 11 has already been registered in the Map point DB, the 3D position calculation is skipped and processing for the next corresponding feature point (one not registered in the Map point DB) is performed. Alternatively, the 3D position may be calculated again, and the 3D position registered in the Map point DB or the 3D position for the corresponding feature point in the frame DB may be updated.

  Next, the map creation unit 11 determines whether or not the estimated state variable of the own device position estimation thread is “tracking success” (step S308). If it is not “tracking success” (step S308; No), it means that the position of the own device has not been estimated, so the measurement of an obstacle or communication quality is skipped and the process proceeds to step S311.

  If the estimated state variable is “successful tracking” (step S308; Yes), the map creation unit 11 measures the distance to the wall or the obstacle with the distance sensor 31, and determines the position and orientation of the own aircraft indicated by VO. Based on the measured distance, the position of the wall or obstacle, or the distance from the own device position to the wall or obstacle is registered in the map storage unit 22 (step S309).

  Then, the communication quality map creating unit 14 measures the communication quality by the communication quality measuring unit 13, and uses the communication quality value obtained by the measurement as the communication quality value at the own device position indicated by VO. Registration in the quality map storage unit 23 (step S310).

  The communication quality can be acquired by using the electric field intensity and the received power that can be observed by the communication unit 44, the RTT (Round-Trip Time) of the actual communication, the effective speed, the error rate, and the delay. An evaluation value obtained by combining these various evaluation values may be used. Further, when using an infrared or visible light communication link, binary information such as whether or not a signal has arrived (whether or not there is an obstacle between them) may be used. If the communication quality has already been measured at that position, the average value or median of the measurement results so far may be adopted, or the worst value may be adopted. Further, the average value, median, best value, worst value, etc. may be stored separately and used depending on the purpose of movement of the autonomous mobile device 100.

  The value to be registered in the communication quality map storage unit 23 is registered after being converted so as to have a smaller positive number (cost value) as the communication quality is better. For example, since the electric field strength and effective speed increase as the communication quality increases, the reciprocal is registered.

  Next, the map creating unit 11 determines whether or not the key frame queue is empty (step S311). If it is empty (step S311; Yes), bundle adjustment processing is performed on the orientation of all key frames and the 3D positions of all Map points to improve accuracy (step S312). If a Map point with a large error is found as a result of the bundle adjustment process, it is deleted from the Map point DB (step S313), and the process proceeds to step S314. If the key frame queue is not empty (step S311; No), the process proceeds to step S314 without doing anything.

  In step S314, the map creation unit 11 stores the map / route cost map in the array S (x, y) based on the information stored in the map storage unit 22 (step S314). The map and route cost map is “S (x, y) = ∞” if the coordinate (x, y) is an immovable area (wall or obstacle), and “S (x, y) = This is an array in which values are stored such that the smaller the shortest distance from the coordinate (x, y) to the immovable region, the smaller the positive number. For example, when the coordinate (x, y) is not an immovable region, the reciprocal of the shortest distance from the coordinate (x, y) to the immovable region can be stored in S (x, y). The array S is stored in the map storage unit 22.

  A specific example of the map / route cost map array S (x, y) is shown in FIG. In FIG. 7, black areas indicate immovable areas (walls and obstacles), and S (x, y) = ∞ in this area. Moreover, the white area | region has shown the area | region where the shortest distance to an immovable area | region is large, and the value of S (x, y) in this area | region is small (for example, less than 1). And the gray area | region has shown the area | region where the shortest distance to an immovable area | region is small, and the value of S (x, y) in this area | region is large (for example, 1 or more).

  Next, the communication quality map creation unit 14 stores the communication quality cost map in the array C (x, y) based on the information stored in the communication quality map storage unit 23 (step S315). The communication quality cost map is an array in which positive numbers that are smaller as communication quality at coordinates (x, y) is better stored in C (x, y). For example, the electric field strength at the coordinates (x, y) and the reciprocal of the effective speed can be stored in C (x, y). The array C is stored in the communication quality map storage unit 23.

  A specific example of the communication quality cost map of the array C (x, y) is shown in FIG. An external communication device 200 (not shown) exists at the left end of FIG. In FIG. 8, a white area indicates an area with good communication quality, and C (x, y) in this area is small (for example, less than 1). The light gray area indicates an area with poor communication quality, and the value of C (x, y) in this area is relatively large (for example, 1 or more and less than 3). The dark gray area indicates an area where the communication quality is so bad that communication is impossible, and the value of C (x, y) in this area is very large (for example, 3 or more).

  Since the coordinates (x, y) through which the autonomous mobile device 100 passes are generally sparse, values of S (x, y) and C (x, y) are often set only sparsely. Therefore, interpolation and extrapolation are performed, and values are densely set for each coordinate (x, y).

  Next, the map creating unit 11 sets MKF in the key frame queue of the loop closing thread (step S316), and returns to step S301.

  The map / route cost map array S (x, y) and the communication quality cost map array C (x, y) created by the above processing are stored so that the values can be referred to across the threads. Stored in the unit 20.

  The bundle adjustment process is a nonlinear optimization method that simultaneously estimates the camera posture (key frame posture) and the 3D position of the Map point, and an error that occurs when the Map point is projected onto the key frame. Optimization is performed so as to minimize it.

  By performing the bundle adjustment process, it is possible to improve the accuracy of the 3D position of the key frame posture and the Map point. However, this processing has a large amount of calculation, and a special problem does not occur because the accuracy cannot be improved without performing this processing. Accordingly, here, the bundle adjustment process is performed only when the key frame is empty (when the other processes are relatively light). When the communication quality is good, the autonomous mobile device 100 itself does not perform bundle adjustment processing, but transmits the information in the image storage unit 21 and the map storage unit 22 to the external communication device 200 through the communication unit 44, The result of the bundle adjustment process performed by the communication device 200 may be sent back to the autonomous mobile device 100.

  Further, when bundle adjustment processing is performed, a Map point may be found in which an error when projected onto a key frame is larger than a predetermined value. Since such a Map point with a large error has an adverse effect on SLAM estimation, it is deleted from the Map point DB in step S313. Instead of deleting from the Map point DB, a flag may be set in the Map point DB for identifying the Map point that requires a large error and requires attention.

  Next, the loop closing thread activated in step S104 of the main flow (FIG. 4) of the autonomous mobile device 100 will be described with reference to FIG. In this thread, the control unit 10 continues to check whether the loop closing process can be performed, and performs the loop closing process when the loop closing process can be performed. Note that the loop closing process uses the difference between the posture value and the current posture value when you were in the same location before when you recognized that you have returned to the same location that you have been before. This refers to correcting the 3D position of the key frame and the related Map point in the trajectory from the previous time to the present.

  First, the control unit 10 determines whether the operation is finished (step S401). If the operation ends (step S401; Yes), the process ends. If the operation is not finished (step S401; No), it is determined whether or not the key frame queue is empty (step S402). If the key frame queue is empty (step S402; Yes), the process returns to step S401. If the key frame queue is not empty (step S402; No), data is extracted from the key frame queue and processed by the LKF (loop closing thread). It is set to a variable representing a key frame number (step S403). Next, the control unit 10 determines whether LKF is greater than 1 (step S404). If LKF is 0 or 1 (step S404; No), the process returns to step S401 and waits for data to enter the key frame queue. When LKF is 2 or more (step S404; Yes), the following processing is performed.

  The control unit 10 refers to the frame DB, and the similarity between the current key frame (key frame whose key frame number is LKF) and “characteristic of the key frame itself” is a predetermined similarity (for example, 0.9, hereinafter “reference image”). (Referred to as “similarity”). The key DB is searched from the frame DB (step S405). Here, when the features of the images (key frames) are represented by feature vectors, this similarity is obtained by normalizing the absolute values of the feature vectors of the two images (the square root of the sum of squares of the elements) to 1. Can be the similarity between the two images. Also, the reciprocal of the distance (square root of the sum of squares of the differences between the elements) of the feature vectors of the two images (the absolute value normalized to 1) may be used as the similarity.

  The control unit 10 determines whether or not a key frame having a similarity of “characteristic of the key frame itself” equal to or higher than the reference image similarity is found (step S406), and if not found (step S406; No), step S401. Returning to step S406 (Yes in step S406), the posture of the key frame in the locus from the found key frame to the current key frame and the 3D position of the Map point included in the key frame in the locus are corrected ( Step S407). For example, the control unit 10 corrects the posture of the current key frame as the same posture as the found key frame. Then, using the difference between the attitude of the found key frame and the attitude of the current key frame, linear correction is applied to the attitude of each key frame in the trajectory from the found key frame to the current key frame. Further, the 3D position of the Map point included in each key frame is also corrected according to the correction amount of the posture of each key frame. Then, the process returns to step S401.

  Since the above loop closing process has a large amount of calculation, when the communication quality is good, the autonomous mobile device 100 itself does not perform the loop closing process, but the information in the image storage unit 21 and the map storage unit 22 is transmitted to the communication unit 44. May be transmitted to the external communication device 200, and the result of the loop closing process performed by the external communication device 200 may be sent back to the autonomous mobile device 100.

  Next, the route plan to the destination which is the process of step S111 of the main flow (FIG. 4) of the autonomous mobile device 100 will be described. The conventional route plan is to select the minimum accumulated cost route from the current position to the destination position given the array S (x, y), which is a map and route cost map, the current position and the destination position. Specifically, if each node is a coordinate, the adjacent relationship is an edge, and a node or edge is weighted according to the array S (x, y), a shortest path search of graph theory such as the Dijkstra algorithm can be performed. An algorithm can be used to efficiently select the minimum accumulated cost path. In the present invention, not only the array S (x, y) but also the array C (x, y), which is a communication quality map, can be used to select a route that maintains the communication quality.

(First embodiment)
The first embodiment of the route plan to the destination will be described with reference to FIG. First, the route search unit 15 sets the threshold value to a value when communication quality is considered good (step S501). Next, x and y are initialized to 0 in order to update the values for all coordinates of each array (step S502).

  Next, the route search unit 15 substitutes the value of (x, y) in the array S that is a map and route cost map into the array T (x, y) that is a route search cost map (step S503). The array T is an array used instead of the conventional map and route cost map S when searching for a route, and is stored in the route search map storage unit 24.

  Next, the route search unit 15 determines whether or not the value of (x, y) in the array C, which is a communication quality map, is larger than a threshold value (step S504). If it is larger than the threshold (step S504; Yes), it means that the communication quality at the coordinates (x, y) is not good, so that T (x, y) is set to ∞ in order not to pass through the place. Is set (step S505), and the process proceeds to step S506. If it is below the threshold (step S504; No), the communication quality at that location (x, y) is good, so the value of T (x, y) remains unchanged at S (x, y). The process proceeds to step S506.

  In step S506, the route search unit 15 determines whether or not the processing in steps S503 to S505 has been completed for all x and y (step S506). If not completed (step S506; No), the process returns to step S503.

  When the processing of step S503 to step S505 is completed for all x and y (step S506; Yes), the setting of the array T (x, y) that is the route search cost map is completed. The unit 15 performs a route search by applying a conventional route planning algorithm to T (step S507).

  Then, the route search unit 15 determines whether or not a route has been found in step S507 (step S508). If a route is found (step S508; Yes), the process ends. When the route does not exist (step S508; No), the route search unit 15 increases the threshold (step S509), returns to step S502, and repeats the retry.

  In step S509, when the threshold value becomes so large that the communication quality indicates “communication impossible”, the route search is given up and the process is terminated. Although the flowchart is complicated, it is not shown in the figure, but in this case, it is necessary to redo the setting of the destination in step S110 of the main flow (FIG. 4).

  Note that before this processing is performed, an area with poor communication quality (light gray or dark gray area in FIG. 8) indicated by the array C (x, y), which is a communication quality map, may be expanded or contracted. Good. In order to perform communication more reliably, an area where the communication quality is bad is expanded. In addition, when it is desired to allow a wide range of motion while allowing instantaneous interruption of communication during movement, an area with poor communication quality is contracted.

  A specific example of the route search will be described with reference to FIGS. For example, when the route from the current position to the target position shown in FIG. 11 is searched by the conventional method using the map / route cost map S (x, y) shown in FIG. 7, the arrow in FIG. The route to be shown is selected. On the other hand, when the communication quality cost map C (x, y) shown in FIG. 8 is applied to the first embodiment described above, the communication quality is bad except for the white portion in FIG. A route search cost map T (x, y) is obtained. If a route from the current position to the target position is searched in this figure, the route indicated by the dotted arrow in FIG. 11B is selected. In this way, by using the route search cost map in consideration of communication quality, it is possible to move to a destination by following a route in which communication is not disconnected.

(Second Embodiment)
Next, a description will be given of a second embodiment that allows a state in which communication cannot be performed for a certain period of time (allowable latency). For example, suppose that the autonomous mobile device 100 performs a natural language conversation with a human as a task to be executed, and causes the external communication device 200 to perform a speech recognition process necessary for performing the natural language conversation. The allowable latency is the time allowed to extend the silence period during the natural language conversation. For example, in this case, the latency can be set to 2 seconds. The amount of data communicated with the external communication device 200 varies depending on the content of the task executed by the autonomous mobile device 100. The communication quality that satisfies the bandwidth necessary for communication of this data amount is set as a threshold value.

  The second embodiment will be described with reference to FIG. First, the route search unit 15 sets the allowable latency to the variable TL based on the content of the task executed by the autonomous mobile device 100 (step S601). In this step, the route search unit 15 corresponds to an allowable latency setting unit. Next, the route search unit 15 sets a communication quality threshold based on the content of the task executed by the autonomous mobile device 100 (step S602). Then, the route search unit 15 substitutes the value of (x, y) in the array S that is a map / route cost map into the array T (x, y) that is a route search cost map (step S603).

  Next, the route search unit 15 performs route search by applying a conventional route planning algorithm to the route search cost map T, and sets the obtained route as a temporary route L (step S604). In this process, the route search unit 15 corresponds to a route candidate search unit. If no route is obtained here, the route search is given up and the process is terminated. Although the flowchart is complicated, it is not shown in the figure, but in this case, it is necessary to redo the setting of the destination in step S110 of the main flow (FIG. 4).

  Next, the route search unit 15 traces the temporary route L from the current position to the target position in order, and a section satisfying C (x, y)> threshold (a section where communication quality is insufficient when following the temporary route L). With respect to the length, a time required to pass through the length (hereinafter referred to as TT) is calculated, and it is determined whether or not TL <TT (step S605). The calculation of TT can be simply performed by adding a value obtained by dividing the length of one route by the moving speed of the autonomous mobile device 100 each time one route of the temporary route L is traced. In step S605, the route search unit 15 corresponds to a communication failure time acquisition unit.

  When the temporary route L is reached from the current position to the target position without a section where TL <TT (step 605; No), the temporary route L is adopted as the main route (step S607), and the process ends. .

  If a section where TL <TT is found when following the temporary route L (step 605; Yes), the temporary route L cannot be adopted, so the temporary route L is determined as TL <TT from the current position. ∞ is substituted for T (x, y) around the point on the section traced to a certain point (a region obtained by expanding the section by a predetermined margin width) (step S606), and the process returns to step S604. By substituting ∞ for T (x, y) in the peripheral region of the section, the section is not selected as a route in the next and subsequent route searches.

  A specific example of the route search in the second embodiment will be described with reference to FIG. 7, FIG. 8, FIG. 13, and FIG. For example, if the route planning process shown in FIG. 12 is performed using the map / route cost map S (x, y) shown in FIG. 7 and the communication quality cost map shown in FIG. A temporary route L indicated by an arrow 13 (a) is obtained. In step S605, if the time TT passing through the light gray portion of FIG. 13B along the arrow is equal to or shorter than the allowable latency TL, the temporary route L is adopted as the main route.

  If the time TT exceeds the allowable latency TL in the middle of passing the light gray portion of FIG. 13B along the arrow, in step S606, as shown in FIG. The area around T (x, y) up to the middle of the arrow (until the position where TT exceeds TL) is set to ∞ (black in the figure). Then, the process returns to step S604, and a new temporary route L is obtained again. This new temporary route L is represented by a dotted arrow in FIG.

  Even if the route of the dotted line arrow is traced, the time TT exceeds the allowable latency TL on the way, so that in step S606, as shown in FIG. The area around T (x, y) of TT (to a position where TT exceeds TL) is set to ∞ (black in the figure). Then, the process returns to step S604, and a new temporary route L is obtained again. This new temporary route L is represented by a dotted arrow in FIG.

  Since the route indicated by the dotted line passes only through a portion having good communication quality, this route is adopted as the main route.

  As described above, by performing the processing as shown in FIG. 12, even if there is an area with poor communication quality, if the time for passing therethrough is less than the allowable latency, the route passing there can be traced. When moving to the destination has to pass through an area with poor communication quality, it is impossible to move to the destination in the first embodiment, but in the second embodiment, the time is less than the allowable latency. If the area can be passed, the destination can be moved.

  For example, if voice data that is the basis of voice recognition is buffered while communication is interrupted, and communication quality is restored, it can be sent to an external device for processing. Processing can continue. Further, when the task of the autonomous mobile device 100 is a natural language conversation, if a time (for example, 2 seconds) in which the extension of the silence period is allowed in the natural language conversation is set as an allowable latency, it can pass through an area with poor communication quality. Natural language conversation can be continued even if it is unavoidable.

  In addition, when the speed of the autonomous mobile device 100 can be increased from the current speed, even if TT <TL in step S605, the process does not immediately proceed to step S606 but moves at the maximum speed in the environment. If TT is recalculated and if the recalculated TT is less than or equal to TL (step S605; No), the route search unit 15 sets the temporary route L as the main route (step S607), and the control unit 10 drives the drive unit 42. Alternatively, the maximum speed may be set to instruct movement. In this process, the control unit 10 corresponds to a speed setting unit. By performing such processing, even if it is not possible to pass through a region with poor communication quality in a time equal to or less than the allowable latency at a normal speed, the moving speed can be increased to allow the passage. Therefore, even in a situation where the destination cannot be reached at the normal speed, the destination can be reached while keeping the allowable latency by increasing the speed.

  In the above-described embodiment, the communication quality is considered only when the route plan to the destination is performed. However, the control unit 10 also determines the communication quality during free autonomous movement in step S109 of the main flow (FIG. 4). With reference to the cost map C (x, y), movement control may be performed so as not to stay for a long time where communication quality is poor. By doing in this way, it is possible to maintain good communication with an external communication device regardless of the position of the destination and whether or not the destination is set.

  It should be noted that each function of the autonomous mobile device 100 of the present invention can also be implemented by a computer such as a normal PC (Personal Computer). Specifically, in the above embodiment, the autonomous movement control processing program performed by the autonomous mobile device 100 has been described as being stored in advance in the ROM of the storage unit 20. However, the program is stored and distributed on a computer-readable recording medium such as a flexible disk, CD-ROM (Compact Disc Read Only Memory), DVD (Digital Versatile Disc), and MO (Magneto-Optical Disc). A computer capable of realizing each of the functions described above may be configured by reading and installing the program on a computer.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to the specific embodiment which concerns, This invention includes the invention described in the claim, and its equivalent range It is. Hereinafter, the invention described in the scope of claims of the present application will be appended.

(Appendix 1)
A map creation unit that creates a map using information of a plurality of images taken by the imaging unit;
A position estimation unit that estimates the position of the device using information of a plurality of images captured by the imaging unit;
A communication quality measurement unit for measuring communication quality when communicating with an external communication device;
A communication quality map creating unit that creates a communication quality map using the own device position estimated by the position estimating unit and the communication quality measured by the communication quality measuring unit;
A route search unit that searches for a route to a destination using the communication quality map created by the communication quality map creation unit;
An autonomous mobile device comprising:

(Appendix 2)
The route search unit searches for a route that does not pass through a communication failure region that is a region where the communication quality is lower than a predetermined threshold in the communication quality map created by the communication quality map creation unit.
The autonomous mobile device according to attachment 1.

(Appendix 3)
Furthermore, a communication failure time acquisition unit that acquires a communication failure time that is a time required to pass through a communication failure region that is a region where communication quality is lower than a predetermined threshold in the route searched by the route search unit,
The route search unit searches for a route that does not pass through the communication failure area when the communication failure time acquired by the communication failure time acquisition unit exceeds the allowable latency, and when the communication failure time is equal to or less than the allowable latency. Search for a route through the poor communication area,
The autonomous mobile device according to attachment 1.

(Appendix 4)
In addition, an allowable latency setting unit that sets the allowable latency based on the task content of the own machine,
The autonomous mobile device according to attachment 3.

(Appendix 5)
In addition, it has a speed setting unit that sets the speed of its own machine,
The communication failure time acquisition unit acquires the communication failure time when the autonomous mobile device moves at the maximum speed,
When the communication failure time when the autonomous mobile device moves at the maximum speed is equal to or less than the allowable latency, the route search unit searches for a route passing through the communication failure region, and the speed setting unit Set the speed to the maximum speed,
The autonomous mobile device according to appendix 3 or 4.

(Appendix 6)
The route search unit sets a route that avoids a communication failure area in which the communication quality is lower than a predetermined threshold in the communication quality map created by the communication quality map creation unit even when the destination is not set. To
The autonomous mobile device according to attachment 1.

(Appendix 7)
The autonomous mobile device according to any one of appendices 1 to 6,
An external communicator communicating with the autonomous mobile device;
An autonomous mobile system comprising:

(Appendix 8)
An autonomous movement method for autonomously moving while communicating with an external communication device,
Search for a route according to the communication quality when communicating with the communication device,
Autonomous movement method.

(Appendix 9)
Search for a route that does not pass through a communication failure area that is an area where the communication quality when communicating with the communication device is lower than a predetermined threshold;
The autonomous movement method according to attachment 8.

(Appendix 10)
Searching for a route through which a communication quality at the time of communicating with the communication device is lower than a predetermined threshold and only passes a time equal to or less than the allowable latency,
The autonomous movement method according to attachment 8.

(Appendix 11)
Even when a destination is not set, a route that avoids a communication failure area that is an area where the communication quality when communicating with the communication device is lower than a predetermined threshold is set.
The autonomous movement method according to attachment 8.

(Appendix 12)
A map creation step of creating a map using information of a plurality of images taken by the imaging unit;
A position estimation step of estimating the position of the device using information of a plurality of images captured by the imaging unit;
A communication quality measurement step for measuring communication quality when communicating with an external communication device;
A communication quality map creating step of creating a communication quality map using the own device position estimated in the position estimating step and the communication quality measured in the communication quality measuring step;
A route search step for searching for a route to the destination using the communication quality map created in the communication quality map creation step;
An autonomous movement method comprising:

(Appendix 13)
On the computer,
A map creation step of creating a map using information of a plurality of images taken by the imaging unit;
A position estimation step of estimating the position of the own device using information of a plurality of images captured by the imaging unit;
Communication quality measurement step for measuring communication quality when communicating with an external communication device,
A communication quality map creating step for creating a communication quality map using the own device position estimated in the position estimating step and the communication quality measured in the communication quality measuring step;
A route search step for searching for a route to a destination using the communication quality map created in the communication quality map creation step,
A program for running

DESCRIPTION OF SYMBOLS 100 ... Autonomous mobile device, 10 ... Control part, 11 ... Map preparation part, 12 ... Position estimation part, 13 ... Communication quality measurement part, 14 ... Communication quality map preparation part, 15 ... Route search part, 20 ... Memory | storage part, 21 ... Image storage unit, 22 ... Map storage unit, 23 ... Communication quality map storage unit, 24 ... Route search map storage unit, 30 ... Sensor unit, 31 ... Distance sensor, 41 ... Imaging unit, 42 ... Drive unit, 43 ... Input unit 44 ... communication unit 45 ... power source 200 ... communication device

Claims (13)

A map creation unit that creates a map using information of a plurality of images taken by the imaging unit;
A position estimation unit that estimates the position of the device using information of a plurality of images captured by the imaging unit;
A communication quality measurement unit for measuring communication quality when communicating with an external communication device;
A communication quality map creating unit that creates a communication quality map using the own device position estimated by the position estimating unit and the communication quality measured by the communication quality measuring unit;
A route search unit that searches for a route to a destination using the communication quality map created by the communication quality map creation unit;
An autonomous mobile device comprising: The route search unit searches for a route that does not pass through a communication failure region that is a region where the communication quality is lower than a predetermined threshold in the communication quality map created by the communication quality map creation unit.
The autonomous mobile device according to claim 1. Furthermore, a communication failure time acquisition unit that acquires a communication failure time that is a time required to pass through a communication failure region that is a region where communication quality is lower than a predetermined threshold in the route searched by the route search unit,
The route search unit searches for a route that does not pass through the communication failure area when the communication failure time acquired by the communication failure time acquisition unit exceeds the allowable latency, and when the communication failure time is equal to or less than the allowable latency. Search for a route through the poor communication area,
The autonomous mobile device according to claim 1. In addition, an allowable latency setting unit that sets the allowable latency based on the task content of the own machine,
The autonomous mobile device according to claim 3. In addition, it has a speed setting unit that sets the speed of its own machine,
The communication failure time acquisition unit acquires the communication failure time when the autonomous mobile device moves at the maximum speed,
When the communication failure time when the autonomous mobile device moves at the maximum speed is equal to or less than the allowable latency, the route search unit searches for a route passing through the communication failure region, and the speed setting unit Set the speed to the maximum speed,
The autonomous mobile device according to claim 3 or 4. The route search unit sets a route that avoids a communication failure area in which the communication quality is lower than a predetermined threshold in the communication quality map created by the communication quality map creation unit even when the destination is not set. To
The autonomous mobile device according to claim 1. The autonomous mobile device according to any one of claims 1 to 6,
An external communicator communicating with the autonomous mobile device;
An autonomous mobile system comprising: An autonomous movement method for autonomously moving while communicating with an external communication device,
Search for a route according to the communication quality when communicating with the communication device,
Autonomous movement method. Search for a route that does not pass through a communication failure area that is an area where the communication quality when communicating with the communication device is lower than a predetermined threshold;
The autonomous movement method according to claim 8. Searching for a route through which a communication quality at the time of communicating with the communication device is lower than a predetermined threshold and only passes a time equal to or less than the allowable latency,
The autonomous movement method according to claim 8. Even when a destination is not set, a route that avoids a communication failure area that is an area where the communication quality when communicating with the communication device is lower than a predetermined threshold is set.
The autonomous movement method according to claim 8. A map creation step of creating a map using information of a plurality of images taken by the imaging unit;
A position estimation step of estimating the position of the device using information of a plurality of images captured by the imaging unit;
A communication quality measurement step for measuring communication quality when communicating with an external communication device;
A communication quality map creating step of creating a communication quality map using the own device position estimated in the position estimating step and the communication quality measured in the communication quality measuring step;
A route search step for searching for a route to the destination using the communication quality map created in the communication quality map creation step;
An autonomous movement method comprising: On the computer,
A map creation step of creating a map using information of a plurality of images taken by the imaging unit;
A position estimation step of estimating the position of the own device using information of a plurality of images captured by the imaging unit;
Communication quality measurement step for measuring communication quality when communicating with an external communication device,
A communication quality map creating step for creating a communication quality map using the own device position estimated in the position estimating step and the communication quality measured in the communication quality measuring step;
A route search step for searching for a route to a destination using the communication quality map created in the communication quality map creation step,
A program for running

Images