JP6733247B2 - Unmanned Flight Device Control System, Unmanned Flight Device Control Method, and Image Projection Device - Google Patents

Description

The present invention relates to an unmanned flight device control system for controlling an unmanned flight device, an unmanned flight device control method, and an image projection device applicable to the unmanned flight device control system.

State inspection of buildings (infrastructure deterioration diagnosis) using unmanned air vehicles is being conducted. For example, the deterioration state of a building can be estimated from the surface state of the building taken by a camera mounted on an unmanned aerial vehicle. Hereinafter, the unmanned aerial vehicle will be referred to as a UAV (Unmanned Aerial Vehicle). The UAV can also be referred to as an unmanned aerial vehicle.

In order to use the UAV for the infrastructure deterioration diagnosis, control for guiding the UAV to the vicinity of the building to be diagnosed is necessary.

In a general technique, the control unit of the UAV grasps the position of the UAV using a GPS (Global Positioning System) signal in order to guide the UAV to the target position. Patent Literature 1 describes an aerial photographic device including a GPS device having a GPS antenna and a camera including an imaging unit.

JP, 2014-62789, A

When performing infrastructure deterioration diagnosis using a UAV whose position is determined by GPS signals, the following problems occur. In a GPS in which a single antenna is attached to a moving body such as a UAV, the real-time positioning accuracy includes an error of several meters. As a result, when the UAV approaches the building subject to the infrastructure deterioration diagnosis, the GPS measurement position error of the UAV is large, and thus the distance between the building and the UAV may be too far or too close. Further, depending on the positional relationship between the UAV and the GPS satellites, the UAV may pass through a place where the GPS signal strength is weak. When the UAV passes through a place where the GPS signal strength is weak, the UAV cannot accurately grasp the position of the UAV, and the guidance accuracy to the target position becomes poor.

In order to solve this problem with a simple configuration, an image for controlling the operation of the UAV is projected on an object such as a building, and the projected image is captured by a camera mounted on the UAV. It is possible to guide the UAV near the target position near the object.

In the UAV guidance control based on such an image, depending on the environment in which the image is projected such as the color of the object onto which the image is projected and the ambient brightness at the time of image projection, the image captured by the camera mounted on the UAV May become unclear and image-based UAV control may not be performed correctly.

Note that such a problem may occur not only in UAV control in infrastructure deterioration diagnosis, but also in UAV control for guiding to a target position near an object such as patrol monitoring of a building.

Therefore, the present invention considers the environment in which the image is projected when an image for controlling the operation of the unmanned aerial vehicle is projected on the object and the unmanned aerial vehicle is guided near the target position near the object. (EN) An unmanned flight device control system capable of controlling an unmanned flight device, an unmanned flight device control method, and an image projection device applicable to the unmanned flight device control system.

The unmanned aerial vehicle control system according to the present invention projects an image for controlling the operation of an unmanned aerial vehicle equipped with an imaging device, the image being generated according to the reflectance of the projection surface of the object onto the object. An image projection unit and a control unit that controls the operation of the unmanned aerial vehicle based on the image captured by the imaging device, and the image projection unit controls the operation of the unmanned aerial vehicle based on the reflectance. The projection position of the image of is determined .

In addition, the unmanned aerial vehicle control method according to the present invention is an image for controlling the operation of an unmanned aerial vehicle equipped with an imaging device, which is an image generated according to the reflectance of the projection surface of the object. The projection position of the image for controlling the operation of the unmanned aerial vehicle based on the reflectance when projecting the image to the operation of the unmanned aerial vehicle based on the image captured by the imaging device. To control.

The image projection apparatus according to the present invention further includes an image generation unit that generates an image for controlling the operation of an unmanned aerial vehicle equipped with an imaging apparatus, according to the reflectance of the projection surface of an object, and the generated image. And a projection unit that projects the image, and the projection unit determines the projection position of the image for controlling the operation of the unmanned aerial vehicle based on the reflectance .

According to the present invention, when the image for controlling the operation of the unmanned aerial vehicle is projected on the object and the unmanned aerial vehicle is guided near the target position near the object, the environment for projecting the image is taken into consideration. Control unmanned aerial vehicle.

1 is a schematic diagram showing an unmanned flight device control system of the present invention and a building to be inspected. FIG. 7 is a schematic diagram showing a situation in which the UAV grasps the movement control content up to the next target position when the UAV shoots at a position deviated from the target position. It is explanatory drawing which shows the example of an information image. It is explanatory drawing which shows the meaning of the symbol in the information image illustrated in FIG. It is explanatory drawing which shows the example of the display mode of a bar. It is a block diagram which shows the structural example of UAV. It is a schematic diagram which shows the example of a visual field of a camera. It is a schematic diagram which shows the example of the camera image produced|generated by imaging the visual field shown in FIG.7(b). It is a schematic diagram which shows the example of the state which overlapped the outline in the camera image which outline information showed on the camera image shown in FIG. It is a schematic block diagram which shows the structural example of a projection station. It is a flowchart which shows the example of a process progress of the 1st Embodiment of this invention. It is a schematic diagram which shows the example of the camera image which cannot recognize an information image. It is a schematic diagram which shows the example of the camera image which the information image and the shadow of UAV overlap and were reflected. It is a schematic diagram which shows the example of the camera image in which the information image and the shadow of UAV did not overlap by moving UAV. It is a schematic diagram which shows the example of the path|route of UAV. It is a schematic diagram which shows the example of the situation where the projection surface of an information image and the target surface of the imaging for a state inspection differ. It is a schematic diagram which shows the example of the pattern image showing a horizontal straight line. It is a schematic diagram which shows the state which projects a pattern image on the surface of the building in which the level difference was produced, and the camera is imaging the projection location. FIG. 19 is a schematic side view of the state shown in FIG. 18. It is a schematic diagram which shows the example of the camera image obtained as a result of having imaged a pattern image. It is a schematic diagram which shows the outline of the unmanned flight device control system of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiment 1.
FIG. 1 is a schematic diagram showing an unmanned aerial vehicle control system of the present invention and a building to be inspected. The unmanned aerial vehicle control system of the present invention includes an unmanned aerial vehicle (hereinafter referred to as UAV) 10 and a projection station 20. As mentioned above, the UAV can also be referred to as an unmanned aerial vehicle.

The UAV 10 is equipped with a camera. The camera mounted on the UAV 10 images the surface of the building 30 to be inspected and generates an image of the surface of the building. With this image, the condition of the building can be inspected (infrastructure deterioration diagnosis).

The projection station 20 includes a projector 21, and the projector 21 projects an image including information indicating the control content of the UAV 10 (hereinafter referred to as an information image) onto the surface of the building 30. The projection station 20 is mounted on a vehicle 29 and can be moved or kept stopped at the same location.

The projection station 20 stores in advance information on the appearance and three-dimensional shape of the building 30 to be inspected. Further, the projection station 20 recognizes the positional relationship between the projection station 20 and the building 30. The positional relationship between the projection station 20 and the building 30 is measured using various surveying devices (not shown), and not only the position in the three-dimensional space but also the orientation is measured with high accuracy. The method of obtaining the three-dimensional shape of the building 30 may be a method using drawings at the time of design or 3D-CAD (3Dimension Computer-Aided Design) data, or 3D-LiDAR (3Dimension Light Detection and Ranging). Alternatively, a method of measuring the actual product in advance may be used. The information on the three-dimensional shape of the building 30 is not limited to the information on the entire building 30, but may be the information on the portion of the building 30 to be observed using the present invention. In addition, as for the method of accurately knowing the positional relationship between the building 30 and the projection station 20, there are many known methods such as triangulation using a total station, and therefore description thereof will be omitted. Basically, the positional relationship between the projection station 20 and the building 30 is measured in advance. However, when there is a possibility that the building 30 is deformed during work due to the influence of wind or the like, or when the projection station 20 is in an unstable position such as on a ship, the positional relationship is measured during work according to the present invention. You may.

The projection station 20 determines a plurality of projection locations of the information image on the surface of the building 30 based on the positional relationship and the stored information, and derives the path 35 of the UAV 10. The projection station 20 also generates an information image to be projected on each of the projection locations. Further, the projection station 20 determines a position (hereinafter, referred to as a target position) for the UAV 10 to capture an information image, an attitude (tilt or direction) at the target position, and a camera direction. The posture and the orientation of the camera are the orientation and the orientation of the camera when the information image is captured. The projection station 20 generates, for example, the content of movement control to the next target position, the information image for instructing the attitude during movement, and the orientation of the camera based on the positional relationship between the target positions. The information image also includes information for instructing the UAV 10 to take an image for a state inspection. The projection station 20 sequentially projects the information images on the surface of the building 30. As will be described later, in general, the UAV 10 generally captures an information image at a position deviated from the target position, and the orientation when capturing the information image and the orientation of the camera are also determined. It doesn't have to be.

The UAV 10 takes a picture of the surface of the building 30 onto which the information image is projected by using a camera. Hereinafter, the image obtained by the image pickup by the camera of the UAV 10 is referred to as a camera image. The UAV 10 operates according to the control content indicated by the information image in the camera image (the information image captured in the camera image). For example, the UAV 10 takes an image for a state inspection or moves in the direction of the next target position based on the information image in the camera image.

The UAV 10 operates based on the information image in the camera image to move in the vicinity of the path 35 and image the surface of the building 30 for condition inspection.

The UAV 10 does not always capture the information image at the target position on the route 35. Since the UAV 10 is affected by various factors such as wind, it is generally difficult for the UAV 10 to reach the same position as the predetermined target position. Therefore, the UAV 10 images the surface of the building 30 when the vicinity of the target position is reached. The UAV 10 calculates the deviation between the position and the target position based on the camera image obtained as a result. Then, the UAV 10 derives the control content for moving from the current position to the next target position based on the movement control content indicated by the information image in the camera image and the calculated shift.

FIG. 2 is a schematic diagram showing a situation in which the UAV 10 grasps the movement control content up to the next target position when the UAV 10 shoots at a position deviated from the target position. In the example shown in FIG. 2, the areas A1 and A2 on the surface of the building 30 are projected portions of the information image. Further, it is assumed that the target position P1 is set as a position for capturing the information image projected on the area A1. Similarly, it is assumed that the target position P2 is defined as a position for capturing the information image projected on the area A2. Further, the attitude of the UAV 10 and the orientation of the camera at each of the target positions P1 and P2 are also defined. As described above, since the UAV 10 is affected by various factors such as wind, it is difficult for the UAV 10 to capture the information image projected on the area A1 at a position that completely matches the target position P1. Therefore, the UAV 10 captures the information image projected on the area A1 at the position P1' near the target position P1. At this time, the attitude of the UAV 10 and the orientation of the camera generally do not match the attitude and the orientation of the camera at the predetermined target position P1.

The UAV 10 has the contour of the information image in the camera image when it is assumed that the information image projected in the area A1 is captured at the target position P1 with the determined posture and camera orientation, and actually, the position P1′. The deviation M (see FIG. 2) between the target position P1 and the position P1′ is calculated based on the contour of the information image in the camera image obtained by capturing the information image in. Further, the information image projected on the area A1 shows the movement control content for moving from the target position P1 to the target position P2. The UAV 10 derives the control content for moving from the position P1′ to the target position P2 based on the movement control content and the calculated displacement M, and directs the control content to the next target position P2 based on the control content. To move. When capturing the information image projected on the area A2, the UAV 10 captures the information image in the vicinity of the target position P2, and similarly derives the control content for moving to the next target position.

The contour information of the information image in the camera image when it is assumed that the information image is captured at the predetermined target position and the predetermined orientation and camera orientation is given to the UAV 10 in advance.

Further, the UAV 10 is generally unable to capture an information image at the front position of the information image due to various influences such as wind. Therefore, the information image is generally distorted in the camera image. The UAV 10 recognizes the control content indicated by the information image shown in the camera image after performing the processing for removing the distortion of the information image.

FIG. 3 is an explanatory diagram showing an example of the information image projected by the projection station 20. FIG. 4 is an explanatory diagram showing the meaning of symbols in the information image illustrated in FIG.

The information image 40 includes, for example, a movement control symbol 41, a camera orientation control symbol 43, and a posture control symbol 44.

The movement control symbol 41 is a symbol indicating the control content regarding the movement of the UAV 10. By the movement control symbol 41, it is possible to give an instruction of a traveling direction, a flight instruction, and a speed instruction. The movement control symbol 41 illustrated in FIG. 3 has an arrow shape, and the direction indicated by the arrow is the instructed traveling method. In the example shown in FIG. 3, the downward movement is instructed. The traveling direction indicated by the movement control symbol 41 is not limited to the downward direction.

Further, the movement control symbol 41 can instruct the UAV 10 to stay in the air by the added bar 42. The bar 42 means an instruction to stay in the air. When the bar 42 is provided at the end of the movement control symbol 41 on the opposite side of the arrowhead, it means that the bar 42 moves in the direction of the arrow (downward in this example) after staying in the air. Further, by displaying the bar 42 at the end portion of the movement control symbol 41 on the arrow head side, it is possible to instruct to move and then fly. In the following description, in order to simplify the description, the UAV 10 recognizes the control content indicated by the information image 40, and then stays in the air, performs image capture for a state inspection within the staying time, and is then instructed. It will be described as moving in the direction. In this case, the bar 42 is displayed at the end opposite to the arrow head. Further, the bar 42 will be described as having a meaning of instructing imaging for a state inspection while in flight.

In addition, the movement control symbol 41 indicates the movement speed when the UAV 10 moves, depending on how the shade of the color inside the movement control symbol 41 (arrow-shaped symbol) changes. Note that, in FIGS. 3 and 4, a change in the shade of the color of the arrow-shaped symbol is represented by a change in the pattern for convenience. The shade of the color in the actual arrow shape may change gradually, for example.

The direction of the bar 42 is set to be perpendicular to the direction of the arrow, for example. However, the method of determining the orientation of the bar 42 is not limited to this example, and may be always horizontal, for example. FIG. 5 is an explanatory diagram showing an example of the display mode of the bar 42. In FIG. 5, the display of shades of color inside the movement control symbol 41 is omitted. FIG. 5A shows an example in which the direction of the bar 42 is set to be perpendicular to the direction of the arrow. FIG. 5B shows an example in which the direction of the bar 42 is always set to be horizontal. When the arrow points to the right as illustrated in FIG. 5, the bar 42 is defined as a vertically oriented bar in the example illustrated in FIG. On the other hand, in the example shown in FIG. 5B, the bar 42 is horizontal and is shown inside the arrow.

The camera orientation control symbol 43 is a symbol representing the control content regarding the orientation of the camera mounted on the UAV 10. The camera orientation control symbol 43 controls the elevation and depression angles of the camera. FIG. 3 illustrates the case where the camera orientation control symbol 43 is a U-shaped rotation arrow. In this example, the U-shaped rotation arrow means that the elevation angle of the camera is directed upward and the horizontal angle is fixed. The correspondence between the control content regarding the orientation of the camera and the shape of the camera orientation control symbol 43 may be determined in advance.

In addition, when the direction of the camera during moving to the next target position and the direction of the camera at the time of shooting corresponding to the next target position are separately defined, the projection station 20 is set to the respective directions. A corresponding camera orientation control symbol may be included in the information image 40.

The attitude control symbol 44 is a symbol representing the control content regarding the attitude of the UAV 10. FIG. 3 illustrates the case where the posture control symbol 44 is a semicircular figure. In this example, the semicircular figure means that the posture of the UAV 10 is horizontally controlled. The relationship between the control content related to the attitude of the UAV 10 and the shape of the attitude control symbol 44 may be determined in advance.

Note that when the moving posture to the next target position and the posture at the time of shooting corresponding to the next target position are separately defined, the projection station 20 controls the posture control corresponding to each posture. The symbol may be included in the information image 40.

Although not shown in FIG. 3, information in the camera image when it is assumed that the information image is captured at a certain target position with a posture and a camera direction determined corresponding to the target position. Information representing the contour of the image may be included in the information image 40. Hereinafter, the information indicating the contour of the information image in the camera image when it is assumed that the information image is captured at a certain target position with the posture and the orientation of the camera determined corresponding to the target position will be referred to as contour information.

The movement control symbol 41, the bar 42, the camera orientation control symbol 43, and the attitude control symbol 44 are identification information for identifying the control content of the UAV 10, and the information image 40 is an image including such identification information. You can Further, FIG. 3 exemplifies a case where the identification information (movement control symbol 41, bar 42, camera orientation control symbol 43, and attitude control symbol 44) for identifying the control content is represented by a graphic symbol. When the identification information for identifying the control content is represented by a graphic symbol or the like as illustrated in FIG. 3, it is possible for the inspection operator to easily recognize it even from a distance.

Further, the identification information for identifying the control content of the UAV 10 may be a symbol such as a QR code (registered trademark) or a bar code.

Further, the identification information for identifying the control content of the UAV 10 may be symbols such as Chinese characters, numbers or hieroglyphs.

Further, the identification information for identifying the control content of the UAV 10 may be, for example, a sentence such as “1 meter flight below after staying for 30 seconds”.

Further, from the viewpoint of the visibility of the inspection operator, the projection station 20 may project the information image 40 by blinking the information (identification information for identifying the control content) in the information image 40, for example. Good. By projecting the information image 40 in this manner, the content of the information image 40 can be made conspicuous, and the inspection operator can easily recognize the content.

The projection station 20 may also include other information in the information image 40 in addition to the identification information for identifying the control content. For example, the projection station 20 may include state information (for example, the current altitude of the UAV 10 or a step number indicating the progress of work) in the information image 40.

The contour information described above is used when determining the deviation M between the target position P1 and the actual position P1'. The information necessary for obtaining the deviation M between the target position P1 and the actual position P1' does not have to be the contour information described above. The projection station 20 may include information necessary for obtaining the deviation M between the target position P1 and the actual position P1' in the information image 40, and the UAV 10 may store the information in advance. For example, the projection station 20 projects the information image 40 including special icons at the four corners instead of the contour information, and the UAV 10 obtains the shift M according to the positional relationship of those icons reflected in the camera image. Good. As described above, by using the shape that is easy to recognize without using the contour, it is possible to reduce the possibility that the UAV 10 misrecognizes the shift M.

Further, the projection station 20 may fixedly include in the information image 40 the information necessary for obtaining the deviation M between the target position P1 and the actual position P1′. For example, it is assumed that the information image 40 includes an inspection area number (not shown) indicating the inspection area. The projection station 20 may fixedly set the first half of the inspection area number by, for example, an alphabet, and arbitrarily set the second half. The UAV 10 may obtain the shift M by the deformation of the fixed portion shown in the camera image. As described above, a part of the information image 40 is fixedly determined, and the shift M is obtained based on the image of the portion in the camera image, so that the element in the information image 40 for obtaining the shift M is the inspection operator. Can be out of the way.

Next, a configuration example of the UAV 10 will be described. FIG. 6 is a block diagram showing a configuration example of the UAV 10. In this example, the case where the UAV 10 has four propellers (not shown in FIG. 6) and four motors that correspond to the four propellers in a one-to-one relationship will be described as an example. However, the number of propellers and motors is not limited to the above example.

The UAV 10 includes a camera 11, a control unit 13, a camera angle control unit 15, a motor driver 16, four motors 17a to 17d, and a communication unit 18.

The camera 11 images the surface of the building 30 (see FIG. 1) to be inspected for infrastructure deterioration diagnosis, and generates a camera image of the surface of the building. When the information image is projected in the field of view of the camera, the information image is also reflected in the camera image. The camera 11 inputs the generated camera image to the control unit 13. The orientation of the camera 11 can be controlled.

Here, the visual field of the camera on the surface of the building 30 (see FIG. 1) and the camera image when the visual field is photographed will be described.

FIG. 7 is a schematic diagram showing an example of the field of view of the camera on the surface of the building 30 (see FIG. 1). FIG. 7A is a schematic top view showing an example of a situation where the camera 11 of the UAV 10 (not shown in FIG. 7) images the surface 30a of the building 30. Moreover, the information image 40 shall be projected on the surface 30a of the building 30. Due to the influence of wind and the like, it is difficult for the camera 11 to perfectly face the surface 30a of the building 30 and image the surface 30a. Therefore, as shown in FIG. 7A, the camera 11 generally images the surface 30a of the building 30 from an oblique direction. FIG. 7B is a schematic diagram showing the field of view of the camera 11 on the surface 30a of the building 30 in the state shown in FIG. 7A. The field of view 49 of the camera 11 on the surface 30a is a quadrangle. In the example illustrated in FIG. 7B, the field of view 49 is trapezoidal because the camera 11 is in a state of capturing an image of the surface 30a of the building 30 from an oblique direction. The field of view 49 is not always trapezoidal. However, the field of view 49 becomes rectangular when the camera 11 is directly facing the surface 30a of the building 30, and as shown in FIG. 7A, the camera 11 is viewed diagonally from the surface 30a of the building 30. The field of view 49 does not have a rectangular shape when the image is captured. The projection station 20 also projects the information image 40 so that the information image 40 is rectangular on the surface 30a. Therefore, the information image 40 projected on the surface 30a is rectangular. The vertices of the visual field 49 are A, B, C and D. Further, the vertices of the information image 40 projected on the surface 30a of the building 30 are defined as a, b, c, and d. In FIG. 7B, various symbols in the information image 40 are omitted. This point is the same in FIG. 8 and the like.

The camera 11 captures a field of view 49 and generates a camera image. FIG. 8 is a schematic diagram showing an example of a camera image generated by capturing the visual field 49 shown in FIG. 7B. The camera image 50 has a rectangular shape. The individual vertices of the camera image 50 correspond to the individual vertices of the visual field 49 shown in FIG. 7B. The apex of the camera image 50 corresponding to the apex A of the visual field 49 is also represented by the symbol A. The same applies to the vertices of other camera images. Further, the individual vertices of the information image 40 (see FIG. 8) shown in the camera image 50 correspond to the individual vertices of the information image 40 (see FIG. 7(b)) projected on the surface 30a of the building 30. ing. The apex of the information image 40 (see FIG. 8) in the camera image 50 corresponding to the apex of the information image 40 shown in FIG. 7B is also represented by the symbol a. The same applies to the other vertices of the information image 40 in the camera image 50. Hereinafter, the contour of the information image 40 in the camera image 50 will be referred to as a contour abcd.

The camera 11 captures a non-rectangular field of view 49 and generates a rectangular camera image 50. Therefore, the rectangular information image 40 projected in the visual field 49 of the camera 11 is distorted in the camera image 50 as shown in FIG.

When the camera image 50 is input from the camera 11, the control unit 13 detects the contour abcd of the information image 40 in the camera image 50.

Further, contour information is given to the control unit 13 in advance. As described above, the contour information is the contour of the information image in the camera image when it is assumed that the information image is captured at a certain target position with the posture and camera orientation determined corresponding to the target position. This is information to represent. FIG. 9 is a schematic diagram showing an example of a state in which the contour in the camera image indicated by the contour information is superimposed on the camera image shown in FIG. The control unit 13 calculates the deviation M between the current position and the target position of the UAV 10 based on the contour 51 in the camera image 50 indicated by the contour information and the contour abcd of the information image 40 in the camera image 50. The deviation M includes not only the position but also the posture information. Since the same applies hereafter, the posture information will be omitted unless otherwise specified.

When the information image cannot be recognized in the camera image, the control unit 13 controls the motors 17a to 17d via the motor driver 16 to move the UAV 10.

When the information image can be recognized in the camera image, the control unit 13 calculates the deviation M between the current position of the UAV 10 and the target position. Further, the control unit 13 performs a process of removing the distortion of the information image 40 in the camera image 50 (in other words, a process of making the information image 40 in the camera image 50 rectangular), and the processed information image 40 becomes The control content shown is grasped and the UAV 10 is controlled according to the control content. For example, the control unit 13 performs imaging for the state inspection of the surface of the building 30 according to the control content indicated by the information image 40. Further, the control unit 13 calculates the deviation M (deviation between the current position of the UAV 10 and the target position), the movement control content to the next target position indicated by the information image 40, and the movement from the current position to the next target position. The UAV 10 is moved by deriving a control content for the control and controlling each of the motors 17a to 17d via the motor driver 16 according to the control content. Further, the control unit 13 controls the posture of the UAV 10 and controls the orientation of the camera 11 via the camera angle control unit 15 according to the control content indicated by the information image 40.

The camera angle control unit 15 controls the orientation of the camera 11 according to the control unit 13. Specifically, the camera angle control unit 15 controls the elevation/depression angle and the horizontal angle of the camera 11.

The motor driver 16 drives the motors 17a to 17d according to the control unit 13. The motor driver 16 can drive each of the motors 17a to 17d separately. Therefore, the control unit 13 can realize control of moving the UAV 10 back and forth, control of moving the UAV 10 left and right, control of moving the UAV 10 up and down, pitch angle control, roll angle control, and yaw angle control.

The communication unit 18 is a communication interface used for communication with the projection station 20.

The control unit 13 and the camera angle control unit 15 may be realized by, for example, a CPU of a computer that operates according to a program. In this case, the CPU may read the program and operate as the control unit 13 and the camera angle control unit 15 according to the program. Further, the control unit 13 and the camera angle control unit 15 may be realized by separate hardware.

Next, a configuration example of the projection station 20 will be described. FIG. 10 is a schematic block diagram showing a configuration example of the projection station 20. The projection station 20 includes a projector 21, a projector control unit 22, a data storage unit 23, a route derivation unit 24, an information image generation unit 25, and a communication unit 27.

The data storage unit 23 is a storage device that stores information about the appearance of a building and the three-dimensional shape of the building.

The route deriving unit 24 recognizes the positional relationship between the projection station 20 and the building. At this time, the route deriving unit 24 may be given the current position of the projection station 20 from the outside, for example. The route deriving unit 24 determines a plurality of projected portions of the information image on the surface of the building 30 based on the positional relationship and the information on the appearance of the building and the three-dimensional shape of the building. Further, the route deriving unit 24 derives the route of the UAV 10.

The route deriving unit 24 defines a plurality of inspection areas by partitioning the surface of the building into a lattice shape based on the information stored in the data storage unit 23, and determines a predetermined location in each inspection area ( For example, the center of the inspection area) may be set as the projection location of the information image. However, the method of determining the projection location of the information image is not limited to the above example, and the projection location may be determined by other methods. Further, the route deriving unit 24 may determine the route of the UAV 10 so that the information image projected on each projection location can be captured. At this time, it is preferable that the route deriving unit 24 defines a route that minimizes the power consumption of the UAV 10. However, the route deriving unit 24 may determine the route based on a criterion other than power consumption.

At this time, the route deriving unit 24 determines the target position for the UAV 10 to shoot the information image projected on the projection location in association with the projection location. Further, the route deriving unit 24 determines the posture of the UAV 10 and the orientation of the camera at the target position. The posture and the orientation of the camera are the orientation and the orientation of the camera when the information image is captured. However, due to the influence of wind or the like, it is difficult for the UAV 10 that actually moves to capture an information image at a predetermined target position and a predetermined posture and camera orientation. Therefore, the UAV 10 only needs to capture the information image near the target position. Further, the attitude of the UAV 10 and the orientation of the camera at that time do not have to match the attitude and the orientation of the camera that are set together with the target position.

The information image generation unit 25 generates an information image for each projection location. The information image generating unit 25 sets a target position (#P _i ) corresponding to the projection location (#i) and a target position (#) corresponding to the next projection location (#i+1). (P _i+1 )), the information image to be projected onto the projection location #i may be generated. Further, the information image generation unit 25 generates contour information for each target position.

The projector 21 projects the information image on the surface of the building. The orientation of the projector 21 is variable.

The projector control unit 22 controls the orientation of the projector 21. By changing the direction of the projector 21 by the projector control unit 22, it is possible to project the information image on the determined projection location.

In addition, when the projector control unit 22 directs the projector 21 to the projection location of the information image, the projector control unit 22 displays the information on the orientation of the projector 21, the appearance of the building, and the three-dimensional shape of the building on the surface of the building. The information image is corrected so that the information image is not distorted. In other words, the projector control unit 22 corrects the information image so that the rectangular information image is projected on the surface of the building. The projector 21 projects the information image corrected by the projector control unit 22. In the following description, the description of the operation of correcting the information image by the projector control unit 22 may be omitted.

The communication unit 27 is a communication interface used for communication with the UAV 10.

The route derivation unit 24, the information image generation unit 25, and the projector control unit 22 may be realized by, for example, a CPU of a computer that operates according to a program. In this case, the CPU may read the program and operate as the route derivation unit 24, the information image generation unit 25, and the projector control unit 22 according to the program. Further, the route deriving unit 24, the information image generating unit 25, and the projector control unit 22 may be realized by different hardware.

The control unit 13 (see FIG. 6) may be provided in the projection station 20 or another information processing device instead of the UAV 10. The operation of the control unit 13 is performed by the projection station 20 or another information processing device, and movement control of the UAV 10, attitude control, control of the orientation of the camera 11, and the like are performed by communication with the outside (for example, the projection station 20). May be realized.

Next, the processing progress of the first embodiment of the present invention will be described. FIG. 11 is a flowchart showing an example of the processing progress of the first embodiment of the present invention. The flowchart shown in FIG. 11 is an example, and the process progress of the present invention is not limited to the flowchart shown in FIG.

In addition, the route deriving unit 24 of the projection station 20 predefines a plurality of projected portions of the information image on the surface of the building, derives the route of the UAV 10, and the target positions for the UAV 10 to capture the information image, and , The orientation of the UAV 10 and the orientation of the camera at each target position are defined. Further, in the first embodiment, in order to simplify the explanation, it is assumed that the route is not changed during the inspection. Further, it is assumed that there is no effect on the UAV 10 due to a large disturbance that occurs instantaneously (for example, the UAV 10 is greatly deviated from the path due to an instantaneous gust). However, a deviation from the target position (see FIG. 2) due to normal wind or the like may occur.

The projection station 20 projects the information image from the projector 21 to the projection location of the image on the surface of the building (step T1). In the first step T1, the projection station 20 projects the first information image on the projection location of the image.

Further, the control unit 13 moves the UAV 10 toward a target position for capturing the target image projected in step T1 (step B1). When the first information image is projected, the inspection operator remotely operates the UAV 10. The inspection operator moves the UAV 10 so that the first information image falls within the field of view of the camera 11. The control unit 13 moves the UAV 10 so that the information image projected on the surface of the building falls within the field of view of the camera 11 according to the remote control by the inspection operator.

Further, the UAV 10 captures an image of the surface of the building with the camera 11 while moving to obtain a camera image (step B2). The camera 11 inputs the generated camera image to the control unit 13.

When the camera image is input, the control unit 13 searches for an information image in the camera image (step B3). The successful search means that the information image can be recognized in the camera image. Further, the failure of the search means that the information image cannot be recognized in the camera image. When the information image can be recognized in the camera image (Yes in step B3), the control unit 13 calculates the deviation between the current position of the UAV 10 and the target position (step B4). If the information image cannot be recognized in the camera image in step B3, such as when the information image is not in the camera image or when the information image is interrupted, the UAV 10 continues to move.

FIG. 12 is a schematic diagram showing an example of a camera image in which an information image cannot be recognized.

FIG. 12A shows a case where no information image is captured in the camera image 50. In this case, since the information image does not exist in the camera image, the control unit 13 cannot recognize the information image in the camera image.

FIG. 12B shows a case where only a part of the information image 40 is shown in the camera image 50. In this case, since the information image 40 is discontinued, the control unit 13 cannot recognize the information image in the camera image.

In FIG. 12C, the information image 40 is shown in the camera image 50, but the area of the information image 40 in the camera image 50 is very small (smaller than the threshold value). In this case, the control unit 13 cannot recognize the information image in the camera image because the area of the information image 40 is too small.

When the information image cannot be recognized in the camera image (No in step B3), the UAV 10 repeats the operation after step B1. When the control unit 13 cannot determine which direction to move based on the most recently obtained camera image when moving in the target position direction corresponding to the first projection location, Subsequently, the UAV 10 is moved according to the remote operation by the inspection operator. For example, when no information image is captured in the most recently obtained camera image, the control unit 13 continues to move the UAV 10 according to the remote operation by the inspection operator. Further, when the control unit 13 can determine which direction to move based on the most recently obtained camera image, the control unit 13 starts an autonomous operation regardless of the remote operation by the inspection worker. For example, as illustrated in FIG. 12B, when a part of the information image 40 is included in the most recently obtained camera image 50, the control unit 13 controls the entire information image 40 within the visual field of the camera 11. It is possible to determine in which direction the UAV 10 should be moved in order to enter the position. In this case, the control unit 13 autonomously moves the UAV 10 so that the entire information image 40 falls within the field of view of the camera 11.

When the information image cannot be recognized in the camera image (No in step B3), steps B1 to B3 are repeated until the information image can be recognized in the camera image.

When the information image can be recognized in the camera image, as illustrated in FIG. 9, the information image 40 is reflected in the camera image 50 in an appropriate size (in other words, the information image 40 is reflected in an area equal to or larger than the threshold value). ), where the contour abcd of the information image 40 can be detected. In this case, the control unit 13 determines the current state of the UAV 10 based on the contour 51 (see FIG. 9) in the camera image 50 indicated by the contour information and the contour abcd (see FIG. 9) of the information image 40 in the camera image 50. The deviation between the position and the target position is calculated (step B4).

The fact that the information image can be recognized in the camera image can be said to mean that the UAV 10 exists near the target position. Note that in reality, the UAV 10 repeats the processing of steps B1 to B3 while moving. Therefore, at the moment when all the information images are included in the camera image, the process proceeds to step B4 assuming that the information image is completely recognized in step B3. In other words, even if the UAV 10 is at a position relatively far from the target position, the control unit 13 executes Step B4 at the moment when the information image enters the edge of the camera image. In order to prevent this, although not shown in FIG. 11, “stop transition to step B4 until the information image falls within a certain range of the camera image” or “information within a certain range of the camera image The step B1 and subsequent steps are repeated after determining the movement direction for inserting the image", or "the transition to step B4 is stopped until the information image is recognized in the camera image for a certain time (or a certain number of frames)", etc. May be processed. Alternatively, since the shift M can be calculated in step B4, it is more preferable to calculate the movement amount corresponding to this shift M and repeat step B1 and subsequent steps. In addition to this, it is more preferable to repeat step B1 while controlling the direction of the camera so that the displacement M has a prescribed shape. This is because it is possible to move the UAV 10 to a more accurate position or move the UAV 10 to a more accurate posture by moving the UAV 10 based on the displacement M or controlling the camera 11.

After calculating the deviation between the current position of the UAV 10 and the target position, the control unit 13 removes the distortion of the information image 40 in the camera image 50 (step B5). As described above, the information image 40 projected on the surface of the building is rectangular, and the field of view 49 of the camera is not rectangular (see FIG. 7B). On the other hand, the camera image 50 obtained by capturing the field of view 49 is rectangular. Therefore, the information image 40 shown in the camera image 50 is not a rectangle but a distorted state (see FIG. 8). In step B5, the control unit 13 removes this distortion.

Subsequently, the control unit 13 recognizes the command (control content) indicated by the information image 40 after the distortion is removed, and executes the command (step B6).

In this example, a case where the control unit 13 recognizes a command (control content) indicated by the information image 40 illustrated in FIG. 3 and executes the command will be described as an example.

Specifically, the control unit 13 recognizes that the control unit 13 stays in the air for a predetermined time and takes an image for inspecting the state of the surface of the building in the air. Then, according to the instruction, the control unit 13 stays in the air for a predetermined time on the spot, and the surface of the building is imaged by the camera 11 in the air. At this time, the information image 40 may remain projected in the visual field of the camera 11.

Further, the control unit 13 recognizes that it descends after staying in the air for a predetermined time, that the camera 11 is directed upward with the horizontal angle of the camera 11 fixed when descending, and that the attitude of the UAV 10 is horizontally controlled. .. At this time, the control unit 13 determines the control content for moving to the next target position based on the shift between the current position and the target position of the UAV 10 calculated in step B4 and the move control content to the effect of lowering. The UAV 10 is moved by deriving and controlling each of the motors 17a to 17d via the motor driver 16 according to the control content. That is, the control unit 13 corrects the control content related to the movement indicated by the information image 40 by the shift calculated in step B4, and moves the UAV 10 based on the corrected control content.

After recognizing the command indicated by the information image 40, the control unit 13 notifies the projection station 20 that the command recognition is completed (step B7).

When the control unit 13 starts moving the UAV 10 based on the corrected control content, the control unit 13 repeats the operations from step B1. In this step B1, unlike the case where the UAV 10 is moved toward the target position corresponding to the first projection position, the control unit 13 autonomously moves the UAV 10 without depending on the remote operation by the inspection operator.

When the projector control unit 22 receives the notification that the command recognition is completed from the UAV 10, the projector control unit 22 causes the projector 21 to stop the projection of the information image that has been projected until then. Further, when the information image generation unit 25 receives the notification that the command recognition is completed from the UAV 10, the information image generation unit 25 specifies the projection location of the next information image (here, the projection location of the second information image), and the projection location. A second information image to be projected on is generated (step T2). In the present example, the information image generation unit 25 issues a command for instructing imaging for state inspection near the second target position and a command for moving the UAV 10 from the second target position to the third target position. An information image including, etc. is generated.

The UAV 10 images the surface of the building with the camera 11 while moving (step B2), and the control unit 13 executes step B3. Until the second information image is projected, the second information image does not appear in the camera image, so the UAV 10 repeats steps B1 to B3.

When the information image generation unit 25 generates the information image (here, the second information image) to be projected on the next projection location in step T2, the projector control unit 22 projects the information image to the projection location. Then, the projector 21 is controlled. As a result, the projector 21 projects the second information image on the projection location of the second information image (step T1).

When the UAV 10 moves to a position where the newly projected information image falls within the field of view of the camera 11 and the information image can be recognized in the camera image (Yes in step B3), the UAV 10 can repeat the processing from step B4. Good. When receiving the notification of step B7, the projection station 20 may stop the projection of the information image that has been projected until then, and execute steps T2 and T1.

By repeating the above processing, the UAV 10 can image the surface of the building for the condition inspection while moving in the vicinity of the determined route.

Further, when the projection station 20 projects the last information image and the control unit 13 executes the command shown in the last information image, the unmanned aerial vehicle control system operates without issuing the notification in step B7. The processing may end. In this case, the projection station 20 should include a predetermined symbol indicating that it is the last information image in the last information image so that the control unit 13 can recognize that it is the last information image. Good.

According to the present embodiment, the control unit 13 repeats steps B1 to B3 until the information image can be recognized in the camera image, and when the information image can be recognized in the camera image, calculates the shift M, The distortion of the information image in the camera image is removed, the command indicated by the information image is recognized, and the command operates according to the command. By the control unit 13 repeating this operation, the surface of the building can be imaged for the condition inspection while moving in the vicinity of the determined route. Therefore, the UAV 10 can be guided to the vicinity of the target position near the building with a simple configuration.

Further, in the present invention, since the UAV 10 is guided by the information image, the UAV 10 does not need to receive the GPS signal. Therefore, according to the present invention, it is not necessary to provide a large number of facilities in place of GPS satellites in order to obtain position information by GPS signals at any place. Also in this respect, the present invention can simplify the system configuration.

Further, in the present invention, the UAV 10 is guided by the information image. Since the UAV 10 approaches the information image to obtain the shift, it is possible to accurately obtain the position and orientation of the information image and the UAV 10. That is, since the UAV 10, which is a measuring device, is close to the information image that is the measurement target, there are few error factors, and as a result, it is possible to perform precise position measurement based on the GPS and other methods, and based on the measurement result. Induction is possible.

Further, the UAV 10 derives the deviation between the current position of the UAV 10 and the target position based on the information image. By using this deviation as the difference, the UAV 10 can move to the target position accurately.

Further, the UAV 10 derives the movement control content up to the next target position based on the shift between the current position of the UAV 10 and the target position and the movement control content indicated by the information image. As a result, the UAV 10 can move to the vicinity of the next target position. That is, the movement of the UAV 10 can be controlled with good accuracy. Further, as described above, in the present invention, the UAV 10 does not need to receive the GPS signal. Therefore, the UAV 10 can be accurately controlled even in a place where it is difficult to receive GPS signals.

Further, the projection station 20 projects an information image in a certain size on a building, and the UAV 10 monitors this information image. Further, since the positional relationship between this information image and the UAV 10 can be calculated inside the UAV 10, the UAV 10 can continuously acquire the positional relationship in almost real time. Therefore, for example, even if the position of the UAV 10 is greatly changed or the posture of the UAV 10 is changed due to a disturbance such as a gust of wind, the UAV 10 can correct the position and the posture before contacting the building.

Next, a modified example of the first embodiment will be described.

In the above embodiment, the case where the control unit 13 removes the distortion of the information image 40 in the camera image 50 in step B5 has been described. Even if the distortion of the information image 40 is not removed, if the control unit 13 can recognize the command indicated by the information image 40, step B5 may be omitted.

Further, in the above embodiment, in step B6, when the surface of the building is imaged for inspection of the state of the building, the surface of the building remains in the state where the information image 40 is projected in the visual field of the camera 11. The case where the image is captured is shown. The control unit 13 may instruct the projection station 20 to stop the projection of the information image when the surface of the building is imaged to inspect the state of the building. In this case, the information image generation unit 25 of the projection station 20 generates, for one projection location, an information image instructing imaging for a state inspection and an information image instructing movement to the next target position. do it. The information image generating unit 25 issues an instruction to notify the projection station 20 of the projection stop of the information image in the information image instructing the imaging for the state inspection, and after the notification (in other words, the projection station 20 An instruction to image the building for condition inspection (after stopping the projection in response to the notification) and an instruction to notify the projection station 20 of the completion of the imaging after the imaging is completed are included. The projection station 20 first projects the information image on the projection location. When the control unit 13 recognizes those commands and notifies the projection station 20 of the projection stop, the projection station 20 stops the projection of the information image. The UAV 10 images the surface of the building in this state. When the UAV 10 notifies the projection station 20 of the completion of image capturing, the information image instructing the movement to the next target position is projected on the same projection location. At this time, the UAV 10 may move slightly due to the influence of wind or the like while performing imaging for the state inspection. Therefore, it is preferable that the UAV 10 performs the operation after step B2 again even after the information image for instructing the movement to the next target position is projected. This is because the deviation between the current position of the UAV 10 and the target position is calculated again. Then, in step B6, the control unit 13 may correct the control content regarding the movement indicated by the information image by the shift.

When the surface of a building is imaged in a state where the information image is projected, in the camera image, a part other than the information image may be relatively dark. As described above, such a situation can be prevented by stopping the projection of the information image by the projection station 20 when the UAV 10 takes an image for the state inspection.

In addition, when the projection station 20 stops the projection of the information image when the UAV 10 takes an image for the state inspection, the projector control unit 22 causes the projector 21 to stop the projection of the information image, and then the projector 21 turns white. Light or light of an appropriate color may be irradiated. At this time, the projector 21 irradiates the surface of the building to be inspected with light according to the projector controller 22. As a result, the UAV 10 images the surface of the building for the condition inspection of the building while the surface of the building is illuminated with light.

By the projector 21 irradiating the surface of the building with light, it is possible to compensate for the insufficient amount of light when the camera 11 of the UAV 10 images the surface of the building. For example, when the camera 11 images a recessed part of a building or when operating an unmanned aerial vehicle control system at night, the amount of light is insufficient and a camera image in which the surface of the building is clearly captured is obtained. Absent. When the camera 11 of the UAV 10 captures an image of the surface of the building for the purpose of inspecting the condition of the building, the projector 21 irradiates the surface of the building with light, so that the insufficient light amount can be compensated as described above. 11 can generate a camera image in which the surface of the building is clearly reflected.

It should be noted that the projection station 20 may include a projector for irradiating light, in addition to the projector 21.

The contour shape of the information image projected from the light projecting station 20 does not always have to be the same. The contour information needs to be known before step B3 of recognizing the information image. Therefore, it may be included in the previous information image, the projection station 20 communicates with the UAV 10 by communication, or the contour information for each information image may be stored in the UAV 10 in advance. The control unit 13 may apply the contour information corresponding to the information image every time the deviation M is calculated. Further, by deforming the contour shape of the information image, it is possible to prevent the information image from interfering with the state inspection of the building.

In each of the embodiments described below, description of the same items as those already described will be omitted.

Embodiment 2.
In the second embodiment of the present invention, the operation when the posture and position of the UAV 10 change due to the large disturbance of the UAV 10 and the control unit 13 loses sight of the information image shown in the camera image will be described. To do. Even if the information image is captured in the camera image before the UAV 10 is subjected to the disturbance and the control unit 13 recognizes the information image, it is projected on the building because the posture or position of the UAV 10 is changed by the disturbance. If the displayed information image goes out of the field of view of the camera 11, the information image will not appear in the camera image. That is, the control unit 13 loses sight of the information image in the camera image.

In this case, the control unit 13 changes the posture of the UAV 10 in each direction, and the camera 11 captures an image. Alternatively, the control unit 13 changes the orientation of the camera 11 via the camera angle control unit 15 and causes the camera 11 to capture an image. At this time, the control unit 13 may change the orientation of the camera 11 while changing the posture of the UAV 10. As a result, if the control unit 13 acquires the camera image in which the information image is captured, the control unit 13 may perform the operations after step B4 (see FIG. 11).

At this time, the control unit 13 determines the position and orientation of the UAV 10 based on the camera images of the latest plurality of frames, based on the position and orientation when the control unit 13 recognized the information image in the camera image. You may detect how much it changed.

Alternatively, the UAV 10 may be equipped with an inertial navigation system. Then, the inertial navigation device may detect how much the position and orientation of the UAV 10 have changed from the position and orientation when the control unit 13 recognized the information image in the camera image.

Then, the control unit 13 determines the orientation of the UAV 10 and the orientation of the camera 11 in order to direct the visual field of the camera 11 to the information image direction based on the amount of change in each of the position and orientation of the UAV 10. May be. The control unit 13 may change the posture of the UAV 10 and the orientation of the camera 11 according to the determination result. In this case, the control unit 13 can quickly turn the field of view of the camera 11 toward the information image direction, and can quickly capture the projected information image.

Further, the control unit 13 may notify the projection station 20 of the amount of change in the position and orientation of the UAV 10 since the control unit 13 recognized the information image in the camera image. The control unit 13 may detect the amount of change based on the camera images of the latest plurality of frames, as in the above example. Alternatively, the UAV 10 may include an inertial navigation device, and the inertial navigation device may detect the amount of change in the position and orientation of the UAV 10 since the control unit 13 recognized the information image in the camera image.

When notified of the amount of change in the position and orientation of the UAV 10, the information image generation unit 25 (see FIG. 10) of the projection station 20 calculates the visual field range of the camera 11 based on the amount of change, and sets the visual field range to the visual field range. An information image to be projected is generated. This information image is an information image for guiding the UAV 10 to the position where the control unit 13 recognized the information image in the camera image. The projector control unit 22 points the projector 21 in the direction of the calculated visual field range of the camera and causes the projector 21 to project the information image. Then, the camera 11 captures an image (step B2), and then the UAV 10 may perform the operation of step B3 and subsequent steps. As a result, the UAV 10 can return to the position where the control unit 13 recognized the information image in the camera image according to the information image. Further, the projector control unit 22 may cause the projector to re-project the original information image at the original projection location after receiving the notification in step 7. In this case as well, the camera 11 may capture an image (step B2), and then the UAV 10 may perform the operation of step B3 and subsequent steps.

A state in which the projection station 20 detects a change amount of the position and orientation of the UAV 10 based on a tracking camera (not shown) that captures the UAV 10 while tracking the UAV 10 and an image of the UAV 10 obtained by the tracking camera. A configuration including a change amount detection unit (not shown) may be used. In this case, when the state change amount detection unit detects that the UAV 10 has suddenly moved due to a disturbance based on the image obtained by the tracking camera, the change in the position and attitude of the UAV 10 from the time immediately before the movement. Detect the amount. Then, as in the case described above, the information image generation unit 25 (see FIG. 10) calculates the visual field range of the camera based on the change amount, and generates the information image projected on the visual field range. This information image is an information image for guiding the UAV 10 to the position where the control unit 13 recognized the information image in the camera image. The projector control unit 22 points the projector 21 in the direction of the calculated visual field range of the camera and causes the projector 21 to project the information image. Subsequent operations are the same as above, and description thereof will be omitted.

According to the second embodiment, even if the UAV 10 causes the field of view of the camera 11 to face a direction different from the information image due to disturbance, the UAV 10 captures the information image again and returns to normal control. can do.

Embodiment 3.
In the third embodiment of the present invention, the unmanned aerial vehicle control system avoids a state in which the information image and the shadow of the UAV 10 overlap and appear in the camera image.

The route deriving unit 24 of the projection station 20 determines a projection location that does not overlap the shadow of the UAV 10 as the projection location of the information image.

For example, the route deriving unit 24 defines a plurality of inspection areas by partitioning the surface of the building into a lattice shape based on the information stored in the data storage unit 23, and a predetermined location in each inspection area ( For example, it is assumed that the center of the inspection area) is defined as the projection location of the information image. In addition, the route deriving unit 24 also determines the target position of the UAV 10, the attitude of the UAV 10 at the target position, and the camera orientation for each projection position. Then, the route deriving unit 24 determines, for each projection location, whether the projection location and the shadow of the UAV 10 overlap. This determination can be made based on the positional relationship among the projection station 20, the target position, and the projection location. The route deriving unit 24 changes a projection position overlapping the shadow of the UAV 10 to a position that does not overlap the shadow of the UAV 10 in the inspection area, and according to the change, the posture of the UAV 10 and the orientation of the camera corresponding to the projection position. change. At this time, the route deriving unit 24 may also change the target position of the UAV 10. If the position of the projection location is changed within the inspection area and the shadow of the UAV 10 overlaps, it is determined that the projection location is not defined for the inspection area. Then, the route deriving unit 24 determines the route for capturing the information image projected on each projection location, targeting the projection location not overlapping the shadow of the UAV 10.

By thus defining the projection location of the information image, it becomes easy to prevent the information image projected on the projection location and the shadow of the UAV 10 from being captured in the camera image in a state of overlapping.

Further, as described above, even when the route deriving unit 24 determines the projection position that does not overlap the shadow of the UAV 10, the UAV 10 is affected by the wind or the like, so that the information image is displayed at the target position corresponding to the projection position. Imaging is generally rare. As a result, even when the projection location is determined as described above, the information image projected on the projection location may overlap the shadow of the UAV 10.

In this case, the control unit 13 may move the UAV 10. FIG. 13 is a schematic diagram showing an example of a camera image in which the information image and the shadow of the UAV 10 overlap each other. When the control unit 13 determines that a part of the contour of the information image 40 is interrupted due to the presence of the black pixel group, a part of the outer peripheral portion of the information image 40 and the shadow 55 are formed as illustrated in FIG. It is determined that they overlap. When the control unit 13 makes such a determination, for example, the control unit 13 may move in an arbitrary direction and execute the operation after step B1 (see FIG. 11). Further, at this time, the control unit 13 may determine the direction in which the UAV 10 is moved based on the portion of the contour of the information image 40 that is interrupted by the shadow 55, and may move the UAV 10 in that direction. FIG. 14 is a schematic diagram showing an example of a camera image in which the information image 40 and the shadow 55 of the UAV 10 do not overlap each other due to the movement of the UAV 10. In the camera image illustrated in FIG. 14, the position of the entire contour of the information image 40 can be detected. After moving the UAV 10, the control unit 13 can calculate the deviation between the current position and the target position by using the camera image illustrated in FIG. 14. After that, the control unit 13 may execute the operations after step B5 (see FIG. 11).

In this way, when the overlap between the information image 40 and the shadow of the UAV 10 is detected, the control unit 13 can avoid the overlap between the information image 40 and the shadow of the UAV 10 by moving the UAV 10. This operation is applicable even when the route deriving unit 24 does not determine the projection location so that the shadow of the UAV 10 and the projection location do not overlap.

In the above description, the operation of avoiding the overlapping of the information image 40 and the shadow of the UAV 10 due to the movement of the UAV 10 has been described. The control unit 13 may avoid the overlapping of the information image 40 and the shadow of the UAV 10 by causing the projection station 20 to change the projection position of the information image. For example, it is assumed that the control unit 13 determines that a part of the outer peripheral portion of the information image 40 and the shadow 55 overlap, as in the above case. In this case, the control unit 13 causes the projection station 20 to avoid the overlapping of the information image 40 and the shadow 55 based on the position of the contour of the information image 40 that is interrupted by the shadow 55, so that the projection station 20 projects the information image onto the projection position. It is determined in which direction and how much is to be moved, and the projection station 20 is notified of the movement direction and movement amount of the projection location. It can be said that this notification is an instruction to change the projection location. The information image generation unit 25 of the projection station 20 that has received this notification changes the content of the information image 40 according to the notified moving direction and moving amount. This is because the content of movement control to the next projection location changes as the projection location of the information image changes. The route deriving unit 24 also determines a target position when the projection location is moved according to the notified movement direction and movement amount, and a posture and a camera direction at the target position. The projector control unit 22 points the projector 21 in the direction of the projection location according to the notified movement direction and movement amount, and causes the projector 21 to project the information image 40. As a result, the UAV 10 obtains the camera image illustrated in FIG. 14, for example. After that, the control unit 13 may execute the operations after step B4 (see FIG. 11).

In this way, the UAV 10 causes the projection station 20 to change the projection location of the information image 40, so that the overlapping of the information image 40 and the shadow of the UAV 10 can be avoided. This operation is applicable even when the route deriving unit 24 does not determine the projection location so that the shadow of the UAV 10 and the projection location do not overlap.

Embodiment 4.
In the fourth embodiment of the present invention, the UAV 10 changes the route in the middle of the route when the power consumption is large due to the influence of the weather or the like and the imaging for the state inspection cannot be performed in all the inspection areas.

As described in the first embodiment, the route deriving unit 24 determines the route of the UAV 10 so that the information image projected on each projection location can be captured, for example. That is, the path of the UAV 10 is determined so as to pass near each projection location.

At this time, it is preferable that the route deriving unit 24 defines a route that minimizes the power consumption of the UAV 10. Here, the UAV 10 consumes a large amount of power when the flight velocity vector changes. Therefore, the route deriving unit 24 may determine, for example, the route that minimizes the velocity vector change of the UAV 10.

It is assumed that the route deriving unit 24 defines the route illustrated in FIG. FIG. 15 illustrates each inspection area on the surface of the building and the path 61 of the UAV 10. The inspection area Ra is the first inspection area and the inspection area Rz is the last inspection area.

The route deriving unit 24 defines a position on the route 61 at which the UAV 10 can be switched to a short cut route (hereinafter, referred to as a switchable position). The number of switchable positions may be one or plural. The switchable position 71 shown in FIG. 15 will be described as an example.

After determining the route 61, the route deriving unit 24 determines a plurality of route candidates that are shortcuts from the switchable position 71 to the end point 79. The route deriving unit 24 determines route candidates so as not to pass the inspection area that has already passed. Moreover, since the route candidate is a short cut from the switchable position 71 to the end point 79, it is not necessary to pass all the inspection areas that have not been passed. Further, the number of inspection areas through which a plurality of route candidates defined with respect to the switchable position 71 may be different for each route candidate. In other words, the lengths of the route candidates from the switchable position 71 to the end point 79 may be different from each other. In addition, in this example, in order to simplify the description, in each route candidate, the movement route for one section from the switchable position 71 to the next inspection area is the same as the predetermined route.

For each switchable position other than the switchable position 71, the route deriving unit 24 determines a plurality of route candidates that are shortcuts to the end point 79 for each switchable position.

Hereinafter, in order to simplify the description, it is assumed that the target position of the UAV 10 corresponding to the inspection area is set as the switchable position.

When notifying the projection station 20 that the command recognition has been completed (step B7), the control unit 13 of the UAV 10 consumes energy after the start of flight, the battery level at that time, and the weather (here, the wind speed of the crosswind). Are taken as an example), and those values are also notified to the projection station 20. It should be noted that the UAV 10 may be equipped with a sensor for measuring these values. However, the control unit 13 may, for example, when the condition that the flightable distance is sufficiently secured (for example, the condition that the battery remaining amount is equal to or more than the threshold value) is satisfied, the energy consumption, the battery remaining amount and the weather It is not necessary to notify the projection station 20 of the crosswind velocity).

In addition, the route deriving unit 24 holds in advance a predictable formula of a flightable distance in which the energy consumption, the remaining battery level, and the weather (wind speed of the crosswind) are used as explanatory variables, and the flightable distance of the UAV 10 is used as an explained variable. .. This prediction formula may be determined in advance by machine learning such as regression analysis. When the route derivation unit 24 is notified of the energy consumption, the remaining battery level, and the weather from the UAV 10, the route derivation unit 24 substitutes these values into the explanatory variables of the prediction formula to calculate the flight distance of the UAV 10.

Further, the route deriving unit 24 subtracts the feasible distance calculated by the prediction formula from the length of the route candidate for each route candidate that is predetermined for the switchable position where the UAV 10 exists. The route derivation unit 24 determines that the route candidate having the smallest subtraction result is the optimum shortcut, and thereafter, the route candidate is set as the route of the UAV 10.

The power consumption of the UAV 10 is affected by weather such as wind. For example, when a strong wind is blowing, a thrust force that opposes the strong wind is required, and thus the power consumption is higher than that when there is no wind. As a result, when the UAV 10 tries to continue moving along the route 61 that is initially determined, the remaining battery power may become 0 in the middle of the route. According to the present embodiment, it is possible to switch to a route that is a shortcut to the end point 79 at the switchable position, so that the flight of the UAV 10 is maintained until the end point 79 in exchange for reducing the number of inspection sections to pass. You can

In the above example, the energy consumption, the remaining battery level, and the weather are illustrated as the explanatory variables used in the prediction formula of the flightable distance, but other explanatory variables may be used.

Embodiment 5.
In the fifth embodiment of the present invention, when the position of the UAV 10 changes due to a large disturbance of the UAV 10, the projection station 20 again derives the route from the changed position to the end point of the route. To do.

When the UAV 10 moves due to a disturbance and the control unit 13 of the UAV 10 loses sight of the information image in the camera image, the control unit 13 recognizes the information image in the camera image as in the case illustrated in the second embodiment. The projection station 20 is notified of the amount of change in the position and orientation of the UAV 10 from the time it was being performed.

Upon receiving this notification, the route derivation unit 24 of the projection station 20 calculates the current position of the UAV 10 based on this notification. Then, the route deriving unit 24 derives the route from that position again. However, the route deriving unit 24 may newly derive the route so as not to pass through the inspection area where the UAV 10 has already passed. Further, the new route may not pass all the inspection areas that have not been passed. In addition, at this time, the route deriving unit 24 preferably selects, for example, a route that minimizes power consumption. For example, as described above, the route deriving unit 24 may determine the route that minimizes the change in the velocity vector of the UAV 10.

According to the present embodiment, even if the UAV 10 moves due to a disturbance such as a gust of wind, the route derivation unit 24 redefines the route from that position. Therefore, it is possible to suppress an increase in power consumption of the UAV 10 and improve the operation efficiency of the UAV 10.

Embodiment 6.
In the sixth embodiment of the present invention, the projection station 20 generates (corrects) an information image according to the reflectance of the projection surface of an object such as a building (hereinafter referred to as “projection target surface”), Project the generated information image.

In the following description, the reflectance means each reflection amount of light at each wavelength peculiar to an object. Therefore, even if the reflectance of a certain wavelength is known for the same object, the reflectance of another wavelength is not always the same. The reflectance in a fixed wavelength region indicates an integrated value of the reflectance in that wavelength region. Further, in the following description, the brightness means the energy of light per unit area at each wavelength. In addition, in the following description, the light spectrum indicates a luminance distribution at each wavelength in a certain wavelength range.

The reflectance is calculated, for example, based on a captured image of the projection target surface captured by the camera 11 mounted on the UAV 10. Specifically, for example, the control unit 13 of the UAV 10 predicts the spectrum of the light illuminating the projection target surface from the captured image of the projection target surface captured by the camera 11, and determines the wavelength of each wavelength from the prediction result. Calculate reflectance. That is, the control unit 13 calculates the reflectance based on the color information of the captured image of the projection target surface. Further, the control unit 13 may calculate the reflectance of each wavelength assuming that sunlight is reflected. Various methods are known for estimating the spectrum of the illumination light, and therefore the description thereof will be omitted.

In addition, the reflectance may be calculated, for example, by the projection station 20 projecting an image serving as a predetermined reference (hereinafter, referred to as a “reference image”) onto the projection target surface. For example, when a white frame-shaped image is projected as a reference image on the projection target surface, the projection target surface changes the spectrum according to the reflectance of each wavelength and outputs the reflected light. The camera 11 of the UAV 10 images the surface of the building including this reflected light (white frame shape) and generates a camera image. Then, the control unit 13 calculates the reflectance by recognizing the spectrum of the frame-shaped portion in the camera image (what color the image of the frame-shaped white light reflected is in the camera). To do. That is, the control unit 13 calculates the reflectance based on the color information of the reference image projected on the projection target surface and the color information of the reference captured image. The color of the reference image is not limited to white, and may be light having a constant spectral shape in a certain wavelength range. As described above, when projecting the reference image, it is not necessary to estimate the spectrum of the light illuminating the projection target surface, and thus more accurate reflectance is obtained.

In the above description, the case where the control unit 13 of the UAV 10 calculates the reflectance has been described as an example, but not limited to the control unit 13, the projector control unit 22 of the projection station 20, the information image generation unit 25, and other components. The device may calculate the reflectance. These configurations for calculating the reflectance function as a reflectance calculating unit. Further, in the above description, the case where the camera 11 mounted on the UAV 10 captures the projection target surface and the reference image projected on the projection target surface has been described as an example. However, the camera that captures these images is not limited to the camera 11, and may be a camera mounted on the projection station 20 or another camera. Also in the following description, a case where the camera 11 mounted on the UAV 10 captures a reference image or the like and the control unit 13 of the UAV 10 calculates the reflectance will be described as an example.

The control unit 13 notifies the projector control unit 22 of the projection station 20 of the calculated reflectance. Upon receiving the reflectance information of each wavelength, the projector controller 22 of the projection station 20 corrects the spectrum of the information image generated by the information image generator 25 according to the reflectance.

When the reflectance is uniform, the projected image is reproduced with the same tint. On the other hand, when the reflectance is different for each wavelength, for example, a frame-shaped image projected with white light is recognized as a different color in the camera image. For example, when the surface of the building is painted green, the reflectance is close to 100% in the wavelength range of 550 nm±30 nm, and the other wavelengths are close to 0%. That is, for example, even if an information image of white light of 400 nm to 650 nm is projected, only the green light component of 550 nm±30 nm is observed by the camera, and the light of other wavelength components is not observed. In this case, information drawn at wavelengths other than 550 nm±30 nm is lost.

Therefore, upon receiving the reflectance, the projector control unit 22 corrects the spectrum of the information image according to the wavelength so that the wavelength falls within the range of 530 nm to 570 nm, for example. As a result, the camera 11 can capture the information image projected on the surface of the building without excess or deficiency. At the same time, the projector controller 22 may correct the brightness of the information image in addition to the color. The projector control unit 22 of the projection station 20 corrects the brightness of the information image based on the reflectance. For example, when the reflectance of the building is low, the projector control unit 22 corrects so that the brightness of the information image is increased.

By correcting the spectrum of the information image based on the reflectance in this way, it is possible to prevent a part or all of the information image projected on the surface of the building from being unrecognizable by the control unit 13 of the UAV 10 and running out of information. .. The information image generating unit 25 may newly generate an information image having a spectrum corresponding to the reflectance received from the control unit 13. Such correction or generation of the information image by the information image generation unit 25 may be generation of an image according to the reflectance.

Further, the information image may be composed of only a specific wavelength. That is, the projector 21 may be, for example, a projector (for example, a laser projector) having a light source that emits only light of a predetermined wavelength for each color of red, green, and blue. The projector control unit 22 may correct the color of the information image by switching the light source of the projector 21 according to the reflectance.

Further, the camera 11 of the UAV 10 may be provided with a filter (light transmitting unit) that is a filter that transmits only light having a predetermined wavelength and that can switch the wavelength of light that is transmitted. For example, the wavelength of the light transmitted by the filter may be set according to each light source of the projector 21. For example, the filter of the camera 11 transmits only the light of the wavelength corresponding to the red light source of the projector 21, transmits only the light of the wavelength corresponding to the green light source of the projector 21, and the blue of the projector 21. It is a filter that can be switched to any one of the states in which only the light of the wavelength corresponding to the light source is transmitted. For example, when the control unit 13 notifies the projection station 20 of red information, the control unit 13 switches the filter of the camera 11 to a state in which only the light of the wavelength corresponding to the red light source of the projector 21 is transmitted. Similarly, when the control unit 13 notifies the projection station 20 of the information that the color is green, the control unit 13 switches the filter of the camera 11 to a state of transmitting only the light of the wavelength corresponding to the green light source of the projector 21. Similarly, when the control unit 13 notifies the projection station 20 of blue information, the control unit 13 switches the filter of the camera 11 to a state in which only the light of the wavelength corresponding to the blue light source of the projector 21 is transmitted. The mode of the filter included in the camera 11 is not particularly limited. For example, in a general color camera, an R/G/B filter having a Bayer arrangement is arranged in each pixel. Only the light of the corresponding light source may be used by this filter. That is, if only the green light source is used, only the green pixel may be used. Similarly, if a light source of another color is used, only pixels corresponding to that light source may be used. Alternatively, a combination of a monochrome camera and a replaceable color filter attached to the outside may be used. Further, the camera 11 may not be provided with the above filter.

Especially when the ambient light is strong with respect to the brightness of the projector such as in direct sunlight, a light source having a narrow wavelength width such as a laser is used for the projector 21 of the projection station 20, and a narrow band filter that transmits only the light of this wavelength is used for the UAV10. It is even better to attach it to. More precisely, the light spectrum of the image captured by the camera 11 of the UAV 10 is such that both the light projected by the projector 21 and the ambient light (such as sunlight) are reflected according to the reflectance of the building surface. It is a light. The brightness acquired by the camera 11 is the integrated brightness. Therefore, for example, when the integrated luminance of the ambient light in a certain wavelength width is 100 and the integrated luminance of the light projected by the projector 21 is 10, the reflected light has a component of the ambient light which is 10 times larger. That is, even if the camera 11 captures the information image at the same time as the ambient light, the integrated luminance difference from the area other than the information image is small, and the control unit 13 may not be able to extract the information image. In order to prevent this, the wavelength range used by the narrow band light source and the filter is limited, and only light of a certain specific wavelength is used. That is, when the integration range of light captured by the camera 11 is narrowed, the integrated brightness of the projector 21 remains unchanged at 10, but the integrated brightness of ambient light decreases from 100 to, for example, 10, so that the brightness difference of the reflected light is the same. As a result, the information image can be detected.

In addition, when the reflection from the projector light is weaker than the reflection from the ambient light, the projection station 20 may compare the images with and without the projector light to extract the reflection component from the projector light. That is, the UAV 10 and the projection station 20 are synchronized with each other by a timer or a timing signal so that the control unit 13 of the UAV 10 can determine the timing at which the projector 21 of the projection station 20 switches the presence/absence of projection. According to this timing, the UAV 10 captures an image when the projector light is included and when it is not included. Since both images include ambient light components, if the control unit 13 generates a difference image between the former and the latter, it is possible to extract only the reflected component from the projector light. That is, the projection station 20 calculates the difference between the captured image of the projection surface when the image is projected on the projection surface of the object and the captured image of the projection surface when the image is not projected on the projection surface of the object. An image corresponding to the reflectance of the projection surface is generated based on

Further, when the projection station 20 includes a camera, the camera of the projection station 20 captures an image of the surface of the building in advance, and the route deriving unit 24 has a weak ambient light component of the surface of the building based on the image. It is also possible to extract a portion (a region in which a shadow is generated without being exposed to direct sunlight). Then, the route deriving unit 24 may determine the area where the ambient light component is weak as the projection location of the information image.

Also, when it is necessary to project the information image on the boundary area between the area where the ambient light component is strong and the area where it is weak, or the area where the pattern is drawn on the projection surface, the intensity distribution of the ambient light in that area and the reflectance of the pattern In consideration of the above, the information image generating unit 25 may generate the information image. That is, for example, the background part of the information image in the part where the sun is shining lowers the brightness level, and the background part of the information image in the part where the sun is not shaded is set to the higher brightness level. It is prevented that the information image is erroneously recognized by the control unit 13 of the UAV 10 depending on the presence or absence. Regarding the pattern, the light spectrum of each part is changed so that it looks uniform when it becomes reflected light, and an image is projected to avoid misrecognition.

According to the present embodiment, the camera 11 can generate a camera image in which the information image is captured even when the surface of the building reflects only light of a specific wavelength. Further, it is possible to prevent the information image projected on the surface of the building from being buried in the ambient light. Therefore, the information image in the camera image can be easily detected.

Embodiment 7.
In the seventh embodiment of the present invention, an operation will be described when the projection surface of the information image and the target surface of the imaging for the state inspection are different.

FIG. 16 is a schematic diagram illustrating an example of a situation in which the projection surface of the information image and the target surface of the imaging for the state inspection are different. It is assumed that the area to be imaged for the state inspection is the area 82. However, the area 82 is horizontal. Further, in the area 82, there is an area that becomes a blind spot when viewed from the projector 21 due to the structure 39 of the building 30. In addition, the information image cannot be projected in the blind area when viewed from the projector 21. Even if the projector 21 projects an information image in an area of the area 82 that is not a blind spot as viewed from the projector 21, the UAV 10 may not be able to capture the information image projected in the horizontal area 82.

In such a situation, the projection surface of the information image and the imaging target surface for the state inspection may be different surfaces.

The area 81 is an area including the projected portion of the information image. The projector 21 projects an information image (hereinafter, this information image is referred to as an information image Q) on the area 81. When generating the information image Q, the information image generating unit 25 (see FIG. 10) of the projector 21 separates the control content of the direction of the moving camera from the control content of the moving camera into the information image Q. Include it. In the present example, the information image generation unit 25 includes, in the information image Q, the control content for instructing that the direction of the camera be directly upward while in the air.

The UAV 10 advances toward the target position P3 based on the information image immediately preceding the information image Q, and performs steps B1 to B3 (see FIG. 11). Then, if the information image can be recognized in the camera image, the UAV 10 performs the operation of step B4 and thereafter.

In step B6, the control unit 13 of the UAV 10 (see FIG. 6) makes the UAV 10 stay in the air according to the information image Q. Further, according to the control content included in the information image Q, the control unit 13 orients the camera in a direct upward direction while in flight. The control unit 13 captures an image for a state inspection while staying in this state. At this time, since the camera 11 faces directly above, the region 82 to be inspected can be imaged and a camera image of the region 82 can be generated.

According to the present embodiment, even if the information image cannot be projected on the target surface for imaging for the state inspection, the information image Q projected on another surface can be used to reach the vicinity of the target position. .. Then, according to the control content included in the information image Q, the UAV 10 can change the camera direction in the air and image the target surface of the imaging for the state inspection, and generate the camera image of the surface.

Embodiment 8.
Deterioration of a building over time may cause defects such as cracks and floats on the surface of the building. The unmanned aerial vehicle control system of the eighth embodiment of the present invention detects such defects. In addition, the unmanned aerial vehicle control system corrects the information image according to the defect so that the projected information image is not distorted.

In the eighth embodiment of the present invention, the projector 21 also projects an image representing a predetermined pattern (hereinafter referred to as a pattern image) onto the surface of the building. The control unit 13 of the UAV 10 detects the state of defects on the surface of the building according to the shape of the pattern shown in the camera image. The projection station 20 corrects the information image according to the state of the defect.

Hereinafter, it will be described more specifically. The projector 21 projects the pattern image on the surface of the building. The pattern image may be, for example, an image representing a mesh pattern. In the present embodiment, in order to simplify the description, a horizontal straight line will be described as an example of the predetermined pattern. FIG. 17 is a schematic diagram showing an example of a pattern image representing a horizontal straight line. Although FIG. 17 shows a pattern image representing one straight line, a plurality of straight lines may be represented in the pattern image.

FIG. 18 is a schematic diagram showing a state in which a pattern image is projected on the surface of a building having a step due to deterioration over time, and the camera 11 of the UAV 10 is capturing the projected portion. FIG. 19 is a schematic side view of the state shown in FIG. In the example shown in FIG. 18 and FIG. 19, the surface 91 of the building is in a state of being more floating than the surface 92, and a step is formed.

The camera 11 images the surface 91 and the surface 92 onto which the pattern image (horizontal straight line) is projected, and generates a camera image. FIG. 20 is a schematic diagram showing an example of a camera image obtained as a result of capturing a pattern image. In the camera image shown in FIG. 20, the surfaces 91 and 92 and the line 93 are shown. The projector 21 projects a horizontal straight line on the surfaces 91 and 92, but due to the step between the surfaces 91 and 92, the straight line appears as a curved line 93 in the camera image.

The projection station 20 also transmits the angle formed by the projection direction of the pattern image and the wall surface to the UAV 10. Projection station 20 may communicate this angle to UAV 10 via communication. Alternatively, the projection station 20 may project an information image including this angle information, and the control unit 13 of the UAV 10 may recognize the angle information from the information image in the camera image.

The control unit 13 of the UAV 10 recognizes the shape of the pattern (line 93 in this example) captured in the camera image. The control unit 13 determines the surface state of the building based on the shape of the pattern captured in the camera image. In this example, the control unit 13 determines that there is a step between the surface 91 and the surface 92 based on the shape of the line 93 shown in the camera image. That is, the control unit 13 detects the size of the step between the surface 91 and the surface 92 and the position where the step is generated. For example, the control unit 13 may store in advance the correspondence between the angle formed by the projection direction of the pattern image and the wall surface and the shape deformation of the pattern in the camera image. Then, the control unit 13 may determine the surface state of the building by identifying the surface state of the building corresponding to the shape deformation of the pattern in the camera image.

The control unit 13 transmits information on the state of the surface of the building (in this example, the size of the step between the surfaces 91 and 92 and the position where the step is generated) to the projection station 20.

The projector controller 22 of the projection station 20 receives the information on the state of the surface of the building transmitted from the controller 13. Then, the projector control unit 22 receives information on the appearance of the building and the three-dimensional shape of the building stored in the data storage unit 23, and information on the state of the surface of the building received from the UAV 10 (control unit 13). Based on, the information image is corrected so that the projected information image is not distorted.

According to the present embodiment, the control unit 13 can detect a defect in a building caused by deterioration over time and the size of the defect. Furthermore, since the control unit 13 transmits information on the state of the surface of the building to the projection station 20, the projection station 20 (projector control unit 22) corrects the information image so that the projected information image is not distorted. can do. Therefore, the projection station 20 can project the information image on the surface of the building in a good condition, and the guiding accuracy of the UAV 10 can be further enhanced.

Each of the embodiments of the present invention is applicable not only to infrastructure deterioration diagnosis, but also to controlling a UAV for patrol monitoring of a building.

Next, the outline of the present invention will be described. FIG. 21 is a schematic diagram showing an outline of the unmanned aerial vehicle control system of the present invention. The unmanned flight device control system 100 includes an image projection unit 101 and a control unit 103.

The image projection unit 101 (for example, the projection station 20) is an image (for example, information image) that controls the operation of an unmanned aerial vehicle (for example, the UAV 10) equipped with an imaging device (for example, the camera 11), and is an object. An image generated according to the reflectance of the projection surface of (for example, a building that is the target of the state inspection) is projected on the target.

The control unit 103 (for example, the control unit 13) controls the operation of the unmanned aerial vehicle based on the image captured by the imaging device.

With such a configuration, when the image for controlling the operation of the unmanned aerial vehicle is projected on the object and the unmanned aerial vehicle is guided to the vicinity of the target position near the object, consider the environment for projecting the image. Control unmanned aerial vehicle.

INDUSTRIAL APPLICABILITY The present invention is preferably applied to an unmanned aerial vehicle control system that controls an unmanned aerial vehicle.

10 UAV (unmanned air vehicle)
11 camera 13 control unit 15 camera angle control unit 16 motor driver 17a to 17d motor 18 communication unit 20 projection station 21 projector 22 projector control unit 23 data storage unit 24 route derivation unit 25 information image generation unit 27 communication unit

Claims (7)

An image for controlling the operation of an unmanned aerial vehicle equipped with an imaging device, the image projection unit for projecting an image generated according to the reflectance of the projection surface of the object onto the object,
A control unit for controlling the operation of the unmanned aerial vehicle based on the image captured by the image capturing apparatus ,
The image projection unit determines a projection position of the image for controlling the operation of the unmanned aerial vehicle based on the reflectance.
No person flight control system. The image projection unit projects a predetermined reference image onto the object,
The unmanned aerial vehicle according to claim 1, further comprising: a reflectance calculation unit that calculates the reflectance based on information on a color of the projected image serving as the reference and information on a color of a captured image of the image serving as the reference. Control system. The unmanned aerial vehicle control system according to claim 1, further comprising: a reflectance calculation unit that calculates the reflectance based on color information of a captured image of the projection surface of the object. The imaging device includes a light transmission unit that transmits light having a wavelength according to the reflectance,
The image projection unit according to any one of claims 1 to 3 for projecting an image to the transmission unit controls the operation of irradiating light having a wavelength that passes through the unmanned flying device Unmanned aerial vehicle control system. The image projection unit captures an image of the projection surface when the image is projected on the projection surface of the object, and captures an image of the projection surface when the image is not projected on the projection surface of the object. The unmanned aerial vehicle control system according to any one of claims 1 to 4 , wherein an image corresponding to the reflectance of the projection surface is generated based on the difference between An image for controlling the operation of an unmanned aerial vehicle equipped with an imaging device, wherein an image generated according to the reflectance of the projection surface of the object is projected onto the object,
When projecting the image, based on the reflectance, determine the projection position of the image for controlling the operation of the unmanned aerial vehicle,
An unmanned flight device control method for controlling an operation of the unmanned flight device based on the image captured by the image capture device. An image generation unit that generates an image for controlling the operation of an unmanned aerial vehicle equipped with an imaging device according to the reflectance of the projection surface of the object,
A projection unit for projecting the generated image ,
The projection unit determines a projection position of the image for controlling the operation of the unmanned aerial vehicle based on the reflectance.
Images projection device.

Images