WO2020006732A1 - Unmanned aerial vehicle landing method and apparatus, and unmanned aerial vehicle - Google Patents

Abstract

An unmanned aerial vehicle landing method and apparatus, and an unmanned aerial vehicle. By means of collecting reference images photographed by an unmanned aerial vehicle during a take-off process and including a take-off point, and performing template matching by stages during a landing process of the unmanned aerial vehicle, a sensor error of the unmanned aerial vehicle can be eliminated and the distance of the unmanned aerial vehicle deviating from the take-off point can be acquired in real time, thereby controlling the unmanned aerial vehicle to continuously approach the take-off point during the landing process and to finally land at the take-off point accurately. During the whole landing process of the unmanned aerial vehicle, no other auxiliary devices are needed, and the landing effect is good, and the requirements on the accuracy of a sensor carried by the unmanned aerial vehicle are not high.

Description

UAV landing method, device and UAV Technical field

The present invention relates to the technical field of aircraft, and in particular, to a method, a device, a drone, and a computer-readable storage medium for landing a drone.

Background technique

With the development of vision algorithms and the application of the vision algorithms on UAV (Unmanned Aerial Vehicle) platform, intelligent tracking has a good tracking effect. And how to realize the entire unmanned process from tracking to landing is an important direction to improve the intelligence of drones.

At present, the drones in the prior art have been able to accurately land the drone at the take-off point of the drone, but this technology needs to place auxiliary equipment such as a signal source at the take-off point and send signals to the drone. Guide the drone to land accurately. In addition, special signs can be set at the takeoff point or based on rich texture images, the texture pattern near the takeoff point can be used directly to guide the drone to land accurately.

However, for small drones, the use of auxiliary equipment to guide the drone to land accurately has the disadvantage of being inconvenient to carry; for the way of setting special signs at the take-off point, there is the disadvantage of inconvenience in operation. For rich texture images, the use of texture patterns near the take-off point combined with vision technology to guide the precise landing of the drone has a sparse texture at the landing point, and there is a poor landing effect when the sensors mounted on the drone have large errors. The landing site has the disadvantage of being inaccurate.

Summary of the invention

Based on this, it is necessary to provide a drone landing method, a device, a drone, and a computer-readable storage medium that can accurately land to the take-off point without the need for other auxiliary equipment.

A drone landing method, the method includes:

During the take-off of the drone, collecting images including take-off points taken by the drone at different flight altitudes, and using the collected images containing the take-off points as reference images;

During the landing of the drone, the step of template matching is repeated until the drone reaches the take-off point; wherein the template matching includes:

Acquiring an image taken by the drone at a current flying height;

Obtaining a reference image that matches the current flying height of the drone;

Template matching the image taken by the drone at the current flight altitude with a reference image that matches the current flight altitude of the drone to obtain the coordinates of the takeoff point or the distance from the drone to the takeoff distance;

Controlling the drone to fly to the take-off point according to the coordinates of the take-off point or the distance of the drone from the take-off point.

In an embodiment of the present invention, during the taking-off of the drone, collecting images taken by the drone at different flight altitudes and including take-off points includes:

An image containing the take-off point is collected every preset distance.

In an embodiment of the invention, the preset distance is 1 meter.

In an embodiment of the invention, the method further includes:

During the take-off of the drone, when the flying height of the drone is greater than a preset height, the acquisition of an image including the take-off point is stopped.

In an embodiment of the invention, the preset height is 15 meters.

In an embodiment of the present invention, before the acquiring an image taken by the drone at a current flight altitude, the method further includes:

It is determined that the current flying altitude of the drone is within a preset altitude range.

In an embodiment of the present invention, the reference image matching the current flight altitude of the drone refers to a reference acquired during the take-off of the drone at the same flight altitude as the current flight altitude. image.

In an embodiment of the present invention, the reference image that matches the current flight altitude of the drone refers to the reference image acquired during the take-off of the drone at the flight height closest to the current flight altitude. Reference image.

In an embodiment of the present invention, the template matching further includes:

Determining a category of the scene in the reference image that matches the current flying height of the drone, wherein the category of the scene includes a richly textured scene or a sparsely textured scene;

Extracting a feature map of the image taken by the drone at the current flight altitude and a feature map of the reference image that matches the current flight altitude of the drone according to the category of the scene;

Then, performing template matching on the image captured by the drone at the current flight altitude with a reference image matching the current flight altitude of the drone includes:

Template matching is performed on the feature map of the image taken by the drone at the current flying height and the feature map of the reference image.

In an embodiment of the present invention, determining the category of a scene in the reference image that matches the current flying altitude of the drone includes:

Extracting a stepwise map of the reference image that matches the current flying height of the drone;

Obtaining a gradient histogram of the reference image that matches the current flying height of the drone according to the one-step graph;

Determining, according to the gradient histogram, whether a value reflecting the texture richness of the scene in the gradient histogram is greater than a first preset value;

If yes, determine that the scene in the reference image is a texture-rich scene.

In an embodiment of the present invention, the feature map of the image taken by the drone at the current flight altitude and the feature map of the reference image that matches the current flight altitude of the drone include a gradient map.

In an embodiment of the present invention, a feature map of an image taken by the drone at a current flight altitude and a feature map of the reference image that matches the current flight altitude of the drone further include a grayscale image.

In an embodiment of the present invention, if the value reflecting the texture richness of the scene in the gradient histogram is not greater than the first preset value, the method further includes:

Acquiring a two-step gradient map of the reference image that matches the current flying height of the drone;

Obtaining a gradient histogram of the reference image that matches the current flying altitude of the drone according to the two-step gradient map;

Determining, according to the gradient histogram, whether a value reflecting the texture richness of the scene in the gradient histogram is greater than a second preset value;

If yes, determine that the scene in the reference image is a sparsely textured scene.

In an embodiment of the present invention, the feature map of the image taken by the drone at the current flight altitude and the feature map of the reference image include a two-step map.

In an embodiment of the present invention, if the value of the richness of the texture reflecting the scene in the reference image in the gradient histogram is not greater than the second preset value, it is determined that the scene in the reference image is No texture scene.

To solve its technical problems, the present invention also proposes a drone landing device, which includes:

An acquisition module, configured to acquire images including take-off points taken by the drone at different flight altitudes during the take-off of the drone, and collect the acquired images including the take-off points As a reference image; and

A template matching module, configured to repeat the steps of template matching during the landing of the drone until the drone reaches the take-off point; wherein the template matching module includes:

An acquisition module, configured to acquire an image taken by the drone at a current flight height; and

Obtaining a reference image that matches the current flying height of the drone;

A matching module, configured to perform template matching between an image taken by the drone at the current flight altitude and a reference image that matches the current flight altitude of the drone to obtain the coordinates of the takeoff point or the drone Distance from the take-off point; and

A control module, configured to control the drone to fly to the take-off point according to the coordinates of the take-off point or the distance of the drone from the take-off point.

In an embodiment of the present invention, the acquisition module is specifically configured to:

An image containing the take-off point is collected every preset distance.

In an embodiment of the invention, the preset distance is 1 meter.

In an embodiment of the present invention, the acquisition module is further configured to:

During the take-off of the drone, when the flying height of the drone is greater than a preset height, no image including the take-off point is collected.

In an embodiment of the invention, the preset height is 15 meters.

In an embodiment of the present invention, the template matching module further includes:

The determining module determines that the current flying altitude of the UAV is within a preset altitude range.

In an embodiment of the present invention, the reference image matching the current flight altitude of the drone refers to a reference acquired during the take-off of the drone at the same flight altitude as the current flight altitude. image.

In an embodiment of the present invention, the reference image that matches the current flight altitude of the drone refers to the reference image acquired during the take-off of the drone at the flight height closest to the current flight altitude. Reference image.

In an embodiment of the present invention, the template matching module further includes:

A texture determination module, configured to determine a category of the scene in the reference image that matches the current flying height of the drone, wherein the category of the scene includes a rich-texture scene or a sparse-texture scene; and

An extraction module for extracting a feature map of the image taken by the drone at the current flight altitude and a feature map of the reference image that matches the current flight altitude of the drone according to the category of the scene; then :

The matching module performs module matching on a feature map of an image taken by the drone at a current flight altitude and a feature map of the reference image.

In an embodiment of the present invention, the texture determining module is specifically configured to:

Extracting a stepwise map of the reference image that matches the current flying height of the drone;

Obtaining a gradient histogram of the reference image that matches the current flying height of the drone according to the one-step graph;

Determining, according to the gradient histogram, whether a value reflecting the texture richness of the scene in the gradient histogram is greater than a first preset value;

If yes, determine that the scene in the reference image is a texture-rich scene.

In an embodiment of the present invention, the feature map of the image taken by the drone at the current flight altitude and the feature map of the reference image that matches the current flight altitude of the drone include a gradient map.

In an embodiment of the present invention, a feature map of an image taken by the drone at a current flight altitude and a feature map of the reference image that matches the current flight altitude of the drone further include a grayscale image.

In an embodiment of the present invention, the texture determining module is further configured to:

If the value in the gradient histogram reflecting the texture richness of the scene in the reference image is not greater than the first preset value;

Then, obtaining a two-step map of the reference image that matches the current flying height of the drone;

Obtaining a gradient histogram of the reference image that matches the current flying altitude of the drone according to the two-step gradient map;

Determining, according to the gradient histogram, whether a value reflecting the texture richness of the scene in the gradient histogram is greater than a second preset value;

If yes, determine that the scene in the reference image is a sparsely textured scene.

In an embodiment of the present invention, the feature map of the image taken by the drone at the current flight altitude and the feature map of the reference image include a two-step map.

In an embodiment of the present invention, the texture determining module is further configured to:

If the value reflecting the texture richness of the scene in the reference image in the gradient histogram is not greater than the second preset value, it is determined that the scene in the reference image is an untextured scene.

To solve its technical problems, the present invention also proposes a drone, including:

body;

A machine arm connected to the fuselage;

A power unit provided on the machine arm;

A processor disposed in the fuselage or arm; and

A memory communicatively connected to the processor, the memory being disposed in the fuselage or the arm; wherein,

The memory stores instructions executable by the processor, and when the processor executes the instructions, the drone landing method described above is implemented.

To solve its technical problem, the present invention also proposes a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the processor causes the processor to execute the drone landing method described above.

The invention collects the reference image including the take-off point taken by the drone during take-off, and performs phased template matching during the landing of the drone. Distance to control the drone to keep close to the take-off point during the landing process, and finally land precisely on the take-off point. During the entire landing process of the drone, no other auxiliary equipment is needed, and the landing effect is good, and the accuracy of the sensors mounted on the drone is not high.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic structural diagram of an embodiment of a drone according to the present invention.

2 is a flowchart of an embodiment of a drone landing method according to the present invention;

3 is a flowchart of one embodiment of template matching in the method shown in FIG. 2 according to the present invention;

4 is a flowchart of an embodiment of step S114 in the flowchart shown in FIG. 3;

FIG. 5 is a structural block diagram of an embodiment of a drone landing device according to the present invention.

detailed description

In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

The invention provides a method and a device for controlling a drone to accurately land at a take-off point, and a drone that can accurately land at a take-off point. With this method, precise landing of the drone can be achieved.

As shown in FIG. 1, the drone landing method of the present invention includes:

S10. During the take-off of the drone, collect images of the take-off points taken by the drone at different flight altitudes, and use the collected images of the take-off points as reference images.

The images containing the take-off point can be collected by the imaging equipment on the drone, and the captured images containing the take-off point can be stored in the drone's memory. It is worth noting that, in the present invention, the take-off point may refer to an area where the drone takes off, or a coordinate point of the take-off position of the drone. During the take-off of the drone, each time the drone reaches a certain height, the imaging equipment will collect an image containing the take-off point. In an embodiment of the present invention, an image including a take-off point is collected every preset distance. The preset distance may be determined according to needs or experience. In an embodiment of the present invention, the preset distance is 1 meter. In other possible embodiments, the preset distance may also be 2 meters, 3 meters, and the like. In addition, when the flying height of the drone is greater than a preset altitude, the acquisition of an image including the take-off point is stopped. Similarly, the preset height may also be determined according to needs or experience. In an embodiment of the present invention, the preset height is 15 meters. In other possible embodiments, when the GPS error on the drone is large, the preset height may also be set to 60 meters, 100 meters, and so on.

S11. During the landing of the drone, the template matching step is repeated until the drone reaches the take-off point.

During the landing of the drone, repeating the template matching operation can obtain the distance of the drone from the take-off point in real time, so as to overcome the sensor error of the drone and control the drone to accurately land to the take-off point.

As shown in FIG. 2, in an embodiment of the present invention, the template matching further includes:

S111. During the landing of the drone, confirm that the current flying height of the drone is within a preset altitude range.

The invention starts the template matching operation according to whether the flying height of the drone meets a preset condition. Therefore, the landing process of the drone can be divided into at least two altitude intervals based on the current flight altitude of the drone, and each altitude interval corresponds to a template matching, so it is also called a staged template matching. In other possible embodiments, template matching may be performed continuously during the entire landing of the drone, instead of performing template matching in stages.

For example, the landing process of a drone can be divided into the following three stages:

The first stage: the current flying height of the drone H≥25 meters;

The second stage: 25 meters> the current flight height of the drone H ≥ 13 meters;

The third stage: 13 meters> the current flight height of the drone H ≥ 4 meters.

That is, when it is detected that the flying height H of the drone is within any one of the above three height ranges, a template matching operation is started, and therefore, the entire landing process of the drone performs three template matching operations.

S112. Acquire an image collected by the drone at a current flying height.

S113. Obtain a reference image that matches the current flying height of the drone.

In an embodiment of the present invention, the reference image matching the current flying height of the drone refers to a reference image acquired at the same flying height as the current flying height of the drone during take-off of the drone. In an embodiment of the present invention, in step S10, an image including a take-off point is collected as a reference image at a distance of 1 meter. If the current flying height of the drone is 15 meters, the reference image that matches the current flying height of the drone is the reference image collected when the drone is flying at a height of 15 meters.

In other possible embodiments, the reference image matching the current flight altitude of the drone may also refer to a reference image acquired at a height closest to the current flight altitude of the drone during the take-off of the drone. For example, the current flying height of the drone is 25 meters. Since the reference image is no longer collected when the flying height of the drone is greater than 15 meters, the reference image that matches the current flying height of the drone is the drone at The last reference image acquired during takeoff, that is, the reference image acquired when the drone was flying at a height of 15 meters.

S114. Determine a category of a scene in the reference image that matches the current flying height of the drone, where the category of the scene includes a rich-texture scene or a sparse-texture scene.

As shown in FIG. 3, in an embodiment of the present invention, the step further includes:

S1141. Extract a step map of the reference image that matches the current flying height of the drone.

In an embodiment of the present invention, a gradient image of the reference image may be extracted using a Sobel template.

In the X direction, use the following 3 × 3 template:

-1	0	1
-2	0	2
-1	0	1

In the Y direction, use the following 3 × 3 template:

-1	-2	-1
0	0	0
1	2	1

Thus, gradient maps in the X direction and the Y direction are obtained, and then a superimposed gradient map can be obtained according to the formula: pixel = (| pixel _x | + | pixel_y |) / 2.

S1142. Obtain a gradient histogram of the reference image that matches the current flying height of the drone according to the above-mentioned step graph.

Gradient histogram is a basic technique in image processing, which can well describe the distribution of the gradient.

The specific approach is:

The median range of a gradient image is 0-199; if the step size used is 2, there are 100 bins in total, which is recorded using the array hist [100]. Iterate through the entire gradient map. Each pixel value is calculated by taking the remainder of 2 as idx, and hist [idx] is accumulated. Record the maximum value max_hist and do normalization processing: each bin is multiplied by 199.0 / max_hist, and the range of values in the last hist array is: 0-199.

S1143. According to the gradient histogram, determine whether a value in the gradient histogram that reflects the texture richness of the scene is greater than a first preset value.

After the gradient histogram is obtained, the distribution of the gradient histogram is further calculated. From 100 bins, the number greater than 15 is counted, where 15 is an empirical value and can be selected based on experience. If the number greater than 15 is counted as count_bin, then the ratio of the number of bins greater than 15 to the total number of bins can be obtained: that is, hist_ratio = count_bin / 100. hist_ratio is a value that reflects the texture richness of the scene. Take the first preset value T = 0.1 according to experience. Determine whether hist_ratio is greater than T.

S1144. If hist_ratio> T, determine that the scene in the reference image is a scene with rich texture.

S1145. If the hist_ratio is not greater than T, a two-level map of the reference image that matches the current flying height of the drone needs to be obtained.

In an embodiment of the present invention, a two-step gradient image of the reference image is still extracted by using a Sobel template.

In the X direction, use the following 3 × 3 template:

-1	0	1
-2	0	2
-1	0	1

In the Y direction, use the following 3 × 3 template:

-1	-2	-1
0	0	0
1	2	1

Thus, gradient maps in the X direction and the Y direction are obtained respectively, and then the superimposed two-level gradient map can be obtained according to the formula: pixel = (| pixel_x | + | pixel_y |) / 2.

S1146. Obtain the gradient histogram of the reference image that matches the current flying height of the drone according to the two-step graph.

This step is basically the same as step S1142, and will not be repeated here.

S1147. According to the gradient histogram, determine whether a value in the gradient histogram that reflects the texture richness of the scene is greater than a second preset value.

Statistical gradient histogram distribution. From 100 bins, count the number greater than 15, where 15 is the empirical value, which can be selected based on experience. By counting the number greater than 15 as count_bin, the ratio of bins greater than 15 to the total bins can be obtained: hist_ratio = count_bin / 100. Take the second preset value T = 0.13 according to experience. Determine whether hist_ratio is greater than T. The value of the second preset value may be the same as or different from the first preset value.

S1148: If hist_ratio> T, determine that the scene in the reference image is a scene with sparse texture.

S1149: If the hist_ratio is not greater than T, it is determined that the scene in the reference image is a scene without texture or a scene with very sparse texture, and the method of the present invention will no longer be applicable.

S115. Extract the feature map of the image collected by the drone at the current flight altitude and the reference matching the drone at the current flight altitude according to the category of the scene in the reference image. Feature map of the image.

For a scene with rich texture and a scene with sparse texture, the feature maps to be extracted in step S115 are different.

For a texture-rich scene, the feature map of the image collected by the drone at the current flying height and the feature map of the reference image that matches the current flying height of the drone include a gradient map. That is, a stepwise map of the image acquired by the drone at the current flight altitude and a stepwise map of the reference image that matches the current flight altitude of the drone.

In other possible embodiments, the feature map may further include a grayscale image of the image collected by the drone at the current flight altitude and a grayscale image of the reference image that matches the current flight altitude of the drone.

For a sparsely textured scene, the feature map of the image collected by the drone at the current flight altitude and the feature map of the reference image that matches the current flight altitude of the drone include a two-step map. That is, a two-step graph of the image acquired by the drone at the current flight altitude and a two-step graph of the reference image that matches the current flight altitude of the drone.

The invention uses different feature maps to perform template matching for different scenarios, which can improve the accuracy and robustness of template matching.

S116. Perform a template matching between the feature map of the image taken by the drone at the current flight height and the feature map of the reference image to obtain the coordinates of the take-off point or the distance of the drone from the take-off point. .

Template matching requires that the scale of the current image and the reference image be approximately equal. Therefore, before template matching, two images need to be pre-processed:

Taking the current flight altitude of the drone at 25 meters, and taking the image collected by the drone at a flight height of 13 meters as a reference image, the scales are approximately equal. If the drone is at a height of 13 meters, The collected reference image and the current image collected by the drone at a flight height of 25 meters both contain the same object, so the image area of this object is basically the same in the two images. Because in the camera model, near objects are large and distant objects are small, the reference image collected by the drone at a flight height of 13 meters is down-sampled according to 13/25 = 0.52 to obtain a new image, then The new image is at the same scale as the current image collected by the drone at a flight height of 25 meters.

Yaw angle estimation: Taking the current yaw angle of the drone as 0 ° as an example, the current image collected is used as a reference, and the current image is turned counterclockwise from -18 ° to 18 °, and every 3 °, a template is made. Match and get a response value. In the 13 template matching results, the yaw angle angle corresponding to the maximum response value is found. At this time, the yaw angle of the drone is corrected to: the current yaw angle of the drone + angle.

Altitude estimation: The error of the altitude is from [-1,1], the step length is 0.5 meters, and a total of 5 template matchings are performed. The maximum corresponding height error is delta_z. At this time, the flying height of the drone is modified as: no one The aircraft's current flight altitude + delta_z.

Calculate the horizontal distance: The reference image collected by the drone at a flight height of 13 meters is pre-processed to obtain an image of the same scale as the current image collected at a flight height of 25 meters. Using the template matching algorithm, the corresponding maximum response value is found. , And convert it to the world coordinate system according to the attitude of the drone: assuming the origin of the pixel coordinates is the optical center position, the pixel coordinates of the matching result are (Δu, Δv), and the camera's internal parameter matrix (including the focal length only) is K , The aircraft rotation matrix is R, and the calculated coordinates of the takeoff point in the world coordinate system are (X, Y, Z), then:

(X, Y, h) ^T = R ^T K ^-1 s (Δu, Δv, 1) ^T

According to the current flying height h of the drone, find the scale s, so that accurate X and Y can be obtained, that is, the horizontal distance of the drone in the X and Y directions from the takeoff point.

S117. Control the drone to fly to the take-off point according to the coordinates of the take-off point or the distance from the drone to the take-off point.

The invention collects the reference image including the take-off point taken by the drone during take-off, and performs phased template matching during the landing of the drone. Distance to control the drone to keep close to the take-off point during the landing process, and finally land precisely on the take-off point.

As shown in FIG. 4, the present invention also proposes a drone landing device 20. The device 20 includes:

An acquisition module 21 is configured to collect images including take-off points taken by the drone at different flight altitudes during the take-off of the drone, and collect the collected images including the take-off points. Images as reference images; and

A template matching module 22 is configured to repeat the steps of template matching during the landing of the drone until the drone reaches the take-off point; wherein the template matching module 22 includes:

An acquisition module 221, configured to acquire an image taken by the drone at a current flying height; and

Obtaining a reference image that matches the current flying height of the drone;

A matching module 225, configured to perform template matching between an image taken by the drone at the current flight altitude and a reference image that matches the current flight altitude of the drone to obtain the coordinates of the takeoff point or the drone The distance of the aircraft from said take-off point; and

A control module 226 is configured to control the drone to fly to the take-off point according to the coordinates of the take-off point or the distance from the drone to the take-off point.

Optionally, the acquisition module 21 is specifically configured to:

An image containing the take-off point is collected every preset distance.

Optionally, the preset distance is 1 meter.

Optionally, the acquisition module 21 is further configured to:

During the take-off of the drone, when the flying height of the drone is greater than a preset height, no image including the take-off point is collected.

Optionally, the preset height is 15 meters.

Optionally, the template matching module 22 further includes:

The determining module 222 determines that the flying height of the drone is within a preset height range.

Optionally, the reference image that matches the current flight altitude of the drone refers to a reference image acquired at the same flight altitude as the current flight altitude during the takeoff of the drone.

Optionally, the reference image matching the current flying height of the drone refers to a reference image acquired at a flying height closest to the current flying height during take-off of the drone.

Optionally, the template matching module 22 further includes:

The texture determining module 223 is configured to determine a category of a scene in the reference image that matches the current flying height of the drone, where the category of the scene includes a rich-texture scene or a sparse-texture scene; and

An extraction module 224, configured to extract a feature map of the image taken by the drone at the current flight altitude and a feature map of the reference image that matches the current flight altitude of the drone according to the category of the scene; then:

The matching module 225 performs module matching on a feature map of an image taken by the drone at a current flight altitude and a feature map of the reference image.

Optionally, the texture determining module 223 is specifically configured to:

Extracting a stepwise map of the reference image that matches the current flying height of the drone;

Obtaining a gradient histogram of the reference image that matches the current flying height of the drone according to the one-step graph;

Determining, according to the gradient histogram, whether a value reflecting the texture richness of the scene in the gradient histogram is greater than a first preset value;

If yes, determine that the scene in the reference image is a texture-rich scene.

Optionally, a feature map of an image taken by the drone at the current flight altitude and a feature map of the reference image that matches the current flight altitude of the drone include a step map.

Optionally, the feature map of the image taken by the drone at the current flight altitude and the feature map of the reference image that matches the current flight altitude of the drone further include a grayscale image.

Optionally, the texture determining module 223 is further configured to:

If the value in the gradient histogram reflecting the texture richness of the scene in the reference image is not greater than the first preset value;

Then, obtaining a two-step map of the reference image that matches the current flying height of the drone;

Obtaining a gradient histogram of the reference image that matches the current flying altitude of the drone according to the two-step gradient map;

Determining, according to the gradient histogram, whether a value reflecting the texture richness of the scene in the gradient histogram is greater than a second preset value;

If yes, determine that the scene in the reference image is a sparsely textured scene.

Optionally, the feature map of the image captured by the drone at the current flight altitude and the feature map of the reference image include a two-step map.

Optionally, the texture determining module 223 is further configured to:

If the value reflecting the texture richness of the scene in the reference image in the gradient histogram is not greater than the second preset value, it is determined that the scene in the reference image is an untextured scene.

In the embodiment of the present invention, the acquisition module 21 may be an imaging device mounted on a drone, such as a camera. The template matching module 22 may be a processor on a drone or a field programmable logic gate array (Field Programmable Gate Array, FPGA). The acquisition module 221, the matching module 225, and the control module 226 may be flight control chips of the drone. The determination module 222 may be a height sensor of the drone, and the texture determination module 223 may be a vision chip of the drone.

In addition, for the detailed functions of the modules in the device, please refer to the description in the drone landing method, which will not be repeated here.

The invention also proposes a drone 30. As shown in FIG. 5, the drone 30 includes a fuselage 31, a boom 32 connected to the fuselage 31, a power unit 33 provided at one end of the boom 32, and A gimbal 35 connected to the body 31, an imaging device 34 connected to the gimbal 35, and a processor 36 and a memory 37 provided in the body 31.

In this embodiment, the number of the arms 32 is four, that is, the aircraft is a quadrotor. In other possible embodiments, the number of the arms 32 may also be three, six, eight, ten, and the like. The drone 30 may also be other movable objects, such as a manned aircraft, an aircraft model, an unmanned airship, a fixed-wing drone, an unmanned hot air balloon, and the like.

The power unit 33 includes a motor 332 provided at one end of the arm 32 and a propeller 331 connected to a rotating shaft of the motor 332. The rotating shaft of the motor 332 rotates to drive the propeller 331 to rotate to provide lift to the drone 30.

The pan / tilt head 35 is used to reduce or even eliminate the vibration transmitted from the power unit 33 to the imaging device 34 to ensure that the imaging device 34 can shoot a stable and clear image or video.

The imaging device 34 may be a binocular camera, a monocular camera, an infrared imaging device, an ultraviolet imaging device, a camcorder, or the like. The imaging device 34 can be directly mounted on the drone 30, or can be mounted on the drone 30 through the gimbal 35 as shown in this embodiment. The gimbal 35 allows the imaging device 34 to surround at least one relative to the drone 30 The shaft turns.

The processor 36 may include multiple functional units, such as a flight control unit for controlling the flight attitude of the aircraft, a target recognition unit for identifying a target, a tracking unit for tracking a specific target, and a navigation unit for navigating the aircraft ( For example, GPS (Global Positioning System), Beidou, and a data processing unit used to process environmental information acquired by relevant airborne equipment (such as imaging equipment 34).

A computer program is stored in the memory 36. When the computer program is executed by the processor, the processor causes the processor to execute the methods described in the embodiments shown in FIGS. 1-3.

The present invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor is caused to execute the method described in the embodiment shown in FIG. 1 to FIG. 3. .

A person of ordinary skill in the art can understand that all or part of the processes in the methods of the foregoing embodiments can be implemented by using a computer program to instruct related hardware. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it may include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or the like.

The technical features of the embodiments described above can be arbitrarily combined. In order to simplify the description, all possible combinations of the technical features in the above embodiments have not been described. However, as long as there is no contradiction in the combination of these technical features, It should be considered as the scope described in this specification.

The above-mentioned embodiment only expresses several implementation manners of the present invention, and the description thereof is more specific and detailed, but it cannot be understood as a limitation on the scope of the invention patent. It should be noted that, for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can be made, which all belong to the protection scope of the present invention. Therefore, the protection scope of the invention patent shall be subject to the appended claims.

Claims (32)

1. A drone landing method, characterized in that the method includes:

   During the take-off of the drone, collecting images including take-off points taken by the drone at different flight altitudes, and using the collected images containing the take-off points as reference images;

   During the landing of the drone, the step of template matching is repeated until the drone reaches the take-off point; wherein the template matching includes:

   Acquiring an image taken by the drone at a current flying height;

   Obtaining a reference image that matches the current flying height of the drone;

   Template matching the image taken by the drone at the current flight altitude with a reference image that matches the current flight altitude of the drone to obtain the coordinates of the takeoff point or the distance from the drone to the takeoff distance;

   Controlling the drone to fly to the take-off point according to the coordinates of the take-off point or the distance of the drone from the take-off point.
2. The method for landing a drone according to claim 1, wherein, during the taking off of the drone, collecting images taken by the drone at different flight altitudes and including take-off points comprises:

   An image containing the take-off point is collected every preset distance.
3. The method for landing a drone according to claim 2, wherein the preset distance is 1 meter.
4. The method for landing a drone according to any one of claims 1-3, further comprising:

   During the take-off of the drone, when the flying height of the drone is greater than a preset height, the acquisition of an image including the take-off point is stopped.
5. The method for landing a drone according to claim 4, wherein the preset height is 15 meters.
6. The method for landing a drone according to any one of claims 1-5, wherein before the acquiring an image taken by the drone at a current flight altitude, the method further comprises:

   It is determined that the current flying altitude of the drone is within a preset altitude range.
7. The method for landing a drone according to any one of claims 1-6, wherein the reference image matching the current flight altitude of the drone refers to a process in which the drone takes off during A reference image acquired at the same flying height as the current flying height.
8. The method for landing a drone according to any one of claims 1-6, wherein the reference image that matches the current flight altitude of the drone refers to during the take-off of the drone, during A reference image acquired at a flight height closest to the current flight height.
9. The method for landing a drone according to any one of claims 1 to 8, wherein the template matching further comprises:

   Determining a category of the scene in the reference image that matches the current flying height of the drone, wherein the category of the scene includes a richly textured scene or a sparsely textured scene;

   Extracting a feature map of the image taken by the drone at the current flight altitude and a feature map of the reference image that matches the current flight altitude of the drone according to the category of the scene;

   Then, performing template matching on the image captured by the drone at the current flight altitude with a reference image matching the current flight altitude of the drone includes:

   Template matching is performed on the feature map of the image taken by the drone at the current flying height and the feature map of the reference image.
10. The drone landing method according to claim 9, wherein the determining a category of a scene in the reference image that matches the current flight altitude of the drone comprises:

    Extracting a stepwise map of the reference image that matches the current flying height of the drone;

    Obtaining a gradient histogram of the reference image that matches the current flying height of the drone according to the one-step graph;

    Determining, according to the gradient histogram, whether a value reflecting the texture richness of the scene in the gradient histogram is greater than a first preset value;

    If yes, determine that the scene in the reference image is a texture-rich scene.
11. The method for landing a drone according to claim 10, wherein a feature map of an image taken by the drone at a current flying height and a feature of the reference image that matches the current flying height of the drone The diagram includes a step diagram.
12. The method for landing a drone according to claim 11, wherein a feature map of an image taken by the drone at a current flight altitude and a feature of the reference image that matches the current flight altitude of the drone The image also includes a grayscale image.
13. The method for landing a drone according to any one of claims 10-12, wherein if a value reflecting the richness of the texture of the scene in the gradient histogram is not greater than the first preset value, The method further includes:

    Acquiring a two-step gradient map of the reference image that matches the current flying height of the drone;

    Obtaining a gradient histogram of the reference image that matches the current flying altitude of the drone according to the two-step gradient map;

    Determining, according to the gradient histogram, whether a value reflecting the texture richness of the scene in the gradient histogram is greater than a second preset value;

    If yes, determine that the scene in the reference image is a sparsely textured scene.
14. The method for landing a drone according to claim 13, wherein a feature map of an image taken by the drone at a current flight altitude and a feature map of the reference image include a two-step map.
15. The method for landing a drone according to claim 13 or 14, characterized in that, if the value of the richness of the texture reflecting the scene in the reference image in the gradient histogram is not greater than the second preset value, It is determined that the scene in the reference image is a textureless scene.
16. A drone landing device, characterized in that the device includes:

    An acquisition module, configured to acquire images including take-off points taken by the drone at different flight altitudes during the take-off of the drone, and collect the acquired images including the take-off points As a reference image; and

    A template matching module, configured to repeat the steps of template matching during the landing of the drone until the drone reaches the take-off point; wherein the template matching module includes:

    An acquisition module, configured to acquire an image taken by the drone at a current flight height; and

    Obtaining a reference image that matches the current flying height of the drone;

    A matching module, configured to perform template matching between an image taken by the drone at the current flight altitude and a reference image that matches the current flight altitude of the drone to obtain the coordinates of the takeoff point or the drone Distance from the take-off point; and

    A control module, configured to control the drone to fly to the take-off point according to the coordinates of the take-off point or the distance of the drone from the take-off point.
17. The drone landing device according to claim 1, wherein the acquisition module is specifically configured to:

    An image containing the take-off point is collected every preset distance.
18. The drone landing device according to claim 17, wherein the preset distance is 1 meter.
19. The drone landing device according to any one of claims 16 to 18, wherein the acquisition module is further configured to:

    During the take-off of the drone, when the flying height of the drone is greater than a preset height, no image including the take-off point is collected.
20. The drone landing device according to claim 19, wherein the preset height is 15 meters.
21. The drone landing device according to any one of claims 16-20, wherein the template matching module further comprises:

    The determining module determines that the current flying altitude of the UAV is within a preset altitude range.
22. The drone landing device according to any one of claims 16 to 21, wherein the reference image matching the current flight altitude of the drone refers to a time during which the drone takes off, during A reference image acquired at the same flying height as the current flying height.
23. The drone landing device according to any one of claims 16 to 21, wherein the reference image that matches the current flight altitude of the drone refers to that during the take-off of the drone, A reference image acquired at a flight height closest to the current flight height.
24. The drone landing device according to any one of claims 16 to 23, wherein the template matching module further comprises:

    A texture determination module, configured to determine a category of the scene in the reference image that matches the current flying height of the drone, wherein the category of the scene includes a rich-texture scene or a sparse-texture scene; and

    An extraction module for extracting a feature map of the image taken by the drone at the current flight altitude and a feature map of the reference image that matches the current flight altitude of the drone according to the category of the scene; then :

    The matching module performs module matching on a feature map of an image taken by the drone at a current flight altitude and a feature map of the reference image.
25. The drone landing device according to claim 24, wherein the texture determining module is specifically configured to:

    Extracting a stepwise map of the reference image that matches the current flying height of the drone;

    Obtaining a gradient histogram of the reference image that matches the current flying height of the drone according to the one-step graph;

    Determining, according to the gradient histogram, whether a value reflecting the texture richness of the scene in the gradient histogram is greater than a first preset value;

    If yes, determine that the scene in the reference image is a texture-rich scene.
26. The drone landing device according to claim 25, wherein a feature map of an image taken by the drone at a current flying height and a feature of the reference image that matches the current flying height of the drone The diagram includes a step diagram.
27. The drone landing device according to claim 26, wherein a feature map of an image taken by the drone at a current flying height and a feature of the reference image that matches the current flying height of the drone The image also includes a grayscale image.
28. The drone landing device according to any one of claims 25-27, wherein the texture determining module is further configured to:

    If the value in the gradient histogram reflecting the texture richness of the scene in the reference image is not greater than the first preset value;

    Then, obtaining a two-step map of the reference image that matches the current flying height of the drone;

    Obtaining a gradient histogram of the reference image that matches the current flying altitude of the drone according to the two-step gradient map;

    Determining, according to the gradient histogram, whether a value reflecting the texture richness of the scene in the gradient histogram is greater than a second preset value;

    If yes, determine that the scene in the reference image is a sparsely textured scene.
29. The drone landing device according to claim 28, wherein the feature map of the image captured by the drone at the current flight altitude and the feature map of the reference image include a two-step map.
30. The drone landing device according to claim 28 or 29, wherein the texture determining module is further configured to:

    If the value reflecting the texture richness of the scene in the reference image in the gradient histogram is not greater than the second preset value, it is determined that the scene in the reference image is an untextured scene.
31. A drone characterized by comprising:

    body;

    A machine arm connected to the fuselage;

    A power unit provided on the machine arm;

    A processor disposed in the fuselage or arm; and

    A memory communicatively connected to the processor, the memory being disposed in the fuselage or the arm; wherein,

    The memory stores instructions executable by the processor, and when the processor executes the instructions, the method according to any one of claims 1-15 is implemented.
32. A computer-readable storage medium, characterized in that a computer program is stored, and when the computer program is executed by a processor, the processor causes the processor to execute the method according to any one of claims 1 to 15.

Images