CN108820233B - Visual landing guiding method for fixed-wing unmanned aerial vehicle - Google Patents
Visual landing guiding method for fixed-wing unmanned aerial vehicle Download PDFInfo
- Publication number
- CN108820233B CN108820233B CN201810730693.8A CN201810730693A CN108820233B CN 108820233 B CN108820233 B CN 108820233B CN 201810730693 A CN201810730693 A CN 201810730693A CN 108820233 B CN108820233 B CN 108820233B
- Authority
- CN
- China
- Prior art keywords
- unmanned aerial
- aerial vehicle
- angle
- landing
- video
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 230000000007 visual effect Effects 0.000 title claims abstract description 17
- 238000003384 imaging method Methods 0.000 claims abstract description 9
- 230000008859 change Effects 0.000 claims abstract description 6
- 238000012937 correction Methods 0.000 claims description 39
- 238000012549 training Methods 0.000 claims description 11
- 238000013527 convolutional neural network Methods 0.000 abstract description 38
- 238000005516 engineering process Methods 0.000 abstract description 23
- 238000013135 deep learning Methods 0.000 abstract description 3
- 230000008569 process Effects 0.000 description 16
- 238000011160 research Methods 0.000 description 13
- 238000000605 extraction Methods 0.000 description 3
- 230000014509 gene expression Effects 0.000 description 3
- 238000009434 installation Methods 0.000 description 3
- 206010063385 Intellectualisation Diseases 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000010485 coping Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 231100000817 safety factor Toxicity 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64D—EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
- B64D45/00—Aircraft indicators or protectors not otherwise provided for
- B64D45/04—Landing aids; Safety measures to prevent collision with earth's surface
- B64D45/08—Landing aids; Safety measures to prevent collision with earth's surface optical
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U10/00—Type of UAV
- B64U10/25—Fixed-wing aircraft
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U70/00—Launching, take-off or landing arrangements
- B64U70/60—Take-off or landing of UAVs from a runway using their own power
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64U—UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR
- B64U2201/00—UAVs characterised by their flight controls
- B64U2201/10—UAVs characterised by their flight controls autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Remote Sensing (AREA)
- Mechanical Engineering (AREA)
- Image Analysis (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The invention provides a visual landing guiding method for a fixed wing unmanned aerial vehicle, which comprises the following steps: the method comprises the steps of extracting time-space domain feature points of adjacent multi-frame images in a video collected by imaging equipment of the fixed-wing unmanned aerial vehicle in the landing stage by using a 3D CNN, and controlling the unmanned aerial vehicle to land autonomously by calculating position change information between the feature points of the adjacent multi-frame images which can be matched with each other. The invention provides a fixed wing unmanned aerial vehicle visual landing guidance method based on a convolutional neural network, wherein image information mainly researched is obtained by an airborne visual system, identification of a runway is not assisted by a ground auxiliary mark, and technologies mainly depended on comprise computer vision and deep learning technologies. The imaging equipment is used as a passive sensor to collect rich landing scene information, and the imaging equipment is light in weight and is very suitable for the characteristic of weak bearing capacity of the unmanned aerial vehicle; meanwhile, the algorithm is realized without the aid of ground auxiliary marks and does not depend on the operation environment, so that the reliability of visual landing guidance of the fixed-wing unmanned aerial vehicle is greatly improved.
Description
Technical Field
The invention relates to the technical field of autonomous landing guidance of fixed-wing unmanned aerial vehicles, in particular to a visual landing guidance method of a fixed-wing unmanned aerial vehicle.
Background
An Unmanned Aerial Vehicle (UAV) is an Unmanned Aerial Vehicle that is programmed to fly. The unmanned aerial vehicle technology has been widely applied to a plurality of fields such as military affairs, communication, surveying and mapping, environment, traffic and the like. In order to further promote the development of the unmanned aerial vehicle technology towards automation and intellectualization, autonomous control becomes a hotspot of current research on the unmanned aerial vehicle technology.
The flight mission of the drone comprises three phases: takeoff, flight and landing phases. The work of the takeoff and flight phases is easier to finish than that of the landing phase, when the takeoff condition is met, the takeoff can be realized through simpler program control, and the requirement on the autonomy of the system is not high; the flight phase is also relatively simple under the guidance of the navigation system. The landing stage is relatively complex, and because the unmanned aerial vehicle moves at a high speed when landing, the operation is complex, the ground interference factors are many, so that the probability of accidents occurring in the stage is often higher than that in the other two stages. In order to further promote the intellectualization and combine with the consideration of safety factors, the autonomous landing becomes one of the problems which need to be solved urgently, and the research on the key technology in the autonomous landing process has great value.
Unmanned aerial vehicle mainly falls into two kinds according to organism structure's characteristics: fixed wing unmanned aerial vehicle and rotor unmanned aerial vehicle. For a fixed wing unmanned aerial vehicle taking off and landing in a wheel type undercarriage running mode, the requirements on navigation accuracy, reliability and safety in the landing process are high. According to statistics, although the landing stage only takes 2% to 3% of the time of the whole flight mission, the number of faults generated is about 80% of the total number of faults in the whole flight mission, so that the improvement of the landing reliability and the guidance precision becomes an important research task.
The current research situation of unmanned aerial vehicle autonomous landing technology:
the autonomous flight landing system which is globally used at present and meets the requirements of the International Civil Aviation Organization (ICAO) mainly comprises a microwave instrument landing system (MLS), a combined precise autonomous landing system (JPALS) and a millimeter wave monopulse secondary radar guidance system (PLS).
Microwave Landing System (MLS)
The MLS works on the principle that the airborne equipment receives a guiding signal transmitted by the ground equipment, and then the airborne equipment calculates the position relation of the airplane relative to the runway to carry out landing guidance. The MLS employs a phased array antenna, the equipment is expensive, and the future development direction mainly focuses on miniaturization and cost reduction of the equipment.
Millimeter wave secondary radar landing system (PLS)
The PLS is mainly used for the take-off and landing guidance of the unmanned aerial vehicle and consists of airborne equipment and ground equipment. PLS has been widely used in drones in the united states. PLS in China is still in an experimental stage at present, and mature equipment is not available yet.
The advantages of PLS are:
1, the guiding precision is high, and the angle measurement precision is irrelevant to the distance;
2 the device has small size and convenient installation and erection, and is suitable for maneuvering and wartime use.
Disadvantages of PLS are:
1 is greatly affected by weather. The influence of rain and fog is obvious, and the acting distance is greatly reduced under severe meteorological conditions;
and 2, the ground equipment calculates navigation information and transmits the navigation information to the airplane through a data link, and the time delay of the navigation information is large.
Combined precision autonomous landing system (JPALS)
JPALS is a military high-precision autonomous landing guidance system developed by the united states that can guide an aircraft to land autonomously with zero visibility. JPALS is based on GPS, the vertical precision can reach 0.3m, which is much higher than the index required by the civil aviation three-class blind landing system. Currently, the united states forces have successfully integrated JPALS on fighters, and future support for drones will improve.
Unmanned aerial vehicle autonomous landing technology based on vision
The vision-based autonomous landing technology becomes a research hotspot in recent years, and is a breakthrough for pursuing international advanced technology in China.
The vision-based unmanned aerial vehicle autonomous landing technology mainly utilizes an imaging system to acquire image information, completes information analysis through an intelligent computing system and guides landing.
At present, the application of the vision autonomous guiding technology in intelligent vehicle navigation is the most mature, an image acquisition device is installed on a vehicle, the height, the pitch angle and the roll angle of the image acquisition device are all fixed and known, and an unknown variable is only a lateral deviation angle relative to a road center line.
Compared with other autonomous landing schemes, the unmanned aerial vehicle autonomous landing technology based on vision has the advantages of simple equipment, passive information, capability of coping with electronic countermeasures and the like, but in the unmanned aerial vehicle autonomous landing process, a yaw angle, a pitch angle and a roll angle are not fixed and unchanged, and the change parameters have mutual influence, so that the difficulty of the algorithm is far higher than that of intelligent vehicle navigation.
Unmanned aerial vehicle mainly falls into two kinds according to organism structure's characteristics: fixed wing unmanned aerial vehicle and rotor unmanned aerial vehicle.
The gyroplane generally has the capability of hovering at any position, turning in a right-angle plane, vertically lifting and the like, and generally utilizes a vertical autonomous landing method for recovery processing. The main working flow is as follows: firstly, the aircraft flies above a landing field under the guidance of a navigation system, and then the landing is realized by vertical take-off and landing. Consequently, rotor unmanned aerial vehicle's vision landing scheme is comparatively easy to be realized.
The main work flow of the fixed wing unmanned aerial vehicle for autonomous landing is as follows: after the target airspace is reached, runway search is carried out, the position relation between the unmanned aerial vehicle and the runway is calculated, and the position relation is provided for a flight control system to carry out position and pose control and landing by sliding. In the period, if the runway condition cannot be identified or the pose of the unmanned aerial vehicle cannot meet the safe landing requirement, the unmanned aerial vehicle is pulled up immediately, and the runway search is carried out again, otherwise, the safety of the unmanned aerial vehicle is possibly influenced.
The typical research results of domestic and foreign research institutions are as follows:
in 2004, Cherian Anoop et al mapped the texture and altitude information of the image by a machine learning method to estimate the altitude of the unmanned aerial vehicle when landing.
In 2008, Sukhaltm et al adopt a combined navigation mode, and utilize extended Kalman filtering to fuse visual information and inertial navigation information, so as to obtain more accurate position and attitude parameters of the unmanned aerial vehicle.
In 2008, Kelly and the like adopt a mode of integrating stereoscopic vision measurement and an inertial navigation sensor to design an unmanned aerial vehicle navigation system based on natural landmarks.
In 2010, Wenzel et al used a miniature thermal infrared imager to detect the spots on the ground and thus resolve the relative position of the drone.
In 2012, Vladimir et al proposed a linear tracking method based on Hough transform of an area of interest to guide an unmanned aerial vehicle to land autonomously.
In 2014, the korean space engineering department proposed a vision-guided landing system applied to a small unmanned aerial vehicle, which guides the unmanned aerial vehicle to fly into a red dome-shaped safety airbag installed on the ground through vision. The unmanned aerial vehicle obtains the position of the landing safety airbag through color identification.
Through the above analysis, the current research situation is summarized as follows:
the MLS is widely used in the military field of China, is mainly used for landing by an unmanned plane, has mature products and technology, and can meet the landing guidance requirement of the unmanned plane by main performance indexes. The MLS has the advantages of high guiding precision and the defects that: the price is expensive, the ground equipment is large in size, the requirements for site environment for installation and erection are high, and quick installation and erection cannot be completed.
The PLS device is small in size, convenient to carry, has outstanding advantages in the aspect of unmanned aerial vehicle landing, and is widely used in the United states. However, PLS in China is still in an experimental stage at present, and mature equipment is not available yet.
3. The united states forces have successfully integrated JPALS on fighters, and the next step will improve the support for drones. At present, China has a large difference between the research and development aspect of a GPS-based landing system and the JPALS of the American army.
4. In the aspect of research on landing technology of a vision-based fixed wing unmanned aerial vehicle, although research institutions have achieved a lot of achievements, distance engineering realization still needs to be deeply researched. The whole landing process of the fixed-wing unmanned aerial vehicle is faster than that of a rotorcraft, and longer space distance is needed, so that the requirements on the real-time performance and the accuracy of an algorithm in the landing process are high, and the difficulty that the fixed-wing unmanned aerial vehicle completes autonomous landing by using a vision technology becomes.
The unmanned aerial vehicle autonomous landing technology based on vision is a research hotspot in recent years and may become a breakthrough for China to catch up with the international advanced autonomous flight landing technology.
Runway area identification belongs to the research category of image classification technology. The main application fields of the image classification technology include: automatic image annotation, content-based image retrieval, video monitoring, medical image processing and the like. Although a plurality of efficient algorithms are abundantly developed in recent years, and the accuracy of image classification is continuously improved, the problem of semantic gap still exists, namely, the bottom layer visual information of the image is difficult to accurately express as human understandable high-level semantics; and the images from different sources are difficult to be described by the same feature extraction method.
Common image classification methods include image space-based methods and feature space-based methods. At present, most of unmanned aerial vehicle autonomous landing research schemes based on vision adopt an image classification method based on a feature space, most of the recognition schemes aiming at real runway targets are realized by means of ground auxiliary marks, the schemes have great dependence on the operating environment, and the reliability of the algorithm can be improved only under specific environmental conditions. The image classification method based on the feature space can reduce the computational complexity, but the accuracy depends on the quality of feature extraction. The traditional image classification method only aims at a specific classification task of a certain class, and when the data needing to be classified changes, a great deal of effort is needed to find a new solution, so that the method does not have good generalization.
Disclosure of Invention
Aiming at the defects, the invention provides a method for solving the runway area tracking problem by adopting a 3D Convolutional Neural Network (CNN). CNN is an efficient recognition method that has been developed in recent years and has attracted considerable attention. The CNN feature detection layer learns the classification features through training data, explicit feature extraction can be avoided, and parallel acceleration hardware such as a GPU or an FPGA can be used for training acceleration. For video information processing, 2D CNN has the following disadvantages: the 2D CNN is focused on the description of a single frame image, the time sequence relation between adjacent frame images cannot be extracted, and when the problem of sensitive time sequence information is processed, the 2D CNN only performs two-dimensional convolution and pooling, so that the model loses time information. The C3D convolutional neural network adopted by the invention is a common 3D CNN, and in each convolution or pooling process, local information of adjacent frames of images is considered at the same time, so that the time sequence relation in video information can be transferred layer by layer. Therefore, the C3D convolutional neural network is selected herein to extract feature points in the landing video.
A fixed wing drone visual landing guidance method includes: the method comprises the steps of extracting time-space domain feature points of adjacent multi-frame images in a video collected by imaging equipment of the fixed-wing unmanned aerial vehicle in the landing stage by using a 3D CNN, and controlling the unmanned aerial vehicle to land autonomously by calculating position change information between the feature points of the adjacent multi-frame images which can be matched with each other.
Further, the method as described above, comprising: correcting a pose control parameter based on the 3D CNN and tracking a close-distance runway based on the 3D CNN;
the pose control parameter based on the 3D CNN is corrected as follows: correcting the pose control parameters by using a 3D CNN-based method aiming at the acquired images corresponding to the adjacent frames; the pose control parameters include: matching the roll angle, the pitch angle and the yaw angle of the characteristic points based on adjacent frames;
the close-range runway tracking based on 3D CNN is as follows: and performing real-time correction on the pose control parameters by adopting the method based on the 3D CNN on the continuous frames in the acquired video.
Further, in the method described above, the slip angle correction algorithm is represented by the following equation:
αnew＝αold+δα
where k is the adjustment coefficient, obtained by extensive video training, δαIs a correction angle for the slip angle; d is the lateral offset of the matched feature points in the adjacent frames of the video, h is the longitudinal offset of the matched feature points in the adjacent frames of the video, alpha is the slip angle, alpha isnewFor corrected slip angle, αoldIs the slip angle before correction.
Further, as in the method described above, the roll angle correction algorithm is represented by the following equation:
δβ＝-lλ
βnew＝βold+δβ
wherein l is an adjustment coefficient obtained through a large amount of video training, beta is a roll angle, and deltaβFor correcting the roll angle, betanewFor the corrected roll angle, betaoldFor roll angle before correction, λ is the relative rotation angle between matching feature points in adjacent frames of the video.
Further, in the method described above, the pitch angle correction algorithm is represented by the following equation:
δγ＝-mh
γnew＝γold+δγ
wherein m is an adjustment coefficient and is obtained through a large amount of video training; h is the longitudinal shift, delta, of the matching feature points in adjacent frames of the videoγModified angle, gamma, of pitch anglenewFor corrected pitch angle, gammaoldTo correct the pitch angle before.
Has the advantages that:
according to the visual landing guidance method of the fixed wing unmanned aerial vehicle based on the convolutional neural network, mainly researched image information is obtained by an airborne vision system, runway identification is realized without the aid of ground auxiliary marks, and technologies mainly depending on the ground vision and deep learning technologies comprise computer vision and deep learning technologies. The imaging equipment is used as a passive sensor to collect rich landing scene information, and the imaging equipment is light in weight and is very suitable for the characteristic of weak bearing capacity of the unmanned aerial vehicle; meanwhile, the algorithm is realized without the aid of ground auxiliary marks and does not depend on the operation environment, so that the reliability of visual landing guidance of the fixed-wing unmanned aerial vehicle is greatly improved.
Drawings
FIG. 1 illustrates a slip angle α between the main axis of a fixed wing drone and a runway;
FIG. 2 illustrates the shift of matched feature points in adjacent frames of a video due to yaw and pitch angles, d being the lateral shift of matched feature points in adjacent frames of a video, and h being the longitudinal shift of matched feature points in adjacent frames of a video;
fig. 3 a roll angle β of the fixed wing drone;
FIG. 4 is a graph of the rotation of matched feature points in adjacent frames of a video caused by roll angle β;
FIG. 5 landing process flow diagram;
fig. 6 is a pitch angle γ of the fixed-wing drone;
fig. 7 is a flowchart of a visual landing guidance method for a fixed-wing drone according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention are described below clearly and completely, and it is obvious that the described embodiments are some, not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 7, the method for guiding visual landing of a fixed-wing drone provided by the present invention includes: the method comprises the steps of extracting time-space domain feature points of adjacent multi-frame images in a video collected by imaging equipment of the fixed-wing unmanned aerial vehicle in the landing stage by using a 3D CNN, and controlling the unmanned aerial vehicle to land autonomously by calculating position change information between the feature points of the adjacent multi-frame images which can be matched with each other.
Specifically, the fixed wing drone has the following characteristics in the autonomous landing stage: because the distance from the runway is very close in the landing process, the flying speed is higher than that of the gyroplane, the video images collected by the airborne camera are only one part of the runway in the advancing direction, the images change rapidly along with the advancing of the unmanned aerial vehicle, and each frame of image is only partially overlapped with the previous frame of image. Since the 3D CNN can apply a convolution kernel to the time-space domain, feature points of adjacent multi-frame images can be extracted using the 3D CNN. After the feature points of the adjacent multi-frame images are extracted, the feature points which can be matched with each other at the overlapped part of the adjacent multi-frame images are calculated through a feature point matching algorithm, and the parameters required by controlling the pose of the fixed-wing unmanned aerial vehicle in the autonomous landing stage can be obtained through the position relation information between the feature points.
The 3D CNN adopted by the invention is a C3D convolutional neural network. The C3D convolutional neural network can extract features that are discriminative in both the temporal and spatial dimensions by performing 3D convolution on convolutional layers of CNN.
Further, the visual landing guidance method for the fixed-wing unmanned aerial vehicle provided by the invention comprises the following steps: correcting a pose control parameter based on the 3D CNN and tracking a close-distance runway based on the 3D CNN;
the pose control parameter based on the 3D CNN is corrected as follows: correcting the pose control parameters by using a 3D CNN-based method aiming at the acquired images corresponding to the adjacent frames;
the close-range runway tracking based on 3D CNN is as follows: and performing real-time correction on the pose control parameters by adopting the method based on the 3D CNN on the continuous frames in the acquired video.
Here, it should be noted that: the pose control parameter correction method based on the 3D CNN is a correction algorithm aiming at adjacent frames, and an operation object is two images; the short-distance runway tracking method based on the 3D CNN is characterized in that a 3D CNN-based pose control parameter correction method is continuously used in the whole landing process according to the continuity of videos, and an operation object is the whole landing process video.
Wherein the pose control parameters are as follows: the slip angle between the main shaft of the fixed-wing unmanned aerial vehicle and the runway direction is referred to as the slip angle for short below; the roll angle of the fixed-wing unmanned aerial vehicle is hereinafter referred to as roll angle for short; the pitch angle of a fixed wing drone is referred to below for short. In the initial stage of autonomous landing of the fixed-wing unmanned aerial vehicle, the autonomous landing process can be started only when the yaw angle and the roll angle are close to zero and the pitch angle is within the safe landing range.
The pose control parameter correction method is a correction method of a yaw angle, a roll angle and a pitch angle based on adjacent frame matching feature points.
The method for correcting the slip angle based on the adjacent frame matching feature points comprises the following steps: and calculating a correction value to correct the slip angle in real time according to the transverse and longitudinal shifts of the matched feature points in the adjacent frames of the video caused by the slip angle.
The roll angle correction method based on the adjacent frame matching feature points comprises the following steps: and calculating a correction value to correct the roll angle in real time according to the rotation angle between the matched characteristic points in the adjacent frames of the video caused by the roll angle.
The pitch angle correction method based on the adjacent frame matching feature points comprises the following steps: and calculating a correction value to correct the pitch angle in real time according to the longitudinal deviation of the matched characteristic points in the adjacent frames of the video caused by the pitch angle.
The close-range runway tracking method based on the 3D CNN comprises the following steps: in the autonomous landing process of the fixed wing unmanned aerial vehicle, the 3D CNN-based pose control parameter correction method is used for correcting the yaw angle, the roll angle and the pitch angle in real time, and because the video has continuity among multiple frames in the autonomous landing process of the fixed wing unmanned aerial vehicle, the pose control parameters between adjacent frames are corrected, so that the pose control parameters in the whole autonomous landing process can be ensured to meet safe landing conditions.
As shown in fig. 1-6:
defining one: defining the vision autonomous landing precondition (landing precondition for short) of the fixed-wing unmanned aerial vehicle as follows: in the initial landing phase, the slip angle α and the roll angle β are both zero. For each model, a pitch angle range which can ensure safe landing is set as [ gamma ]0,γ1]。
Definition II: defining the safe landing condition (landing condition for short) of the visual autonomous landing of the fixed-wing unmanned aerial vehicle as follows: the values of the slip angle α and the roll angle β approach zero, that is:
α∈[-εα,+εα]，εα→0 (1)
β∈[-εβ,+εβ]，εβ→0 (2)
the range of the pitch angle gamma is
γ∈[γ0,γ1] (3)
In the whole landing process, the landing condition defined in the definition II must be met, and if the yaw angle alpha, the roll angle beta and the pitch angle gamma deviate from the safe landing condition, the correction must be carried out in time, so that the accumulated expansion of deviation is avoided.
Sideslip angle correction algorithm
The offset correction algorithm for the slip angle α can be expressed by the following equations (4) and (5):
αnew＝αold+δα (5)
k is an adjustment coefficient obtained by a large amount of video training and the principle is to keep the slip angle alpha to satisfy the formula (1), deltaαIs a correction angle for the slip angle; arctan is an arctangent function, d is the lateral offset of matched feature points in adjacent frames of the video, h is the longitudinal offset of matched feature points in adjacent frames of the video, αnewFor corrected slip angle, αoldIs the slip angle before correction.
When the unmanned aerial vehicle generates lateral deviation in the landing process, the matched characteristic points in the adjacent frames of the video shown in the figure 2 are caused to deviate, and the correction angle delta is calculated according to the formula (4)αAnd then, the slip angle alpha is corrected by using the formula (5) so that the slip angle alpha meets the formula (1) of the safe landing condition. As can be seen from the expressions (4) and (5), since arctan is a monotonically increasing function, the correction angle and d are proportional relationsSystem, and h are inversely proportional.
Roll angle correction algorithm
The offset correction algorithm for the roll angle β can be expressed by the following equations (6) and (7):
δβ＝-lλ (6)
βnew＝βold+δβ (7)
l is an adjustment coefficient obtained through a large amount of video training, and the principle is to keep the roll angle beta to meet the formula (2). DeltaβFor correcting the roll angle, betanewFor the corrected roll angle, betaoldFor roll angle before correction, λ is the relative rotation angle between matching feature points in adjacent frames of the video.
When the unmanned aerial vehicle rolls during landing, characteristic points matched in adjacent frames of the video shown in FIG. 4 are caused to shift, and the correction angle delta is calculated according to the formula (6)βThen, the roll angle β is corrected using the expression (7) so that the roll angle β satisfies the expression (2) of the safe landing condition.
Pitch angle correction algorithm
The offset correction algorithm for the pitch angle γ can be expressed by equations (8) and (9) as follows:
δγ＝-mh (8)
γnew＝γold+δγ (9)
m is an adjustment coefficient and is obtained through a large amount of video training, and the principle is to keep the pitch angle gamma to meet the formula (3). h is the longitudinal shift, delta, of the matching feature points in adjacent frames of the videoγModified angle, gamma, of pitch anglenewFor corrected pitch angle, gammaoldTo correct the pitch angle before.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (2)
1. A visual landing guiding method for a fixed-wing unmanned aerial vehicle is characterized by comprising the following steps: extracting adjacent multi-frame images in a video acquired by imaging equipment of the fixed-wing unmanned aerial vehicle in a landing stage by using 3D CNN, and controlling the unmanned aerial vehicle to land autonomously by calculating position change information between feature points of the adjacent multi-frame images which can be matched with each other;
the method comprises the following steps: correcting a pose control parameter based on the 3D CNN and tracking a close-distance runway based on the 3D CNN;
the pose control parameter based on the 3D CNN is corrected as follows: correcting the pose control parameters by using a 3D CNN-based method aiming at the acquired images corresponding to the adjacent frames; the pose control parameters include: matching the roll angle, the pitch angle and the yaw angle of the characteristic points based on adjacent frames;
the close-range runway tracking based on 3D CNN is as follows: performing real-time correction on pose control parameters by adopting the 3D CNN-based method on the continuous frames in the acquired video;
the slip angle correction algorithm is represented by the following formula:
αnew＝αold+δα
where k is the adjustment coefficient, obtained by extensive video training, δαIs a correction angle for the slip angle; d is the lateral offset of the matched feature points in the adjacent frames of the video, h is the longitudinal offset of the matched feature points in the adjacent frames of the video, arctan is an arctangent function, alpha is a side offset angle, and alpha isnewFor corrected slip angle, αoldIs the slip angle before correction;
the roll angle correction algorithm is represented by the following equation:
δβ＝-lλ
βnew＝βold+δβ
wherein l is an adjustment coefficient obtained through a large amount of video training, beta is a roll angle, and deltaβFor correcting the roll angle, betanewFor the corrected roll angle, betaoldLambda is the relative rotation angle between matching feature points in adjacent frames of the video for the roll angle before correction.
2. The method of claim 1, wherein the pitch angle correction algorithm is represented by the following equation:
δγ＝-mh
γnew＝γold+δγ
wherein m is an adjustment coefficient obtained by a large amount of video training, h is a longitudinal offset of matching feature points in adjacent frames of the video, and deltaγModified angle, gamma, of pitch anglenewFor corrected pitch angle, gammaoldTo correct the pitch angle before.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810730693.8A CN108820233B (en) | 2018-07-05 | 2018-07-05 | Visual landing guiding method for fixed-wing unmanned aerial vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810730693.8A CN108820233B (en) | 2018-07-05 | 2018-07-05 | Visual landing guiding method for fixed-wing unmanned aerial vehicle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108820233A CN108820233A (en) | 2018-11-16 |
| CN108820233B true CN108820233B (en) | 2022-05-06 |
Family
ID=64135588
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810730693.8A Active CN108820233B (en) | 2018-07-05 | 2018-07-05 | Visual landing guiding method for fixed-wing unmanned aerial vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108820233B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109341700B (en) * | 2018-12-04 | 2023-06-30 | 中国航空工业集团公司西安航空计算技术研究所 | Visual auxiliary landing navigation method for fixed-wing aircraft under low visibility |
| CN109712185B (en) * | 2018-12-07 | 2022-11-08 | 天津津航计算技术研究所 | Helicopter landing process neutral attitude estimation method based on deep learning |
| CN110109469A (en) * | 2019-03-19 | 2019-08-09 | 南京理工大学泰州科技学院 | It is a kind of with color, identification, positioning, following function quadrotor drone control system |
| CN110543182B (en) * | 2019-09-11 | 2022-03-15 | 济宁学院 | Autonomous landing control method and system for small unmanned gyroplane |
| CN110631588B (en) * | 2019-09-23 | 2022-11-18 | 电子科技大学 | Unmanned aerial vehicle visual navigation positioning method based on RBF network |
| CN110673642B (en) * | 2019-10-28 | 2022-10-28 | 深圳市赛为智能股份有限公司 | Unmanned aerial vehicle landing control method and device, computer equipment and storage medium |
| CN112241180B (en) * | 2020-10-22 | 2021-08-17 | 北京航空航天大学 | Visual processing method for landing guidance of unmanned aerial vehicle mobile platform |
| CN112797982A (en) * | 2020-12-25 | 2021-05-14 | 中国航空工业集团公司沈阳飞机设计研究所 | Unmanned aerial vehicle autonomous landing measurement method based on machine vision |
| CN113011557B (en) * | 2021-02-22 | 2021-09-21 | 山东航空股份有限公司 | Method and system for judging unstable approach of airplane based on convolutional neural network |
| CN113067157B (en) * | 2021-03-25 | 2022-02-01 | 北京理工大学 | Conformal phased array antenna design system and design method based on deep reinforcement learning |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106874849A (en) * | 2017-01-09 | 2017-06-20 | 北京航空航天大学 | Flying bird detection method and device based on video |
| CN107451552A (en) * | 2017-07-25 | 2017-12-08 | 北京联合大学 | A kind of gesture identification method based on 3D CNN and convolution LSTM |
| CN107909014A (en) * | 2017-10-31 | 2018-04-13 | 天津大学 | A kind of video understanding method based on deep learning |
| CN108182388A (en) * | 2017-12-14 | 2018-06-19 | 哈尔滨工业大学（威海） | A kind of motion target tracking method based on image |
-
2018
- 2018-07-05 CN CN201810730693.8A patent/CN108820233B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN108820233A (en) | 2018-11-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108820233B (en) | Visual landing guiding method for fixed-wing unmanned aerial vehicle | |
| Al-Kaff et al. | Survey of computer vision algorithms and applications for unmanned aerial vehicles | |
| CN113485441B (en) | Distribution network inspection method combining unmanned aerial vehicle high-precision positioning and visual tracking technology | |
| Kong et al. | Autonomous landing of an UAV with a ground-based actuated infrared stereo vision system | |
| CN110426046B (en) | Unmanned aerial vehicle autonomous landing runway area obstacle judging and tracking method | |
| McGee et al. | Obstacle detection for small autonomous aircraft using sky segmentation | |
| CN104808685A (en) | Vision auxiliary device and method for automatic landing of unmanned aerial vehicle | |
| CN105352495B (en) | Acceleration and light stream Data Fusion of Sensor unmanned plane horizontal velocity control method | |
| US9892646B2 (en) | Context-aware landing zone classification | |
| CN105501457A (en) | Infrared vision based automatic landing guidance method and system applied to fixed-wing UAV (unmanned aerial vehicle) | |
| CN101109640A (en) | Unmanned aircraft landing navigation system based on vision | |
| CN103149939A (en) | Dynamic target tracking and positioning method of unmanned plane based on vision | |
| Li et al. | UAV autonomous landing technology based on AprilTags vision positioning algorithm | |
| Xu et al. | Use of land’s cooperative object to estimate UAV’s pose for autonomous landing | |
| CN106200672A (en) | A kind of unmanned plane barrier-avoiding method based on light stream | |
| Al-Kaff et al. | Intelligent vehicle for search, rescue and transportation purposes | |
| Zarandy et al. | A novel algorithm for distant aircraft detection | |
| CN112380933B (en) | Unmanned aerial vehicle target recognition method and device and unmanned aerial vehicle | |
| CN112596071A (en) | Unmanned aerial vehicle autonomous positioning method and device and unmanned aerial vehicle | |
| US11987382B2 (en) | Method for aircraft localization and control | |
| CN114564042A (en) | Unmanned aerial vehicle landing method based on multi-sensor fusion | |
| Xia et al. | Integrated emergency self-landing method for autonomous uas in urban aerial mobility | |
| CN114689030A (en) | Unmanned aerial vehicle auxiliary positioning method and system based on airborne vision | |
| Shang et al. | An Integrated Navigation Method for UAV Autonomous Landing Based on Inertial and Vision Sensors | |
| Qi et al. | Detection and tracking of a moving target for UAV based on machine vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |