CN112740222A - Altitude determination method, aircraft, and computer-readable storage medium - Google Patents
Altitude determination method, aircraft, and computer-readable storage medium Download PDFInfo
- Publication number
- CN112740222A CN112740222A CN201980060094.2A CN201980060094A CN112740222A CN 112740222 A CN112740222 A CN 112740222A CN 201980060094 A CN201980060094 A CN 201980060094A CN 112740222 A CN112740222 A CN 112740222A
- Authority
- CN
- China
- Prior art keywords
- image
- height
- determining
- target
- aircraft
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims abstract description 146
- 239000011159 matrix material Substances 0.000 claims description 48
- 238000004590 computer program Methods 0.000 claims description 12
- 238000012795 verification Methods 0.000 claims description 12
- 238000004422 calculation algorithm Methods 0.000 claims description 10
- 238000013507 mapping Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims description 6
- 238000012937 correction Methods 0.000 claims description 5
- 238000003062 neural network model Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 230000004927 fusion Effects 0.000 description 6
- 238000013527 convolutional neural network Methods 0.000 description 4
- 125000004122 cyclic group Chemical group 0.000 description 4
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004540 process dynamic Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/13—Satellite images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Molecular Biology (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Astronomy & Astrophysics (AREA)
- Remote Sensing (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
一种高度确定方法、飞行器及计算机可读存储介质，该方法包括：获取拍摄装置拍摄的图像，其中，所述图像包括第一图像和第二图像(S101)；确定所述第一图像和第二图像中的第一目标图像区域和第二目标图像区域(S102)；根据所述第一图像和第二图像中的第一目标图像区域，确定飞行器距离水面的第一高度(S103)；根据所述第一图像和第二图像中的第二目标图像区域，确定所述飞行器距离水面的第二高度(S104)；根据所述第一高度和所述第二高度，确定飞行器距离水面的目标高度(S105)。该方法旨在准确的确定高度。
A method for determining altitude, an aircraft, and a computer-readable storage medium, the method comprising: acquiring an image captured by a photographing device, wherein the image includes a first image and a second image (S101); determining the first image and the second image the first target image area and the second target image area in the two images (S102); according to the first target image area in the first image and the second image, determine the first height of the aircraft from the water surface (S103); Determine the second height of the aircraft from the water surface in the second target image area in the first image and the second image (S104); determine the target of the aircraft from the water surface according to the first height and the second height height (S105). This method aims to accurately determine the height.
Description
Technical Field
The present application relates to the field of aircraft technologies, and in particular, to an altitude determination method, an aircraft, and a computer-readable storage medium.
Background
At present, the height of the aircraft is calculated by the aircraft through the visual odometer, but the visual odometer cannot effectively process dynamic textures of images, the calculation precision of the height is poor, the influence of 1-2 m height change of the aircraft on the aircraft is small in the high-altitude flight process, and the height of the aircraft is particularly important at the moment because the aircraft is close to the ground in the low-altitude flight process.
In the low-altitude Flight process, the height of the aircraft is generally measured and calculated by an ultrasonic sensor or a Time of Flight (TOF) sensor of the aircraft, but the TOF sensor adopts laser ranging, and is mirror reflection for the water surface, when the aircraft moves forwards and backwards, the aircraft body tilts, laser emitted by the TOF sensor is completely reflected by the water surface, so that the TOF sensor cannot receive reflected light, the height of the aircraft from the water surface cannot be accurately calculated, and the aircraft is prone to falling high and falling into water. Therefore, how to accurately measure and calculate the height of the aircraft from the water surface is a problem to be solved urgently at present.
Disclosure of Invention
Based on the above, the application provides an altitude determining method, an aircraft and a computer readable storage medium, aiming at accurately measuring and calculating the altitude of the aircraft from the water surface and improving the flight safety of the aircraft.
In a first aspect, the present application provides an altitude determination method for an aircraft, the aircraft including a camera, the method comprising:
acquiring an image shot by the shooting device, wherein the image comprises a first image and a second image;
determining a first target image area and a second target image area in the first image and the second image, wherein the first target image area is an image area which does not include the inverted image of the aircraft, and the second target image area is an image area which includes the inverted image of the aircraft;
determining a first height of the aircraft from the water surface according to a first target image area in the first image and the second image;
determining a second height of the aircraft from the water surface according to a second target image area in the first image and the second image;
and determining the target height of the aircraft from the water surface according to the first height and the second height.
In a second aspect, the present application further provides an aircraft comprising a camera, a memory, and a processor;
the shooting device is used for shooting images;
the memory is used for storing a computer program;
the processor is configured to execute the computer program and, when executing the computer program, implement the following steps:
acquiring an image shot by the shooting device, wherein the image comprises a first image and a second image;
determining a first target image area and a second target image area in the first image and the second image, wherein the first target image area is an image area which does not include the inverted image of the aircraft, and the second target image area is an image area which includes the inverted image of the aircraft;
determining a first height of the aircraft from the water surface according to a first target image area in the first image and the second image;
determining a second height of the aircraft from the water surface according to a second target image area in the first image and the second image;
and determining the target height of the aircraft from the water surface according to the first height and the second height.
In a third aspect, the present application also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, causes the processor to carry out the steps of the height determination method as described above.
The embodiment of the application provides an altitude determining method, an aircraft and a computer readable storage medium, wherein two images are shot, each image comprises an image area including an inverted image of the aircraft and an image area not including the inverted image of the aircraft, one altitude of the aircraft from the water surface can be determined based on the image areas including the inverted image of the aircraft in the two images, the other altitude of the aircraft from the water surface can be determined based on the image areas not including the inverted image of the aircraft in the two images, and finally the altitude of the aircraft from the water surface can be accurately determined based on the two altitudes of the aircraft from the water surface, so that the flight safety of the aircraft is effectively improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 is a schematic structural diagram of an aircraft implementing the altitude determination method of the present application;
FIG. 2 is a flow chart illustrating steps of a method for height determination according to an embodiment of the present application;
FIG. 3 is a schematic illustration of an aircraft flying over water in an embodiment of the present application;
FIG. 4 is a schematic flow diagram of sub-steps of the height determination method of FIG. 2;
FIG. 5 is a schematic flow diagram of sub-steps of the height determination method of FIG. 2;
fig. 6 is a block diagram illustrating a schematic structure of an aircraft according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The flow diagrams depicted in the figures are merely illustrative and do not necessarily include all of the elements and operations/steps, nor do they necessarily have to be performed in the order depicted. For example, some operations/steps may be decomposed, combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
The application provides an altitude determining method which is applied to an aircraft and used for accurately determining the altitude of the aircraft from the water surface and improving the flight safety of the aircraft. The aircraft includes an unmanned aerial vehicle, and the unmanned aerial vehicle includes a rotary wing type unmanned aerial vehicle, such as a four-rotor unmanned aerial vehicle, a six-rotor unmanned aerial vehicle, an eight-rotor unmanned aerial vehicle, a fixed wing type unmanned aerial vehicle, or a combination of a rotary wing type unmanned aerial vehicle and a fixed wing type unmanned aerial vehicle, which is not limited herein. Referring to fig. 1, fig. 1 is a schematic structural diagram of an aircraft implementing the altitude determining method of the present application, and as shown in fig. 1, the aircraft 100 includes a flying body 101 and a shooting device 102, and the shooting device 102 is used for shooting an image below the aircraft. The camera 102 may be a binocular camera or a monocular camera.
Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the features of the embodiments can be combined with each other without conflict.
Referring to fig. 2, fig. 2 is a schematic flowchart illustrating steps of a height determining method according to an embodiment of the present application.
Specifically, as shown in fig. 2, the height determining method includes steps S101 to S105.
S101, acquiring an image shot by the shooting device, wherein the image comprises a first image and a second image.
The shooting device comprises a binocular shooting device and a monocular shooting device, images shot by the shooting device comprise a first image and a second image, if the shooting device is the binocular shooting device, the first image and the second image comprise a left eye image and a right eye image of the binocular shooting device, and if the shooting device is the monocular shooting device, the first image and the second image are two images shot by the monocular shooting device at different moments, namely the shooting time of the first image is different from that of the second image. The aircraft acquires the image taken by the camera.
S102, determining a first target image area and a second target image area in the first image and the second image.
After the image shot by the shooting device is acquired, a first target image area and a second target image area in the first image are determined, and a first target image area and a second target image area in the second image are determined. The first target image area is an image area which does not comprise the inverted image of the aircraft, and the second target image area is an image area which comprises the inverted image of the aircraft. Referring to fig. 3, fig. 3 is a schematic view of the aircraft flying on the water surface in the embodiment of the present application, and as shown in fig. 3, the aircraft flies on the water surface, that is, the aircraft is located above the water surface a, and the water surface has an aircraft reflection B.
In one embodiment, a preset aircraft water surface reflection identification model is obtained; and determining a first target image area and a second target image area in the first image and the second image through the aircraft water surface reflection identification model, namely inputting the first image into the aircraft water surface reflection identification model to obtain the first target image area and the second target image area in the first image, and inputting the second image into the aircraft water surface reflection identification model to obtain the first target image area and the second target image area in the second image. The image area of the image, which does not comprise the reflection of the aircraft, and the image area comprising the reflection of the aircraft can be accurately and quickly determined through the aircraft water reflection identification model.
The aircraft water surface reflection recognition model is a neural network model trained in advance, namely a sample image set marked with or without aircraft water surface reflection is obtained, and the neural network model is trained based on the sample image set until the neural network model converges to obtain the aircraft water surface reflection recognition model. It is understood that the neural network models described above include, but are not limited to, convolutional neural network models, cyclic neural networks, and cyclic convolutional neural networks.
In an embodiment, it is determined whether a water surface area exists in the first image and the second image; if the water surface area exists in the first image and the second image, further determining a first target image area and a second target image area in the first image and the second image; if at least one of the first image and the second image does not have a second target image area, determining a first height of the aircraft from the water surface according to the first target image areas in the first image and the second image, namely determining a homography matrix based on the two first target image areas, and determining the first height according to the homography matrix; the first altitude is determined as a target altitude of the aircraft from the water surface.
Specifically, a pre-stored water surface area identification model is obtained; through the water surface area identification model, whether a water surface area exists in the first image and the second image or not is determined, namely the first image is input into the water surface area identification model, a type label output by the water surface area identification model is obtained, if the type label is a label corresponding to the water surface area, the water surface area exists in the first image can be determined, and if the type label is not a label corresponding to the water surface area, the water surface area does not exist in the first image. Whether the water surface area exists in the image can be accurately and quickly determined through the water surface area identification model.
The water surface area identification model is a neural network model trained in advance, namely a sample image set marked with a water surface area is obtained, and the neural network model is trained based on the sample image set until the neural network model converges to obtain the water surface area identification model. It is understood that the neural network models described above include, but are not limited to, convolutional neural network models, cyclic neural networks, and cyclic convolutional neural networks.
S103, determining a first height of the aircraft from the water surface according to the first target image area in the first image and the second image.
And determining a first height of the aircraft from the water surface according to the first target area in the first image and the first target area in the second image, namely through the image area which does not comprise the inverted image of the aircraft in the first image and the image area which does not comprise the inverted image of the aircraft in the second image. The first height of the aircraft from the water surface can be accurately determined through two image areas which do not comprise the reflection of the aircraft.
In an embodiment, as shown in fig. 4, step S103 specifically includes: and substeps 1031 to S1032.
And S1031, determining a homography matrix between the first image and the second image according to the first target image area in the first image and the first target image area in the second image.
Specifically, based on a least square method or a random sampling consistency algorithm, a homography matrix between the first image and the second image is determined according to an image area not including the aircraft reflection in the first image and an image area not including the aircraft reflection in the second image.
In one embodiment, a plurality of pixel point matching pairs are extracted from a first target image area of a first image and a first target image area of a second image based on a preset pixel point matching algorithm; and calculating a homography matrix between the first image and the second image according to the matching pairs of the plurality of pixel points. Wherein, a pixel matching pair comprises a pixel extracted from the first target image region of the first image and a pixel extracted from the first target image region of the second image, and the pixel matching algorithm includes but is not limited to orb (organized FAST and Rotated brief) algorithm, Harris corner detection algorithm and Surf algorithm. It can be understood that the number of pixel point matching pairs may be set based on actual conditions, which is not specifically limited in the present application. Optionally, the number of pixel point matching pairs is 4, 10, 16, 25, 50, 76 or 100 pairs.
S1032, determining the first height according to the homography matrix.
Specifically, an internal reference matrix and binocular distance of a binocular shooting device are obtained; and determining a first height according to the homography matrix, the internal reference matrix and the binocular distance. The internal reference matrix comprises internal parameter values of binocular shooting devices, and the binocular distance is the distance between a left-eye shooting device and a right-eye shooting device in the binocular shooting devices.
Wherein, let the homography matrix be H, according to the definition of the homography matrix, can know:
and S104, determining a second height of the aircraft from the water surface according to the second target image areas in the first image and the second image.
And determining a second height of the aircraft from the water surface according to the second target area in the first image and the second target area in the second image, namely through the image area comprising the reflection of the aircraft in the first image and the image area comprising the reflection of the aircraft in the second image.
In an embodiment, a depth map of a second target image area is determined from a second target image area of the first image and a second target image area of the second image, and a target depth value is determined from the depth map; determining a normal vector of the water surface; and determining a second height of the aircraft from the water surface according to the normal vector of the water surface and the target depth value. The normal vector of the water surface can be a unit normal vector. It is understood that the depth map of the second target image region may be determined based on a Semi-Global Block Matching (SGBM) algorithm according to the second target image region of the first image and the second target image region of the second image.
Specifically, a half of the target depth value is taken as a third height, and the direction correction processing is carried out on the third height through the normal vector of the water surface, so that a second height of the aircraft from the water surface is obtained. The normal vector of the water surface is vertical to the water surface, the target depth value is an inclined depth value, and the accurate second height can be obtained by performing direction correction processing on half of the target depth value through the normal vector of the water surface. Illustratively, let the target depth value be d2Then, a second height d'2＝NTd2/2，NTIs the normal vector of the water surface.
In one embodiment, an average depth value corresponding to the depth map is obtained according to the depth map; taking the average depth value as a target depth value; or determining a central pixel point of a second target image area; and acquiring the depth value corresponding to the central pixel point from the depth map, and taking the depth value corresponding to the central pixel point as a target depth value. The average depth value is obtained by the following specific method: acquiring the depth value of each pixel point from the depth map, and counting the number of the pixel points in the depth map; and accumulating the depth value of each pixel point to obtain a total depth value, and calculating to obtain an average depth value of the depth map according to the total depth value and the number of the pixel points.
In one embodiment, a homography matrix between a first image and a second image is determined according to a first target area in the first image and a first target image area in the second image; and determining the normal vector of the water surface according to the homography matrix, namely calculating to obtain a unit normal vector of the water surface according to the homography matrix, and taking the unit normal vector as the normal vector of the water surface.
And S105, determining the target height of the aircraft from the water surface according to the first height and the second height.
The target height of the aircraft from the water surface is determined through the first height and the second height of the aircraft from the water surface, and the target height of the aircraft from the water surface can be accurately determined by comprehensively considering the image area of the inverted image of the aircraft and the other image areas without the inverted image of the aircraft in the image. After the target height of the aircraft from the water surface is determined, the target height is sent to a flight control system of the aircraft, the flight control system controls the aircraft to fly based on the target height, and accidents caused by the fact that the aircraft falls into high water are avoided.
In an embodiment, as shown in fig. 5, step S105 specifically includes: substeps S1051 to S1053.
And S1051, verifying the second height according to the first height.
Specifically, a difference between the first height and the second height is calculated, and it is determined whether an absolute value of the difference between the first height and the second height is less than or equal to a preset threshold; if the absolute value of the difference value between the first height and the second height is smaller than or equal to a preset threshold value, determining that the second height passes the verification; and if the absolute value of the difference between the first height and the second height is greater than a preset threshold, determining that the second height is not verified. It is understood that the preset threshold may be set based on actual situations, and the present application is not limited thereto. Optionally, the preset threshold is 3 cm.
Illustratively, the first height is d1And the second height is d'2，dthIs a preset threshold value, then is | | d'2-d1||≤dthOf second height d'2Through verification, the result is in d |)'2-d1||>dthOf second height d'2Failing the verification.
S1052, when the second height passes the verification, determining a first weight coefficient corresponding to the first height and a second weight coefficient corresponding to the second height.
And when the second altitude passes the verification, namely the absolute value of the difference between the first altitude and the second altitude is smaller than or equal to a preset threshold value, determining a first weight coefficient corresponding to the first altitude and a second weight coefficient corresponding to the second altitude, and when the second altitude does not pass the verification, taking the first altitude as the altitude of the aircraft from the water surface.
Specifically, a mapping relation table between pre-stored weight coefficients and absolute values of the difference values is obtained; calculating a difference between the first height and the second height; and inquiring a mapping relation table, taking the weight coefficient corresponding to the absolute value of the difference value as a first weight coefficient corresponding to the first height, and determining a second weight coefficient corresponding to the second height based on the first weight coefficient. Wherein the sum of the first weight coefficient and the second weight coefficient is 1. It is understood that the mapping relation table between the pre-stored weight coefficients and the absolute values of the differences may be set based on actual conditions, and this is not specifically limited in this application.
S1053, determining the target height of the aircraft from the water surface according to the first height, the second height, the first weight coefficient and the second weight coefficient.
Specifically, calculating a product of the first height and a first weight coefficient, and taking the product of the first height and the first weight coefficient as a first fusion height; calculating the product of the second height and the second weight coefficient, and taking the product of the second height and the second weight coefficient as a second fusion height; and calculating the sum of the first fusion height and the second fusion height, and taking the sum of the first fusion height and the second fusion height as the target height of the aircraft from the water surface. Exemplary, target height d ═ α · d'2+(1-α)·d1Alpha is a second weight coefficient corresponding to the second height, and 1-alpha is a first weight coefficient corresponding to the first height.
The altitude determining method provided by the above embodiment captures two images, each image includes an image area including an aircraft reflection and an image area not including the aircraft reflection, and based on the image area including the aircraft reflection in the two images, an altitude of the aircraft from the water surface can be determined, and based on the image area not including the aircraft reflection in the two images, another altitude of the aircraft from the water surface can be determined, and finally based on the two altitudes of the aircraft from the water surface, the altitude of the aircraft from the water surface can be accurately determined, and the flight safety of the aircraft is effectively improved.
Referring to fig. 6, fig. 6 is a schematic block diagram of an aircraft according to an embodiment of the present application. The aircraft 200 includes a processor 201, a memory 202, and a camera 203, and the processor 201, the memory 202, and the camera 203 are connected by a bus 204, such as an I2C (Inter-integrated Circuit) bus 204. The aircraft may be a rotary-wing unmanned aircraft, such as a four-rotor unmanned aircraft, a six-rotor unmanned aircraft, an eight-rotor unmanned aircraft, or a fixed-wing unmanned aircraft, or a combination of a rotary-wing type and a fixed-wing unmanned aircraft, which is not limited herein.
Specifically, the Processor 201 may be a Micro-controller Unit (MCU), a Central Processing Unit (CPU), a Digital Signal Processor (DSP), or the like.
Specifically, the Memory 202 may be a Flash chip, a Read-Only Memory (ROM) magnetic disk, an optical disk, a usb disk, or a removable hard disk.
Specifically, the photographing device 203 may be a monocular photographing device and a binocular photographing device.
Wherein the processor 201 is configured to run a computer program stored in the memory 202, and when executing the computer program, implement the following steps:
acquiring an image shot by the shooting device, wherein the image comprises a first image and a second image;
determining a first target image area and a second target image area in the first image and the second image, wherein the first target image area is an image area which does not include the inverted image of the aircraft, and the second target image area is an image area which includes the inverted image of the aircraft;
determining a first height of the aircraft from the water surface according to a first target image area in the first image and the second image;
determining a second height of the aircraft from the water surface according to a second target image area in the first image and the second image;
and determining the target height of the aircraft from the water surface according to the first height and the second height.
Optionally, the processor, when determining the first target image area and the second target image area in the first image and the second image, is configured to:
acquiring a preset aircraft water surface reflection recognition model;
and determining a first target image area and a second target image area in the first image and the second image through the aircraft water surface reflection recognition model.
Optionally, the processor is configured to, when determining a first altitude of the aircraft from the water surface according to the first target image region in the first image and the second image, implement:
determining a homography matrix between the first image and the second image according to a first target image area in the first image and a first target image area in the second image;
determining the first height according to the homography matrix.
Optionally, the processor, when implementing determining the homography matrix between the first image and the second image, is configured to implement:
extracting a plurality of pixel point matching pairs from a first target image area of the first image and a first target image area of the second image based on a preset pixel point matching algorithm;
and calculating a homography matrix between the first image and the second image according to the plurality of pixel point matching pairs.
Optionally, the camera comprises a binocular camera, and the first and second images comprise left and right eye images captured by the binocular camera; the processor, when enabled to determine the first height from the homography matrix, is configured to enable:
acquiring an internal reference matrix and binocular distances of the binocular shooting device;
and determining the first height according to the homography matrix, the internal reference matrix and the binocular distance.
Optionally, the processor is configured to, when determining a second altitude of the aircraft from the water surface according to a second target image region in the first image and the second image, implement:
determining a depth map of a second target image area according to a second target image area of the first image and a second target image area of the second image, and determining a target depth value according to the depth map;
determining a normal vector of the water surface;
and determining the second height according to the normal vector and the target depth value.
Optionally, the processor, when determining the second height according to the normal vector and the target depth value, is configured to:
and taking a half of the target depth value as a third height, and performing direction correction processing on the third height through the normal vector to obtain the second height.
Optionally, the processor, when implementing determining a target depth value from the depth map, is configured to implement:
acquiring an average depth value corresponding to the depth map according to the depth map;
and taking the average depth value as a target depth value.
Optionally, the processor, when implementing determining a target depth value from the depth map, is configured to implement:
determining a central pixel point of the second target image area;
and acquiring the depth value corresponding to the central pixel point from the depth map, and taking the depth value corresponding to the central pixel point as a target depth value.
Optionally, the processor, when determining the normal vector of the water surface, is configured to:
determining a homography matrix between the first image and the second image according to the first target area in the first image and the first target image area in the second image;
and determining the normal vector according to the homography matrix.
Optionally, the processor is configured to, when determining a target altitude of the aircraft from the water surface based on the first altitude and the second altitude, implement:
according to the first height, checking the second height;
when the second height passes the verification, determining a first weight coefficient corresponding to the first height and a second weight coefficient corresponding to the second height;
and determining the target height of the aircraft from the water surface according to the first height, the second height, the first weight coefficient and the second weight coefficient.
Optionally, the processor is configured to, when determining a target altitude of the aircraft from the water surface based on the first altitude and the second altitude, implement:
when the second altitude fails to be verified, determining the first altitude as a target altitude of the aircraft from the water surface.
Optionally, the processor is configured to, when verifying the second height according to the first height, implement:
calculating a difference between the first height and the second height and determining whether an absolute value of the difference is less than or equal to a preset threshold;
if the absolute value of the difference is smaller than or equal to a preset threshold value, determining that the second height passes the verification;
and if the absolute value of the difference is larger than a preset threshold, determining that the second height is not verified.
Optionally, when determining the first weight coefficient corresponding to the first height, the processor is configured to:
acquiring a mapping relation table between the pre-stored weight coefficient and the absolute value of the difference;
calculating a difference between the first height and the second height;
and querying the mapping relation table, and taking a weight coefficient corresponding to the absolute value of the difference value as a first weight coefficient corresponding to the first height.
Optionally, the processor is further configured to implement:
determining whether a water surface area exists in the first image and the second image;
and if the water surface area exists in the first image and the second image, executing a step of determining a first target image area and a second target image area in the first image and the second image.
Optionally, the processor is further configured to implement:
if at least one of the first image and the second image does not have a second target image area, determining a first height of the aircraft from the water surface according to the first target image area in the first image and the second image;
determining the first altitude as a target altitude of the aircraft from the water surface.
Optionally, the processor, when performing determining whether a water surface region exists in the first image and the second image, is configured to perform:
acquiring a pre-stored water surface area identification model;
and determining whether the water surface area exists in the first image and the second image through the water surface area identification model.
It should be noted that, as will be clearly understood by those skilled in the art, for convenience and brevity of description, the specific working process of the aircraft described above may refer to the corresponding process in the foregoing embodiment of the height determining method, and is not described herein again.
Embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, where the computer program includes program instructions, and the processor executes the program instructions to implement the steps of the height determining method provided in the foregoing embodiments.
The computer readable storage medium may be the internal storage unit of the control terminal or the unmanned aerial vehicle described in any of the foregoing embodiments, for example, a hard disk or a memory of the control terminal or the unmanned aerial vehicle. The computer readable storage medium may also be an external storage device of the control terminal or the unmanned aerial vehicle, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are equipped on the control terminal or the unmanned aerial vehicle.
It is to be understood that the terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the specification of the present application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
While the invention has been described with reference to specific embodiments, the scope of the invention is not limited thereto, and those skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the invention. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (35)
1. An altitude determination method applied to an aircraft including a camera, the method comprising:
acquiring an image shot by the shooting device, wherein the image comprises a first image and a second image;
determining a first target image area and a second target image area in the first image and the second image, wherein the first target image area is an image area which does not include the inverted image of the aircraft, and the second target image area is an image area which includes the inverted image of the aircraft;
determining a first height of the aircraft from the water surface according to a first target image area in the first image and the second image;
determining a second height of the aircraft from the water surface according to a second target image area in the first image and the second image;
and determining the target height of the aircraft from the water surface according to the first height and the second height.
2. The height determination method of claim 1, wherein the determining a first target image area and a second target image area in the first image and the second image comprises:
acquiring a preset aircraft water surface reflection recognition model;
and determining a first target image area and a second target image area in the first image and the second image through the aircraft water surface reflection recognition model.
3. The altitude determination method according to claim 1 or 2, wherein determining the first altitude of the aircraft from the water surface from the first target image region in the first and second images comprises:
determining a homography matrix between the first image and the second image according to a first target image area in the first image and a first target image area in the second image;
determining the first height according to the homography matrix.
4. The height determination method of claim 3, wherein the determining a homography matrix between the first image and the second image comprises:
extracting a plurality of pixel point matching pairs from a first target image area of the first image and a first target image area of the second image based on a preset pixel point matching algorithm;
and calculating a homography matrix between the first image and the second image according to the plurality of pixel point matching pairs.
5. The height determination method according to claim 3 or 4, wherein the photographing device includes a binocular photographing device, and the first and second images include left and right eye images photographed by the binocular photographing device; the determining the first height according to the homography matrix includes:
acquiring an internal reference matrix and binocular distances of the binocular shooting device;
and determining the first height according to the homography matrix, the internal reference matrix and the binocular distance.
6. The altitude determination method according to any one of claims 1 to 5, wherein determining a second altitude of the aircraft from the water surface from a second target image region in the first and second images comprises:
determining a depth map of a second target image area according to a second target image area of the first image and a second target image area of the second image, and determining a target depth value according to the depth map;
determining a normal vector of the water surface;
and determining the second height according to the normal vector and the target depth value.
7. The height determination method of claim 6, wherein determining the second height from the normal vector and the target depth value comprises:
and taking a half of the target depth value as a third height, and performing direction correction processing on the third height through the normal vector to obtain the second height.
8. The height determination method according to claim 6 or 7, wherein the determining a target depth value from the depth map comprises:
acquiring an average depth value corresponding to the depth map according to the depth map;
and taking the average depth value as a target depth value.
9. The height determination method according to claim 6 or 7, wherein the determining a target depth value from the depth map comprises:
determining a central pixel point of the second target image area;
and acquiring the depth value corresponding to the central pixel point from the depth map, and taking the depth value corresponding to the central pixel point as a target depth value.
10. The method of any one of claims 6 to 9, wherein said determining a normal vector to the water surface comprises:
determining a homography matrix between the first image and the second image according to the first target area in the first image and the first target image area in the second image;
and determining the normal vector according to the homography matrix.
11. The altitude determination method according to any one of claims 1 to 10, wherein determining a target altitude of an aircraft from a water surface based on the first altitude and the second altitude comprises:
according to the first height, checking the second height;
when the second height passes the verification, determining a first weight coefficient corresponding to the first height and a second weight coefficient corresponding to the second height;
and determining the target height of the aircraft from the water surface according to the first height, the second height, the first weight coefficient and the second weight coefficient.
12. The altitude determination method of claim 11, wherein determining a target altitude of the aircraft from the water surface based on the first altitude and the second altitude comprises:
when the second altitude fails to be verified, determining the first altitude as a target altitude of the aircraft from the water surface.
13. The height determination method of claim 11 or 12, wherein the verifying the second height from the first height comprises:
calculating a difference between the first height and the second height and determining whether an absolute value of the difference is less than or equal to a preset threshold;
if the absolute value of the difference is smaller than or equal to a preset threshold value, determining that the second height passes the verification;
and if the absolute value of the difference is larger than a preset threshold, determining that the second height is not verified.
14. The height determining method according to any one of claims 11 to 13, wherein the determining a first weight coefficient corresponding to the first height comprises:
acquiring a mapping relation table between the pre-stored weight coefficient and the absolute value of the difference;
calculating a difference between the first height and the second height;
and querying the mapping relation table, and taking a weight coefficient corresponding to the absolute value of the difference value as a first weight coefficient corresponding to the first height.
15. The height determination method according to any one of claims 1 to 14, further comprising:
determining whether a water surface area exists in the first image and the second image;
and if the water surface area exists in the first image and the second image, executing a step of determining a first target image area and a second target image area in the first image and the second image.
16. The height determination method of claim 15, further comprising:
if at least one of the first image and the second image does not have a second target image area, determining a first height of the aircraft from the water surface according to the first target image area in the first image and the second image;
determining the first altitude as a target altitude of the aircraft from the water surface.
17. The height determination method according to claim 15 or 16, wherein the determining whether a water surface area exists in the first and second images comprises:
acquiring a pre-stored water surface area identification model;
and determining whether the water surface area exists in the first image and the second image through the water surface area identification model.
18. An aircraft, characterized in that the aircraft comprises a camera, a memory and a processor;
the shooting device is used for shooting images;
the memory for storing a computer program;
the processor is configured to execute the computer program and, when executing the computer program, implement the following steps:
acquiring an image shot by the shooting device, wherein the image comprises a first image and a second image;
determining a first target image area and a second target image area in the first image and the second image, wherein the first target image area is an image area which does not include the inverted image of the aircraft, and the second target image area is an image area which includes the inverted image of the aircraft;
determining a first height of the aircraft from the water surface according to a first target image area in the first image and the second image;
determining a second height of the aircraft from the water surface according to a second target image area in the first image and the second image;
and determining the target height of the aircraft from the water surface according to the first height and the second height.
19. The aircraft of claim 18, wherein the processor, when determining the first and second target image regions in the first and second images, is configured to:
acquiring a preset aircraft water surface reflection recognition model;
and determining a first target image area and a second target image area in the first image and the second image through the aircraft water surface reflection recognition model.
20. The vehicle of claim 18 or 19, wherein the processor is configured to, when determining the first altitude of the vehicle from the water surface based on the first target image region in the first and second images,:
determining a homography matrix between the first image and the second image according to a first target image area in the first image and a first target image area in the second image;
determining the first height according to the homography matrix.
21. The aircraft of claim 20, wherein the processor, when effecting determining the homography matrix between the first image and the second image, is configured to effect:
extracting a plurality of pixel point matching pairs from a first target image area of the first image and a first target image area of the second image based on a preset pixel point matching algorithm;
and calculating a homography matrix between the first image and the second image according to the plurality of pixel point matching pairs.
22. The aircraft of claim 20 or 21, wherein the camera comprises a binocular camera, and the first and second images comprise left and right eye images captured by the binocular camera; the processor, when enabled to determine the first height from the homography matrix, is configured to enable:
acquiring an internal reference matrix and binocular distances of the binocular shooting device;
and determining the first height according to the homography matrix, the internal reference matrix and the binocular distance.
23. The vehicle of any one of claims 18 to 22, wherein the processor is configured to, when determining a second altitude of the vehicle from the water surface based on a second target image region in the first and second images,:
determining a depth map of a second target image area according to a second target image area of the first image and a second target image area of the second image, and determining a target depth value according to the depth map;
determining a normal vector of the water surface;
and determining the second height according to the normal vector and the target depth value.
24. The aircraft of claim 23 wherein the processor, in determining the second altitude based on the normal vector and the target depth value, is configured to:
and taking a half of the target depth value as a third height, and performing direction correction processing on the third height through the normal vector to obtain the second height.
25. The aircraft of claim 23 or 24, wherein the processor, when effecting determining a target depth value from the depth map, is configured to effect:
acquiring an average depth value corresponding to the depth map according to the depth map;
and taking the average depth value as a target depth value.
26. The aircraft of claim 23 or 24, wherein the processor, when effecting determining a target depth value from the depth map, is configured to effect:
determining a central pixel point of the second target image area;
and acquiring the depth value corresponding to the central pixel point from the depth map, and taking the depth value corresponding to the central pixel point as a target depth value.
27. The aircraft of any one of claims 23 to 26, wherein the processor, when effecting determination of the normal vector of the water surface, is operable to effect:
determining a homography matrix between the first image and the second image according to the first target area in the first image and the first target image area in the second image;
and determining the normal vector according to the homography matrix.
28. The vehicle of any one of claims 18 to 27, wherein the processor is configured to, in determining a target altitude of the vehicle from the water surface based on the first altitude and the second altitude, perform:
according to the first height, checking the second height;
when the second height passes the verification, determining a first weight coefficient corresponding to the first height and a second weight coefficient corresponding to the second height;
and determining the target height of the aircraft from the water surface according to the first height, the second height, the first weight coefficient and the second weight coefficient.
29. The vehicle of claim 28, wherein the processor, when determining the target altitude of the vehicle from the water surface based on the first altitude and the second altitude, is configured to:
when the second altitude fails to be verified, determining the first altitude as a target altitude of the aircraft from the water surface.
30. The aircraft of claim 28 or 29, wherein the processor is configured to, when verifying the second altitude based on the first altitude, perform:
calculating a difference between the first height and the second height and determining whether an absolute value of the difference is less than or equal to a preset threshold;
if the absolute value of the difference is smaller than or equal to a preset threshold value, determining that the second height passes the verification;
and if the absolute value of the difference is larger than a preset threshold, determining that the second height is not verified.
31. The vehicle of any one of claims 28 to 30, wherein the processor, when enabled to determine a first weight coefficient corresponding to the first altitude, is configured to enable:
acquiring a mapping relation table between the pre-stored weight coefficient and the absolute value of the difference;
calculating a difference between the first height and the second height;
and querying the mapping relation table, and taking a weight coefficient corresponding to the absolute value of the difference value as a first weight coefficient corresponding to the first height.
32. The aircraft of any one of claims 18 to 31, wherein the processor is further configured to implement:
determining whether a water surface area exists in the first image and the second image;
and if the water surface area exists in the first image and the second image, determining a first target image area and a second target image area in the first image and the second image.
33. The aircraft of claim 32, wherein the processor is further configured to implement:
if at least one of the first image and the second image does not have a second target image area, determining a first height of the aircraft from the water surface according to the first target image area in the first image and the second image;
determining the first altitude as a target altitude of the aircraft from the water surface.
34. The aircraft of claim 32 or 33, wherein the processor, when effecting a determination of whether a water surface area is present in the first and second images, is configured to effect:
acquiring a pre-stored water surface area identification model;
and determining whether the water surface area exists in the first image and the second image through the water surface area identification model.
35. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which, when executed by a processor, causes the processor to implement the height determination method according to any one of claims 1 to 17.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/CN2019/122172 WO2021102994A1 (en) | 2019-11-29 | 2019-11-29 | Height determination method, aircraft and computer-readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112740222A true CN112740222A (en) | 2021-04-30 |
Family
ID=75589259
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201980060094.2A Pending CN112740222A (en) | 2019-11-29 | 2019-11-29 | Altitude determination method, aircraft, and computer-readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112740222A (en) |
| WO (1) | WO2021102994A1 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113850820B (en) * | 2021-09-08 | 2025-03-14 | 广州航新航空科技股份有限公司 | Blade image processing method, system, device and storage medium |
| CN114511542A (en) * | 2022-02-10 | 2022-05-17 | 湖南迪普视智能科技有限公司 | Tin bath liquid level detection method, device and equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100002222A1 (en) * | 1999-09-03 | 2010-01-07 | Arete Associates | Lidar with streak-tube imaging, including hazard detection in marine applications; related optics |
| CN108140245A (en) * | 2017-12-25 | 2018-06-08 | 深圳市道通智能航空技术有限公司 | Distance measuring method, device and unmanned plane |
| CN110299030A (en) * | 2019-06-28 | 2019-10-01 | 汉王科技股份有限公司 | Handheld terminal, aircraft and its airspace measurement method, control method |
-
2019
- 2019-11-29 CN CN201980060094.2A patent/CN112740222A/en active Pending
- 2019-11-29 WO PCT/CN2019/122172 patent/WO2021102994A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100002222A1 (en) * | 1999-09-03 | 2010-01-07 | Arete Associates | Lidar with streak-tube imaging, including hazard detection in marine applications; related optics |
| CN108140245A (en) * | 2017-12-25 | 2018-06-08 | 深圳市道通智能航空技术有限公司 | Distance measuring method, device and unmanned plane |
| CN110299030A (en) * | 2019-06-28 | 2019-10-01 | 汉王科技股份有限公司 | Handheld terminal, aircraft and its airspace measurement method, control method |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021102994A1 (en) | 2021-06-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9269188B2 (en) | Densifying and colorizing point cloud representation of physical surface using image data | |
| CN107025635B (en) | Processing method, processing device and electronic device for image saturation based on depth of field | |
| CN108474658B (en) | Ground form detection method and system, unmanned aerial vehicle landing method and unmanned aerial vehicle | |
| JP6632208B2 (en) | Information processing apparatus, information processing method, and program | |
| CN106937532B (en) | System and method for detecting actual user | |
| JP2021060868A5 (en) | ||
| JP2016157336A (en) | Image processing apparatus, image processing method, and program | |
| CN105335699B (en) | Intelligent determination method and application of three-dimensional coordinates of reading and writing elements in reading and writing scenes | |
| CN109425348A (en) | A kind of while positioning and the method and apparatus for building figure | |
| US9816786B2 (en) | Method for automatically generating a three-dimensional reference model as terrain information for an imaging device | |
| CN111126116A (en) | Method and system for identifying river garbage by unmanned boat | |
| WO2021016806A1 (en) | High-precision map positioning method, system and platform, and computer-readable storage medium | |
| CN105241423A (en) | A method for estimating stock volume of high canopy density forest based on unmanned aerial vehicle photography images | |
| CN111798507A (en) | Power transmission line safety distance measuring method, computer equipment and storage medium | |
| CN112740222A (en) | Altitude determination method, aircraft, and computer-readable storage medium | |
| CN109314774B (en) | System and method for stereoscopic imaging | |
| WO2024067473A1 (en) | Aircraft speed monitoring method and apparatus, and storage medium and aircraft | |
| JP6673504B2 (en) | Information processing device, database generation device, method, program, and storage medium | |
| CN115797451A (en) | Acupuncture point identification method, device and equipment and readable storage medium | |
| US10577101B2 (en) | Water surface detection method and apparatus, unmanned aerial vehicle landing method and apparatus and unmanned aerial vehicle | |
| WO2021051361A1 (en) | High-precision map positioning method and system, platform and computer-readable storage medium | |
| CN113168532A (en) | Target detection method, device, drone, and computer-readable storage medium | |
| CN108109171A (en) | Unmanned plane aerophotograph revolves detection method, device, equipment and the storage medium of drift angle | |
| WO2021035746A1 (en) | Image processing method and device, and movable platform | |
| JP6861592B2 (en) | Data thinning device, surveying device, surveying system and data thinning method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| AD01 | Patent right deemed abandoned | ||
| AD01 | Patent right deemed abandoned |
Effective date of abandoning: 20250307 |