CN109407697A - A kind of unmanned plane pursuit movement goal systems and method based on binocular distance measurement - Google Patents
A kind of unmanned plane pursuit movement goal systems and method based on binocular distance measurement Download PDFInfo
- Publication number
- CN109407697A CN109407697A CN201811100610.3A CN201811100610A CN109407697A CN 109407697 A CN109407697 A CN 109407697A CN 201811100610 A CN201811100610 A CN 201811100610A CN 109407697 A CN109407697 A CN 109407697A
- Authority
- CN
- China
- Prior art keywords
- target
- aerial vehicle
- unmanned aerial
- image
- binocular
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000005259 measurement Methods 0.000 title abstract description 9
- 230000009017 pursuit movement Effects 0.000 title abstract 2
- 238000013459 approach Methods 0.000 claims abstract description 24
- 230000033001 locomotion Effects 0.000 claims abstract description 12
- 238000004891 communication Methods 0.000 claims description 26
- 238000012545 processing Methods 0.000 claims description 22
- 238000001914 filtration Methods 0.000 claims description 3
- 238000012549 training Methods 0.000 claims description 3
- 230000004807 localization Effects 0.000 abstract 1
- 239000007787 solid Substances 0.000 abstract 1
- 238000005516 engineering process Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/10—Simultaneous control of position or course in three dimensions
- G05D1/101—Simultaneous control of position or course in three dimensions specially adapted for aircraft
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Image Analysis (AREA)
Abstract
The present invention relates to a kind of unmanned plane pursuit movement goal systems and method based on binocular distance measurement, belong to unmanned plane tracking technique field, solve the problems, such as that aerial non-cooperative moving targets can not be tracked in the prior art.Include: unmanned aerial vehicle platform, movement destination image is obtained by binocular camera in real time, carry out benchmark image tracking and binocular solid matching, obtains the 3 d space coordinate of moving target, and approach and track automatically to moving target；Ground control station for guiding unmanned aerial vehicle platform tentatively close to moving target, to receive the movement destination image that unmanned aerial vehicle platform is sent and display, and is tentatively demarcated the target area in image.The advantages of carrying out the three-dimensional localization of target with binocular camera, there is non-contact high-frequency measurement, implement simple and high real-time；By the real-time location information for obtaining moving target, control unmanned plane accurately approaches and tracks automatically non-cooperative moving targets, disposes moving target for subsequent unmanned plane and provides basis.
Description
Technical Field
The invention relates to the technical field of unmanned aerial vehicle tracking, in particular to an unmanned aerial vehicle tracking moving target system and method based on binocular vision ranging.
Background
In recent years, with the development of unmanned aerial vehicle technology, the low-slow small aircraft is rapidly expanded in the field of military and civilian, is easy to be utilized by lawless persons for illegal investigation, spreading of leaflet and even terrorist attack, and brings great hidden danger to public safety and social stability. Therefore, how to effectively manage the target of "low slow small" has become a worldwide problem. At present, the physical intercepting means for controlling the black flying unmanned aerial vehicle have modes such as microwave and laser, but the physical intercepting means has the problems of high cost in use and maintenance, easy secondary damage and the like. Utilize unmanned aerial vehicle to carry on throwing the net device, provide target information through ground photoelectric equipment or radar and machine year vision system, guide unmanned aerial vehicle and carry out net formula soft killer and kill and intercept after being close the target to unmanned aerial vehicle counter-control unmanned aerial vehicle is the feasible mode of a management and control "low slow little" target.
At present, related technologies exist for tracking a moving target by an unmanned aerial vehicle-mounted vision system, but most of the technologies mainly focus on visual image tracking, a target cannot be obtained relative to a specific three-dimensional coordinate of the unmanned aerial vehicle of the same party, the unmanned aerial vehicle-mounted vision system is generally used for tracking a plane target such as a ground target, and the non-cooperative moving target in the air cannot be accurately tracked. In the method in the prior art, an unmanned aerial vehicle is positioned above a target and used for tracking a plane target, the target is a cooperative target, a marked red rectangular frame with obvious characteristics is arranged, and the position of the target is identified based on the obvious characteristics and cannot be used for tracking a non-cooperative target in the air. In the other method, an unmanned aerial vehicle is used for tracking ground pedestrians, and the pedestrian gesture features and the gesture motion model are established for recognition and tracking; it is also impossible to track non-cooperative targets in the air.
Disclosure of Invention
In view of the above analysis, the present invention aims to provide a system and a method for tracking a moving target by an unmanned aerial vehicle based on binocular vision ranging, so as to solve the problem that an aerial non-cooperative moving target cannot be tracked in the prior art.
The purpose of the invention is mainly realized by the following technical scheme:
on the one hand, provide an unmanned aerial vehicle tracking moving target system based on binocular vision range finding, include: unmanned aerial vehicle platform, ground control station;
the unmanned aerial vehicle platform acquires a moving target image in real time through a binocular camera, performs reference image tracking and binocular stereo matching, acquires a space three-dimensional coordinate of the moving target, and automatically approaches and tracks the moving target;
and the ground control station is used for guiding the unmanned aerial vehicle platform to initially approach the moving target, receiving and displaying the moving target image sent by the unmanned aerial vehicle platform, and performing initial calibration on the target area in the image.
The invention has the following beneficial effects:
the system adopts the binocular camera to carry out three-dimensional positioning of the target, and has the advantages of non-contact measurement, simplicity in implementation and high real-time performance. The image frame rate reaches 60 frames per second, namely the update rate of the target position also reaches 60Hz, and the unmanned aerial vehicle is controlled to accurately and automatically approach and track the non-cooperative moving target through the position information of the moving target obtained in real time, so that a basis is provided for the subsequent unmanned aerial vehicle to treat the moving target.
On the basis of the scheme, the invention is further improved as follows:
further, the unmanned aerial vehicle platform includes: the system comprises a binocular camera, an airborne image processing module, an airborne communication module and a motion control module;
the ground control station, comprising: the system comprises an industrial personal computer, a ground communication module and a display module;
the airborne image processing module is respectively connected with the binocular camera, the airborne communication module and the motion control module; the system comprises a binocular camera, a ground control station, a camera and a display, wherein the binocular camera is used for acquiring a moving target image and data sent by the ground;
the airborne communication module sends the image data coded by the airborne image processing module to the ground control station and receives the data of the ground control station;
the motion control module is used for controlling the unmanned aerial vehicle to approach and track the moving target according to the three-dimensional coordinates of the moving target sent by the airborne image processing module;
the ground communication module receives the data transmitted by the airborne communication module and sends the data to the industrial personal computer, and meanwhile, the ground communication module sends the data of the ground control station to the airborne communication module;
the industrial personal computer decodes the received data and sends the data to the display module for display.
Further, the airborne image processing module detects and tracks a target area in each frame of reference image acquired in real time by adopting a KCF algorithm; and performing binocular stereo matching by using the pixel gray value as a matching element according to the coordinate position of the target area in each frame of reference image.
On the other hand, the method for tracking the moving target by the unmanned aerial vehicle based on binocular vision ranging comprises the following steps:
step S1, the ground control station guides the unmanned aerial vehicle platform to approach the moving target preliminarily, so that the moving target enters the field of view of the binocular camera;
step S2, performing binocular stereo matching in a binocular camera field of view, measuring the position of a target through binocular ranging, and acquiring three-dimensional position information of the target relative to the unmanned aerial vehicle in real time;
and step S3, according to the three-dimensional position information obtained in real time, the unmanned aerial vehicle automatically tracks the target quickly and accurately.
The invention has the following beneficial effects:
the method adopts the binocular camera to carry out three-dimensional positioning of the target, and has the advantages of non-contact measurement, simplicity in implementation and high real-time performance. The image frame rate reaches 60 frames per second, namely the update rate of the target position also reaches 60Hz, and the unmanned aerial vehicle is controlled to accurately and automatically approach and track the non-cooperative moving target through the position information of the moving target obtained in real time, so that a basis is provided for the subsequent unmanned aerial vehicle to treat the moving target.
On the basis of the scheme, the invention is further improved as follows:
further, the performing reference image tracking includes:
the unmanned aerial vehicle platform transmits the images acquired by the binocular camera to the ground control station in real time for display;
acquiring a target area calibrated in a display image, and transmitting the position coordinate of the target area to an unmanned aerial vehicle platform;
and detecting and tracking the target area in each frame of reference image acquired in real time by adopting a KCF algorithm according to the position coordinates of the target area.
Further, the performing reference image tracking by using a KCF algorithm includes:
after the coordinate position of a calibrated target area is obtained, constructing and acquiring a positive sample and a negative sample of a target based on image gray scale characteristics, and training a target detector;
in each frame of reference image, detecting in a search area by using the target detector, and taking the area with the highest confidence coefficient as a target area; and when the confidence value of the target area is smaller than a preset value, expanding the range of the search area and searching again.
Further, the unmanned aerial vehicle platform uses the pixel gray value as a matching element to perform binocular stereo matching according to the coordinate position of the target area in each frame of reference image, and the method comprises the following steps:
converting each frame of RGB image acquired by the left eye and the right eye into a gray image;
taking a target area in one of the eye gray level images as a reference, and calculating average pixel point gray level values and corresponding area coordinates of each area with the same size in the same height area in the other eye gray level image at intervals in sequence;
calculating the absolute difference value of the average pixel point gray value of each area and the reference target area,
and when the minimum value is smaller than a preset minimum value and the difference between the second minimum value and the minimum value is smaller than a preset difference value, taking the area corresponding to the minimum value as a target area in the other matched target image.
Further, the step of acquiring the three-dimensional position information of the moving target in real time through binocular ranging includes performing Kalman filtering to remove outliers appearing in part of the frame images when the three-dimensional coordinate depth value of the moving target is obtained.
Further, according to the three-dimensional position information obtained in real time, the unmanned aerial vehicle can quickly and accurately track the target automatically, and the method comprises the following steps: and constructing a position PID controller according to the real-time three-dimensional coordinate information of the target as an input value, and controlling the unmanned aerial vehicle to quickly track the approaching target.
Further, still include: and when the unmanned aerial vehicle platform tracks the wrong target, cancelling the current tracking and acquiring the calibrated target area again.
In the invention, the technical schemes can be combined with each other to realize more preferable combination schemes. Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The drawings are only for purposes of illustrating particular embodiments and are not to be construed as limiting the invention, wherein like reference numerals are used to designate like parts throughout.
FIG. 1 is a structural diagram of a system for tracking a moving target by an unmanned aerial vehicle based on binocular vision ranging in the embodiment of the invention;
fig. 2 is a flowchart of a method for tracking a moving target by an unmanned aerial vehicle based on binocular vision ranging in the embodiment of the invention.
Detailed Description
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate preferred embodiments of the invention and together with the description, serve to explain the principles of the invention and not to limit the scope of the invention.
Example 1
The invention discloses a binocular vision ranging-based unmanned aerial vehicle tracking moving target system, as shown in fig. 1, comprising: unmanned aerial vehicle platform, ground control station;
the unmanned aerial vehicle platform acquires a moving target image in real time through a binocular camera, performs reference image tracking and binocular stereo matching, acquires a space three-dimensional coordinate of the moving target, and automatically approaches and tracks the moving target;
and the ground control station is used for guiding the unmanned aerial vehicle platform to initially approach the moving target, receiving and displaying the moving target image sent by the unmanned aerial vehicle platform, and initially calibrating the target area in the image.
During implementation, after the ground control station and other auxiliary equipment are used for guiding the unmanned aerial vehicle platform to approach the moving target preliminarily, the moving target enters the field of view of the binocular camera, three-dimensional position information of the moving target relative to the unmanned aerial vehicle is acquired in real time through binocular stereo matching, the unmanned aerial vehicle is controlled to approach the target quickly and accurately and perform automatic tracking, and subsequent unmanned aerial vehicles are convenient to perform interception treatment such as net throwing.
Compared with the prior art, the binocular vision ranging unmanned aerial vehicle tracking moving target system provided by the embodiment adopts the binocular camera to perform three-dimensional positioning of the target, and has the advantages of non-contact measurement, simplicity in implementation and high real-time performance. The image frame rate reaches 60 frames per second, namely the update rate of the target position also reaches 60Hz, and the unmanned aerial vehicle is controlled to accurately and automatically approach and track the non-cooperative moving target through the position information of the moving target obtained in real time, so that a basis is provided for the subsequent unmanned aerial vehicle to treat the moving target.
Particularly, unmanned aerial vehicle platform installs binocular camera, still includes: the system comprises an airborne image processing module, an airborne communication module and a motion control module; wherein,
the airborne image processing module is respectively connected with the binocular camera, the airborne communication module and the motion control module; the system comprises a double-sided camera, a ground control station, a binocular stereo matching unit and a camera, wherein the double-sided camera is used for acquiring a moving target image and data sent by the ground control station; specifically, a KCF algorithm is adopted to detect and track a target area in each frame of reference image acquired in real time; and performing binocular stereo matching by using the pixel gray value as a matching element according to the coordinate position of the target area in each frame of reference image.
The airborne communication module is connected with the image processing module through the network port, receives data coded by the airborne image processing module and transmits the data to the ground control station; meanwhile, receiving data sent by a ground control station;
the motion control module is used for controlling the unmanned aerial vehicle to approach and track the target according to the three-dimensional coordinates of the target point sent by the airborne image processing module;
a ground control station comprising: the system comprises an industrial personal computer, a ground communication module and a display module;
the ground communication module receives the data transmitted by the airborne communication module and sends the data to the industrial personal computer; meanwhile, the data of the ground control station is sent to the airborne communication module.
And the industrial personal computer is used for decoding the received data and sending the data to the display module for displaying.
The display module is used for displaying images and data information and can be a display or a touch screen.
Example 2
The embodiment discloses a method for tracking a moving target by an unmanned aerial vehicle based on binocular vision ranging, which is implemented by adopting the system in embodiment 1, and as shown in fig. 2, the method comprises the following steps:
step S1, the ground control station guides the unmanned aerial vehicle to approach the target preliminarily, so that the target enters the field of view of the binocular camera;
step S2, in the field of view of the binocular camera, reference image tracking and binocular stereo matching are carried out, and three-dimensional position information of the moving target is obtained in real time through binocular ranging;
and step S3, according to the three-dimensional position information obtained in real time, the unmanned aerial vehicle platform automatically tracks the target quickly and accurately.
Compared with the prior art, the method for tracking the moving target by the unmanned aerial vehicle based on binocular vision ranging, provided by the embodiment, adopts the binocular camera to perform three-dimensional positioning of the target, and has the advantages of non-contact measurement, simplicity in implementation and high real-time performance. The image frame rate reaches 60 frames per second, namely the update rate of the target position also reaches 60Hz, and the unmanned aerial vehicle is controlled to accurately and automatically approach and track the non-cooperative moving target through the position information of the moving target obtained in real time, so that a basis is provided for the subsequent unmanned aerial vehicle to treat the moving target.
Specifically, in step S1, the ground control station guides the drone to initially approach the target, and the ground operator may control the drone platform to gradually approach the moving target using a remote control or other ground-assisted device until the moving target enters the field of view of the binocular camera (left and right), i.e., the moving target is clearly visible on the display screen of the ground control station.
In the process that the unmanned aerial vehicle initially approaches the moving target, the airborne image processing module does not process the image of the binocular camera, the video stream is subjected to H264 real-time coding and is transmitted to the airborne communication module, and the airborne communication module sends the video stream to the ground control station for displaying.
In step S2, after it is determined that the moving object enters the field of view of the binocular camera, the drone platform performs reference image tracking and binocular stereo matching, and acquires three-dimensional position information of the moving object relative to the drone platform in real time through binocular ranging; specifically, the method comprises the following steps:
step S201, the unmanned aerial vehicle platform transmits the image acquired by the binocular camera to a ground control station in real time;
when the target enters the field of view of the binocular camera, the airborne image processing module acquires images and does not process the images, the images are directly transmitted to the airborne communication module after being coded, and then the images are transmitted to the ground control station to be displayed in real time.
Step S202, a target area calibrated in a display image is obtained, and the position coordinates of the target area are transmitted to an unmanned aerial vehicle platform;
the ground operator can determine a target area by drawing a rectangle in a video image of the display/touch screen through a mouse/gesture; and clicking near the center of the target area in a mouse left button/finger clicking mode, and taking a certain range of near area as the target area according to a set threshold value. The position coordinates of the target area are represented by two vertex coordinate values on a certain diagonal line in a rectangle, and the ground control station obtains the position coordinates of the target area in an original image according to the size ratio of a display image to the original image (and the size ratio of the display image to the image in the binocular camera) and sends the position coordinates to an airborne image processing module of the unmanned aerial vehicle platform;
it should be noted that, when a binocular camera is used for distance measurement to obtain the position of a moving target, a certain image of a target in the camera is used as a reference, and the other image of the target is subjected to binocular stereo matching; in the embodiment, a left eye image is used as a reference image for image processing, and a right eye image is only used for binocular stereo matching; therefore, the operator needs to frame the target area in the display/touch screen displaying the left eye image.
Step S203, after receiving the coordinate position of the target area, the unmanned aerial vehicle platform automatically tracks the target area of the reference image of each frame, determines the coordinate position of the target area in each frame of image, and then enters step S204;
the image tracking is automatically carried out in the left eye so as to carry out binocular matching in real time, the image tracking can adopt various existing algorithms, in the embodiment, a KCF (Kernel Correlation Filter) algorithm is adopted, the image tracking algorithm with excellent tracking effect and tracking speed is mainly used for training a discriminant classifier by using a Kernel Correlation Filter, sample acquisition is carried out by using a cyclic matrix, and the algorithm is accelerated by using fast Fourier change; considering the problems that when a moving target moves fast, the tracking fails due to the fact that a detection window is too small, and the like, the self-adaptive adjustment of the detection window is added, namely when the confidence value of the current search area matched with the target image is too low, the search range is expanded to match and track again; meanwhile, optimization is carried out in combination with the confidence coefficient of the matched target image in the updating process of the target detector; so as to improve the accuracy and adaptability of image tracking.
Specifically, after acquiring the coordinate position of a target area, an airborne image processing module constructs a positive sample and a negative sample of an acquired target image based on image gray scale features, trains a target detector, detects the target image area in a target search area by using the target detector in each frame of left-eye image, and takes the area with the highest confidence coefficient as the target area to realize automatic tracking of the image (target area). When the confidence value of the target area image with the highest confidence detected in the search area is smaller than a preset value (preferably, 0.6), the target search range is expanded and searched again until the requirement is met, so that the KCF tracking algorithm can adapt to the fast moving target.
Step S204, the unmanned aerial vehicle platform performs binocular stereo matching by taking the pixel gray value as a matching element according to the coordinate position of a target area in each frame of image;
the airborne image processing module selects the pixel gray value of a target area as a matching element, performs binocular vision matching in the same-size areas with the same height (namely the same horizontal height) of the right eye block by block, finds out the target area by taking the average pixel gray absolute difference value of the areas as an evaluation standard, and calculates the target position. Specifically, the method comprises the following steps:
step S20401, converting the RGB images acquired by the left and right cameras into a gray-scale image;
step S20402, calculating the average pixel point gray value and the area coordinates of each area with the same size from left to right at intervals in the areas with the same height in the right eye gray scale map by taking the target area in the left eye gray scale map as a reference;
step S20403, calculating the absolute difference value of the average pixel point gray values of each area and the left target area;
step S20404, a minimum value dmincdiff and a next-smallest value dsecondiff of the absolute difference are obtained, and when the minimum value is smaller than a preset minimum value (preferably, 25 or less) and a difference between the next-smallest value and the minimum value is smaller than a preset difference value (preferably, 8), a region corresponding to the minimum value is used as a matching target region, that is, a target region is found in the right-side view field.
And step S205, according to the stereo matched binocular target area, on the basis of the pixel coordinates of the central point of the target area of the left and right visual fields, and according to the binocular distance measurement principle, the space three-dimensional coordinates of the moving target relative to the camera focus are obtained.
It should be noted that, when the three-dimensional coordinate depth value of the moving target (the vertical distance of the moving target relative to the plane where the image coordinate of the unmanned aerial vehicle platform is located) is obtained, kalman filtering is performed, so that the error outliers occurring in a part of frames can be effectively removed. Therefore, the accuracy of the target tracking and ranging algorithm is improved.
In step S3, the drone automatically tracks the target quickly and accurately according to the three-dimensional position information acquired in real time in step S2.
And constructing a position PID controller according to the real-time three-dimensional coordinate information of the target as an input value, controlling the unmanned aerial vehicle to quickly track and approach the target, achieving the expected interception distance and facilitating the unmanned aerial vehicle to intercept the target.
In order to further ensure the stability of the moving target in the tracking process and avoid the problems of tracking errors and the like caused by external environmental factors, in the tracking and ranging process of the target image, if a ground operator finds that the unmanned aerial vehicle platform tracks an error target, the ground operator can send a signal to the airborne image processing module in a mode of clicking a right mouse button and the like, cancel the current tracking, select a calibrated target area again, repeat the steps S202-S3 and continue to track the moving target.
Specifically, the method embodiment and the apparatus embodiment are based on the same inventive concept, and specific implementation points can be referred to each other.
Those skilled in the art will appreciate that all or part of the flow of the method implementing the above embodiments may be implemented by hardware associated with computer program instructions, and the program may be stored in a computer readable storage medium. The computer readable storage medium is a magnetic disk, an optical disk, a read-only memory or a random access memory.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention.
Claims (10)
1. The utility model provides an unmanned aerial vehicle tracking moving target system based on binocular vision range finding which characterized in that includes: unmanned aerial vehicle platform, ground control station;
the unmanned aerial vehicle platform acquires a moving target image in real time through a binocular camera, performs reference image tracking and binocular stereo matching, acquires a space three-dimensional coordinate of the moving target, and automatically approaches and tracks the moving target;
and the ground control station is used for guiding the unmanned aerial vehicle platform to initially approach the moving target, receiving and displaying the moving target image sent by the unmanned aerial vehicle platform, and performing initial calibration on the target area in the image.
2. The method of claim 1, wherein the drone platform comprises: the system comprises a binocular camera, an airborne image processing module, an airborne communication module and a motion control module;
the ground control station, comprising: the system comprises an industrial personal computer, a ground communication module and a display module;
the airborne image processing module is respectively connected with the binocular camera, the airborne communication module and the motion control module; the system comprises a binocular camera, a ground control station, a camera and a display, wherein the binocular camera is used for acquiring a moving target image and data sent by the ground;
the airborne communication module sends the data coded by the airborne image processing module to the ground control station and receives the data of the ground control station;
the motion control module is used for controlling the unmanned aerial vehicle to approach and track the moving target according to the three-dimensional coordinates of the moving target sent by the airborne image processing module;
the ground communication module receives the data transmitted by the airborne communication module and sends the data to the industrial personal computer, and meanwhile, the ground communication module sends the data of the ground control station to the airborne communication module;
the industrial personal computer decodes the received data and sends the data to the display module for display.
3. The system according to claim 2, wherein the airborne image processing module adopts a KCF algorithm to detect and track the target area in each frame of reference image acquired in real time; and performing binocular stereo matching by using the pixel gray value as a matching element according to the coordinate position of the target area in each frame of reference image.
4. A method for tracking a moving target by an unmanned aerial vehicle based on binocular vision ranging by applying the system of any one of claims 1 to 3, comprising the following steps:
the ground control station guides the unmanned aerial vehicle platform to approach the moving target preliminarily, so that the moving target enters the field of view of the binocular camera;
in a binocular camera field of view, performing reference image tracking and binocular stereo matching, and acquiring three-dimensional position information of a moving target in real time through binocular ranging;
and according to the three-dimensional position information acquired in real time, the unmanned aerial vehicle platform carries out rapid and accurate automatic tracking on the target.
5. The method of claim 4, wherein the performing reference image tracking comprises:
the unmanned aerial vehicle platform transmits the images acquired by the binocular camera to the ground control station in real time for display;
acquiring a target area calibrated in a display image, and transmitting the position coordinate of the target area to an unmanned aerial vehicle platform;
and detecting and tracking the target area in each frame of reference image acquired in real time by adopting a KCF algorithm according to the position coordinates of the target area.
6. The method of claim 5, wherein the performing reference image tracking using a KCF algorithm comprises:
after the coordinate position of a calibrated target area is obtained, constructing and acquiring a positive sample and a negative sample of a target based on image gray scale characteristics, and training a target detector;
in each frame of reference image, detecting in a search area by using the target detector, and taking the area with the highest confidence coefficient as a target area; and when the confidence value of the target area is smaller than a preset value, expanding the range of the search area and searching again.
7. The method of claim 6, wherein the unmanned aerial vehicle platform performs binocular stereo matching with pixel gray scale values as matching primitives according to the coordinate position of the target area in each frame of reference image, and comprises:
converting each frame of RGB image acquired by the left eye and the right eye into a gray image;
taking a target area in one of the eye gray level images as a reference, and calculating average pixel point gray level values and corresponding area coordinates of each area with the same size in the same height area in the other eye gray level image at intervals in sequence;
calculating the absolute difference value of the average pixel point gray value of each area and the reference target area;
and when the minimum value is smaller than a preset minimum value and the difference between the second minimum value and the minimum value is smaller than a preset difference value, taking the area corresponding to the minimum value as a target area in the other matched target image.
8. The method of claim 7, wherein the obtaining of the three-dimensional position information of the moving object in real time through binocular ranging includes performing kalman filtering to remove outliers appearing in the partial frame image when the depth value of the three-dimensional coordinates of the moving object is obtained.
9. The method according to claim 8, wherein the fast and accurate automatic tracking of the target by the drone according to the three-dimensional position information obtained in real time comprises: and constructing a position PID controller according to the real-time three-dimensional coordinate information of the target as an input value, and controlling the unmanned aerial vehicle to quickly track the approaching target.
10. The method of claim 9, further comprising: and when the unmanned aerial vehicle platform tracks the wrong target, cancelling the current tracking and acquiring the calibrated target area again.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811100610.3A CN109407697A (en) | 2018-09-20 | 2018-09-20 | A kind of unmanned plane pursuit movement goal systems and method based on binocular distance measurement |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811100610.3A CN109407697A (en) | 2018-09-20 | 2018-09-20 | A kind of unmanned plane pursuit movement goal systems and method based on binocular distance measurement |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109407697A true CN109407697A (en) | 2019-03-01 |
Family
ID=65465142
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811100610.3A Pending CN109407697A (en) | 2018-09-20 | 2018-09-20 | A kind of unmanned plane pursuit movement goal systems and method based on binocular distance measurement |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109407697A (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110018692A (en) * | 2019-04-24 | 2019-07-16 | 五邑大学 | A kind of unmanned plane method of guidance, system, device and storage medium |
| CN110262534A (en) * | 2019-06-25 | 2019-09-20 | 北京机械设备研究所 | A kind of unmanned plane tracking and intercepting system and method for low slow Small object |
| CN110287957A (en) * | 2019-06-24 | 2019-09-27 | 北京中电慧视科技有限公司 | A kind of localization method and positioning device of low slow Small object |
| CN110393165A (en) * | 2019-07-11 | 2019-11-01 | 浙江大学宁波理工学院 | A kind of off-lying sea cultivation net cage bait-throwing method based on Autoamtic bait putting ship |
| CN110400333A (en) * | 2019-07-26 | 2019-11-01 | 中国安全生产科学研究院 | Coach's formula binocular stereo vision device and High Precision Stereo visual pattern acquisition methods |
| CN110647156A (en) * | 2019-09-17 | 2020-01-03 | 中国科学院自动化研究所 | Target object docking ring-based docking equipment pose adjusting method and system |
| CN110764537A (en) * | 2019-12-25 | 2020-02-07 | 中航金城无人系统有限公司 | Automatic tripod head locking system and method based on motion estimation and visual tracking |
| CN111754542A (en) * | 2019-03-27 | 2020-10-09 | 顺丰科技有限公司 | Target object determination method, tracking method, device, equipment and storage medium thereof |
| CN112539732A (en) * | 2020-12-04 | 2021-03-23 | 杭州电子科技大学 | Unmanned aerial vehicle cluster state and trajectory data acquisition platform |
| WO2021128189A1 (en) * | 2019-12-26 | 2021-07-01 | 深圳市大疆创新科技有限公司 | Data processing method and apparatus, unmanned aerial vehicle and flight control system |
| CN113365030A (en) * | 2021-06-01 | 2021-09-07 | 珠海大横琴科技发展有限公司 | Multi-angle target tracking method and tracking system |
| CN113449566A (en) * | 2020-03-27 | 2021-09-28 | 北京机械设备研究所 | Intelligent image tracking method and system for low-speed small target in human-in-loop |
| WO2021217372A1 (en) * | 2020-04-27 | 2021-11-04 | 深圳市大疆创新科技有限公司 | Control method and device for movable platform |
| CN114127510A (en) * | 2019-06-04 | 2022-03-01 | 菲力尔无人机系统公司 | 3D localization and mapping system and method |
| CN114155290A (en) * | 2021-11-18 | 2022-03-08 | 合肥富煌君达高科信息技术有限公司 | System and method for large-field-of-view high-speed motion measurement |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106444829A (en) * | 2016-09-22 | 2017-02-22 | 北京机械设备研究所 | Guidance unmanned aerial vehicle net cast interception method for "low-altitude, slow-speed and small" target |
| CN107424177A (en) * | 2017-08-11 | 2017-12-01 | 哈尔滨工业大学（威海） | Positioning amendment long-range track algorithm based on serial correlation wave filter |
| CN107610157A (en) * | 2016-07-12 | 2018-01-19 | 深圳雷柏科技股份有限公司 | A kind of unmanned plane target method for tracing and system |
| CN107621266A (en) * | 2017-08-14 | 2018-01-23 | 上海宇航系统工程研究所 | The space non-cooperative target Relative Navigation of distinguished point based tracking |
| US20180095479A1 (en) * | 2016-03-31 | 2018-04-05 | Ninebot (Beijing) Tech. Co., Ltd | Method, mobile device and computer storage medium for processing information |
| CN107894590A (en) * | 2017-10-17 | 2018-04-10 | 湖北林青测控科技有限公司 | New full passive type unmanned plane management-control method, storage medium, electronic equipment |
| CN108010067A (en) * | 2017-12-25 | 2018-05-08 | 北京航空航天大学 | A kind of visual target tracking method based on combination determination strategy |
| CN108062764A (en) * | 2017-11-30 | 2018-05-22 | 极翼机器人(上海)有限公司 | A kind of object tracking methods of view-based access control model |
-
2018
- 2018-09-20 CN CN201811100610.3A patent/CN109407697A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180095479A1 (en) * | 2016-03-31 | 2018-04-05 | Ninebot (Beijing) Tech. Co., Ltd | Method, mobile device and computer storage medium for processing information |
| CN107610157A (en) * | 2016-07-12 | 2018-01-19 | 深圳雷柏科技股份有限公司 | A kind of unmanned plane target method for tracing and system |
| CN106444829A (en) * | 2016-09-22 | 2017-02-22 | 北京机械设备研究所 | Guidance unmanned aerial vehicle net cast interception method for "low-altitude, slow-speed and small" target |
| CN107424177A (en) * | 2017-08-11 | 2017-12-01 | 哈尔滨工业大学（威海） | Positioning amendment long-range track algorithm based on serial correlation wave filter |
| CN107621266A (en) * | 2017-08-14 | 2018-01-23 | 上海宇航系统工程研究所 | The space non-cooperative target Relative Navigation of distinguished point based tracking |
| CN107894590A (en) * | 2017-10-17 | 2018-04-10 | 湖北林青测控科技有限公司 | New full passive type unmanned plane management-control method, storage medium, electronic equipment |
| CN108062764A (en) * | 2017-11-30 | 2018-05-22 | 极翼机器人(上海)有限公司 | A kind of object tracking methods of view-based access control model |
| CN108010067A (en) * | 2017-12-25 | 2018-05-08 | 北京航空航天大学 | A kind of visual target tracking method based on combination determination strategy |
Non-Patent Citations (4)
| Title |
|---|
| LUIS RODOLFO GARCIA CARRILLO: "《四旋冀无人机控制 基于视觉的悬停与导航》", 30 June 2016, 国防工业出版社 * |
| 彭鹏菲 等: "《舰载无人机系统及作战应用》", 31 January 2016, 国防工业出版社 * |
| 杜军平 等: "《跨尺度运动图像的目标检测与跟踪》", 30 June 2018, 北京邮电大学出版社 * |
| 齐俊桐 等: "《无人机输电线路巡检与架设技术》", 30 November 2014, 东北大学出版社 * |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111754542A (en) * | 2019-03-27 | 2020-10-09 | 顺丰科技有限公司 | Target object determination method, tracking method, device, equipment and storage medium thereof |
| CN110018692A (en) * | 2019-04-24 | 2019-07-16 | 五邑大学 | A kind of unmanned plane method of guidance, system, device and storage medium |
| CN114127510A (en) * | 2019-06-04 | 2022-03-01 | 菲力尔无人机系统公司 | 3D localization and mapping system and method |
| CN110287957A (en) * | 2019-06-24 | 2019-09-27 | 北京中电慧视科技有限公司 | A kind of localization method and positioning device of low slow Small object |
| CN110287957B (en) * | 2019-06-24 | 2021-09-07 | 北京中电慧视科技有限公司 | Low-slow small target positioning method and positioning device |
| CN110262534A (en) * | 2019-06-25 | 2019-09-20 | 北京机械设备研究所 | A kind of unmanned plane tracking and intercepting system and method for low slow Small object |
| CN110393165A (en) * | 2019-07-11 | 2019-11-01 | 浙江大学宁波理工学院 | A kind of off-lying sea cultivation net cage bait-throwing method based on Autoamtic bait putting ship |
| CN110393165B (en) * | 2019-07-11 | 2021-06-25 | 浙江大学宁波理工学院 | Open sea aquaculture net cage bait feeding method based on automatic bait feeding boat |
| CN110400333A (en) * | 2019-07-26 | 2019-11-01 | 中国安全生产科学研究院 | Coach's formula binocular stereo vision device and High Precision Stereo visual pattern acquisition methods |
| CN110400333B (en) * | 2019-07-26 | 2020-06-26 | 中国安全生产科学研究院 | Training binocular stereoscopic vision device and high-precision stereoscopic vision image acquisition method |
| CN110647156B (en) * | 2019-09-17 | 2021-05-11 | 中国科学院自动化研究所 | Target object docking ring-based docking equipment pose adjusting method and system |
| CN110647156A (en) * | 2019-09-17 | 2020-01-03 | 中国科学院自动化研究所 | Target object docking ring-based docking equipment pose adjusting method and system |
| CN110764537A (en) * | 2019-12-25 | 2020-02-07 | 中航金城无人系统有限公司 | Automatic tripod head locking system and method based on motion estimation and visual tracking |
| WO2021128189A1 (en) * | 2019-12-26 | 2021-07-01 | 深圳市大疆创新科技有限公司 | Data processing method and apparatus, unmanned aerial vehicle and flight control system |
| CN113474741A (en) * | 2019-12-26 | 2021-10-01 | 深圳市大疆创新科技有限公司 | Data processing method and device, unmanned aerial vehicle and flight control system |
| CN113449566A (en) * | 2020-03-27 | 2021-09-28 | 北京机械设备研究所 | Intelligent image tracking method and system for low-speed small target in human-in-loop |
| CN113449566B (en) * | 2020-03-27 | 2024-05-07 | 北京机械设备研究所 | Intelligent image tracking method and system for 'low-small' target of human in loop |
| WO2021217372A1 (en) * | 2020-04-27 | 2021-11-04 | 深圳市大疆创新科技有限公司 | Control method and device for movable platform |
| CN114096931A (en) * | 2020-04-27 | 2022-02-25 | 深圳市大疆创新科技有限公司 | Control method and device for movable platform |
| CN112539732A (en) * | 2020-12-04 | 2021-03-23 | 杭州电子科技大学 | Unmanned aerial vehicle cluster state and trajectory data acquisition platform |
| CN113365030A (en) * | 2021-06-01 | 2021-09-07 | 珠海大横琴科技发展有限公司 | Multi-angle target tracking method and tracking system |
| CN114155290A (en) * | 2021-11-18 | 2022-03-08 | 合肥富煌君达高科信息技术有限公司 | System and method for large-field-of-view high-speed motion measurement |
| CN114155290B (en) * | 2021-11-18 | 2022-09-09 | 合肥富煌君达高科信息技术有限公司 | System and method for large-field-of-view high-speed motion measurement |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109407697A (en) | A kind of unmanned plane pursuit movement goal systems and method based on binocular distance measurement | |
| US11915502B2 (en) | Systems and methods for depth map sampling | |
| CN111326023B (en) | Unmanned aerial vehicle route early warning method, device, equipment and storage medium | |
| EP3469306B1 (en) | Geometric matching in visual navigation systems | |
| CN104820998B (en) | A kind of human testing based on unmanned motor platform and tracking and device | |
| Levinson et al. | Traffic light mapping, localization, and state detection for autonomous vehicles | |
| CN102436738B (en) | Traffic monitoring device based on unmanned aerial vehicle (UAV) | |
| CN106447680B (en) | The object detecting and tracking method that radar is merged with vision under dynamic background environment | |
| US20150138310A1 (en) | Automatic scene parsing | |
| CN103824070A (en) | Rapid pedestrian detection method based on computer vision | |
| KR20180042254A (en) | Systems and methods for object tracking | |
| CN107610157B (en) | Unmanned aerial vehicle target tracking method and system | |
| CN108776974B (en) | A kind of real-time modeling method method suitable for public transport scene | |
| CN105760846A (en) | Object detection and location method and system based on depth data | |
| KR20150049529A (en) | Apparatus and method for estimating the location of the vehicle | |
| CN109828267A (en) | The Intelligent Mobile Robot detection of obstacles and distance measuring method of Case-based Reasoning segmentation and depth camera | |
| CN112906777A (en) | Target detection method and device, electronic equipment and storage medium | |
| CN106056624A (en) | Unmanned aerial vehicle high-definition image small target detecting and tracking system and detecting and tracking method thereof | |
| KR20180133745A (en) | Flying object identification system using lidar sensors and pan/tilt zoom cameras and method for controlling the same | |
| CN114034296A (en) | Navigation signal interference source detection and identification method and system | |
| CN108090922A (en) | Intelligent Target pursuit path recording method | |
| CN114511592B (en) | Personnel track tracking method and system based on RGBD camera and BIM system | |
| CN110287957B (en) | Low-slow small target positioning method and positioning device | |
| CN117406234A (en) | Target ranging and tracking method based on single-line laser radar and vision fusion | |
| CN111489398B (en) | Imaging equipment calibration method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |