CN106527480A - Multi-sensor fusion obstacle avoiding system of unmanned aerial vehicle and method - Google Patents

Abstract

The invention discloses a multi-sensor fusion obstacle avoidance control system and method for an unmanned aerial vehicle. The system includes an ultrasonic ranging sensor, a binocular ranging sensor, an airborne vision processor, and a fusion obstacle avoidance processor; the unmanned aerial vehicle includes The airborne flight control system and the remote control signal receiver, the ultrasonic ranging sensor, the airborne vision processor and the remote control signal receiver are respectively connected with the fusion obstacle avoidance processor; the binocular ranging sensor is connected with the airborne vision processor. The method of the present invention expands on the basis of traditional ultrasonic obstacle avoidance, so that it can fuse the depth information collected by binocular stereo vision, and output the flight mode information required by the flight control system by fusing the decision-making and data conversion of the obstacle avoidance processor , to achieve obstacle avoidance. The present invention does not need to change the structure and parameters of the existing flight control system, can effectively avoid the complex adaptation of the obstacle avoidance system and various flight control system products, has high practicability, and is suitable for popularization and installation on various types of unmanned aerial vehicles.

Description

A multi-sensor fusion obstacle avoidance control system and method for an unmanned aerial vehicle

technical field

The invention relates to a multi-sensor fusion obstacle avoidance control system and method for an unmanned aerial vehicle, belonging to the technical field of unmanned aerial vehicle fusion obstacle avoidance.

Background technique

In recent years, with the continuous development of the UAV open source community at home and abroad and the vigorous promotion of the UAV capital market, a large number of UAV manufacturers have been born around the world. Various types of drones are widely used in geological reconnaissance, forest fire prevention, resource survey, film and television aerial photography, power line inspection, large-scale cluster performances, 3D modeling and other scenarios. At the same time, the user group of drones is expanded by professional aircraft model enthusiasts From ordinary amateurs, this makes the requirements of consumer groups for drone flight platforms also continue to increase, gradually changing from stable flight to intelligent flight under complex terrain. In the process of using UAVs, there will be a series of safety problems, such as the damage of the UAV platform and airborne equipment caused by the collision of UAVs with obstacles on the route, and the threat to the safety of public life and property.

However, although mature open-source flight control systems such as APM and Pixhawk are widely popularized in the current market, the professional technology threshold required for real engineering realization is too high, and it is difficult for ordinary drone enthusiasts to carry out secondary development and use, even if there are Professional R&D teams can achieve obstacle avoidance after technical support, but with the need for software upgrades, the cost of code maintenance is relatively high, and it is difficult to promote its use. At the same time, most commercial flight controllers are not open-source, and do not support obstacle avoidance by modifying the source code and adding sensors. Even if individual manufacturers such as DJI develop a Guidance visual obstacle avoidance system that supports secondary development, it is still expensive and complicated to debug. , the problem of fewer control interfaces. Therefore, for most ordinary UAV enthusiasts, there is an urgent need for a set of UAV stereo vision obstacle avoidance modules with low cost, simple debugging, precise control and generalization on different flight platforms.

Contents of the invention

The technical problem to be solved by the present invention is to provide a multi-sensor fusion obstacle avoidance control system and method for UAVs, which can realize real-time and dynamic monitoring of surrounding obstacle information by UAVs and autonomously realize avoidance.

The present invention adopts the following technical solutions for solving the problems of the technologies described above:

A multi-sensor fusion obstacle avoidance control system for an unmanned aerial vehicle, the unmanned aerial vehicle includes an airborne flight control system, a remote control signal receiver, and the system also includes an ultrasonic ranging sensor, a binocular ranging sensor, and an airborne visual processing device and fusion obstacle avoidance processor; the ultrasonic ranging sensor measures the distance information of the obstacle in front of the UAV in real time and transmits it to the fusion obstacle avoidance processor; the binocular ranging sensor acquires the image information of the obstacle in front of the UAV in real time And sent to the airborne vision processor, the airborne vision processor calculates the depth information of the obstacle according to the image information and sends it to the fusion obstacle avoidance processor at the same time; the fusion obstacle avoidance processor filters and fuses the distance information and depth information to obtain the The distance between the machine and the obstacle in front, calculate the threat factor of the obstacle according to the distance, and judge whether the obstacle is a threat according to the preset threshold value; The obstacle avoidance processor directly transmits the ground remote control signal received by the remote control signal receiver to the airborne flight control system, and the flight of the UAV is controlled by the ground remote control signal; when the threat factor is greater than or equal to the preset threshold, the fusion obstacle avoidance The processor calculates the obstacle avoidance mode channel signal, and replaces the flight mode channel signal in the ground remote control signal received by the remote control signal receiver with the obstacle avoidance mode channel signal, and controls the UAV to avoid obstacles by the obstacle avoidance mode channel signal.

As a preferred solution of the system of the present invention, the remote control signal receiver, ultrasonic ranging sensor, and airborne vision processor are all connected to the fusion obstacle avoidance processor through the UART, and the fusion obstacle avoidance processor is connected to the airborne flight control system through the UART. system connection.

As a preferred solution of the system of the present invention, the binocular ranging sensor adopts a variable baseline structure of the USB2.0 bus transmission protocol, and the initial baseline distance is 63mm.

As a preferred solution of the system of the present invention, the integrated obstacle avoidance processor adopts a STM32F407 chip.

As a preferred solution of the system of the present invention, the ultrasonic distance measuring sensor adopts a double-tube transceiver structure, and the model is HC-SR04.

A multi-sensor fusion obstacle avoidance control method for an unmanned aerial vehicle, comprising the steps of:

Step 1, using ultrasonic waves to measure the distance information of obstacles in front of the UAV in real time;

Step 2, use the binocular camera to obtain the image information of the obstacle in front of the UAV in real time, and calculate the depth information of the obstacle in front of the UAV according to the image information;

Step 3, filter and fuse the distance information and depth information through the filtering algorithm, and solve the threat factor of the obstacle, judge whether there is a threat to the obstacle according to the threat factor, and when the obstacle is not threatening, the ground information received by the UAV is The remote control signal is directly transmitted to the airborne flight control system; when obstacles are threatening, the obstacle avoidance mode channel signal is calculated, and the flight mode channel signal in the ground remote control signal is replaced with the obstacle avoidance mode channel signal to control the UAV to avoid obstacles action.

As a preferred solution of the method of the present invention, the specific process of calculating the depth information of the obstacle in front of the UAV according to the image information in step 2 is as follows:

Set the binocular baseline distance of the binocular ranging sensor, and perform offline visual calibration of the binocular ranging sensor, and calculate the internal and external system parameters of the binocular ranging sensor according to the imaging correction of the three-dimensional calibration target in the binocular cross field of view and focal length, using the internal and external system parameters and focal length to perform real-time correction, matching and three-dimensional reconstruction of image information, so as to obtain the depth information of obstacles in front of the UAV.

Compared with the prior art, the present invention adopts the above technical scheme and has the following technical effects:

1. In the present invention, the remote control signal receiver, the on-board vision processor, and the ultrasonic ranging sensor are respectively connected to the fusion obstacle avoidance processor, and the obstacle warning information and control signals are directly connected to the system obstacle avoidance feedback loop to realize the simulation of human eyes. The closed-loop position control of the loop finally realizes the automatic perception and avoidance of the multi-rotor UAV in any outdoor scene.

2. The present invention provides a set of simple and feasible UAV multi-sensor fusion obstacle avoidance system. It does not need to change the structure and parameters of the existing flight control system for installation and use, and can effectively avoid complicated obstacle avoidance systems and various flight control systems. The adaptation of the product realizes the collision detection and real-time avoidance between drones and obstacles at the same time. It has high universality and is suitable for promotion and installation on various drones. It has a simple structure and low manufacturing and processing costs.

Description of drawings

Fig. 1 is the overall structural diagram of the multi-sensor fusion obstacle avoidance control system of the unmanned aerial vehicle of the present invention.

Fig. 2 is a flow chart of the implementation of the multi-sensor fusion obstacle avoidance control method of the unmanned aerial vehicle of the present invention.

Fig. 3 is a schematic perspective view of an embodiment of the present invention.

Among them, 1-power system, 2-ultrasonic ranging sensor, 3-binocular ranging sensor, 4-fusion obstacle avoidance processor, 5-airborne vision processor, 6-airborne flight control system, 7-remote control signal receiver.

detailed description

Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

As shown in Figure 1, a multi-sensor fusion obstacle avoidance control system for unmanned aerial vehicles is composed of a binocular ranging sensor 3, an ultrasonic ranging sensor 2, an airborne vision processor 5, and a fusion obstacle avoidance processor 4; The man-machine includes an airborne flight control system 6 , a remote control signal receiver 7 and a power system 1 . Its connection structure is: the binocular ranging sensor is connected to the onboard vision processor, the ultrasonic ranging sensor, the onboard vision processor, and the remote control signal receiver are respectively connected to the fusion obstacle avoidance processor, and the fusion obstacle avoidance controller is also connected to the airborne flight controller. connected to the control system. As shown in FIG. 3 , it is a schematic diagram of an embodiment of the present invention.

The invention provides a universal obstacle avoidance module for unmanned aerial vehicles, which is installed between the remote control signal receiver and the airborne flight control system of the unmanned aerial vehicle. Whether there are obstacles in the forward direction of the aircraft: If no obstacles are detected or the detected obstacles are far away and do not pose a threat to flight safety, the fusion obstacle avoidance processor will directly transmit the decoded remote control signal to the UAV flight control system At this time, the UAV is directly controlled by the ground station personnel or controlled by remote control. If the obstacle detected in the forward direction is relatively close and it is judged that it has posed a threat to flight safety, the fusion obstacle avoidance processor will automatically cut off the path between the remote control signal receiver and the flight control system, and the signal of the remote control signal receiver will be superimposed and fused The obstacle avoidance command signal of the obstacle avoidance processor is processed by the fusion obstacle avoidance processor and then output to the UAV flight control system. At the same time, the fusion obstacle avoidance processor will automatically make obstacle avoidance decisions based on the relative distance between the current UAV and the obstacle. Then issue the planned flight mode command to the flight control system to complete obstacle avoidance. After completing obstacle avoidance and judging that there is no risk of collision in the forward direction, the integrated obstacle avoidance processor returns the control right of the remote control, directly connects the remote control signal of the UAV with the control path between the flight control system, and the route is restored.

The algorithm in the onboard vision processor supports binocular ranging with variable baselines. By increasing or reducing the binocular baseline distance, the precise measurement range of binocular ranging can be adjusted. The camera performs offline visual calibration, and then calculates the internal and external system parameters and focal length of the binocular ranging sensor according to the imaging correction of the three-dimensional calibration target in the cross field of view, and can automatically set the required measurement range program according to the user. Look up the table to set the relevant initial matching parameter values, and finally use these parameters to perform real-time correction, matching, and three-dimensional reconstruction of the objects in the binocular ranging sensor, and then obtain the object size and distance in the cross field of view.

Its working steps are shown in Figure 2:

Step 1: During the route flight of the UAV, the binocular ranging sensor and the ultrasonic ranging sensor will detect the distance of the obstacle in front and the azimuth relative to the current aircraft heading in real time, and the fusion obstacle avoidance processor will correct the solution through the advanced filtering algorithm. Calculate the distance between the actual obstacle and the drone;

Step 2: The fusion obstacle avoidance processor will calculate the threat factor of the nearest obstacle in front of the current body, that is, whether the difference between the fused obstacle distance and the set minimum safety distance is greater than zero, if not, trigger the obstacle avoidance alarm message, and turn to Step 3, if satisfied, go to step 6;

Step 3: The integrated obstacle avoidance processor will modify the path between the remote control signal receiver and the airborne flight control system, and switch the response receiver mode signal by judging the position information of the obstacle, so as to obtain the control right of the UAV , and go to step 4;

Step 4: When the distance between the aircraft and the obstacle has reached a safe distance, the threat is triggered, and the integrated obstacle avoidance processor will plan the flight path in real time, and control the UAV to fly according to the planned path, and then turn to step 5;

Step 5: After flying around the obstacle, the integrated obstacle avoidance processor automatically switches the drone's self-driving route mode, and the drone automatically guides back to the previous flying route through the onboard route planning, and returns the control to the User, so far the obstacle avoidance alarm is released, and the obstacle avoidance is completed, go to step 6;

Step 6: The user continues to control the flight of the drone through the ground station or the remote control, and returns to step 1.

The invention measures in real time the relative distance between the global nearest obstacle and the UAV in the forward direction of flight through a binocular-ultrasonic ranging sensor, and transmits the data to a fusion obstacle avoidance processor for fusion processing of the data. Among them, an advanced filtering method is used to process the distance information to obtain more accurate obstacle threat information at the current moment. The fusion obstacle avoidance processor will judge whether the relative distance and orientation of the processed obstacles will pose a threat to the flight safety of the UAV. If it is judged that there is no threat to flight safety, the fusion obstacle avoidance processor will shield the obstacle avoidance superposition control amount, and directly send the signal sent by the remote control signal receiver to the flight control system; if the fusion obstacle avoidance processor finds that the obstacle ahead will If it poses a huge threat to the flight safety of the UAV, it will immediately superimpose the control amount after the distance calculation on the input of the remote control signal, switch to the brake mode through the control mode channel to keep hovering, and plan in real time according to the obstacle orientation information Fly around the direction, so as to obtain the control right of the UAV; until the obstacles are bypassed according to the pre-planned flight around the route and there is no threat of new obstacles, the integrated obstacle avoidance processor automatically shields the mode control amount of the distance conversion, and realizes the remote control signal The direct connection between the receiver and the flight control system returns the control of the drone to the user.

The above embodiments are only to illustrate the technical ideas of the present invention, and can not limit the protection scope of the present invention with this. All technical ideas proposed in accordance with the present invention, any changes made on the basis of technical solutions, all fall within the protection scope of the present invention. Inside.

Claims (7)

1. the Multi-sensor Fusion obstruction-avoiding control system of a kind of unmanned plane, the unmanned plane include onboard flight control system, distant Control signal receiver, it is characterised in that the system also includes ultrasonic distance-measuring sensor, binocular distance measuring sensor, airborne vision Processor, fusion avoidance processor；The ultrasonic distance-measuring sensor measures the range information of unmanned plane preceding object thing in real time And it is sent to fusion avoidance processor；Binocular distance measuring sensor obtains the image information of unmanned plane preceding object thing in real time and transmits To airborne vision processor, airborne vision processor is sent to fusion simultaneously according to the depth information that image information calculates barrier Avoidance processor；Fusion avoidance processor adjusts the distance information and depth information is filtered fusion, obtains unmanned plane away from front barrier Hinder the distance of thing, the threatening factors of barrier are calculated according to the distance, and according to threshold decision barrier set in advance whether Exist and threaten；When threatening factors are less than threshold value set in advance, disturbance in judgement thing merges avoidance processor by remote control without threat The ground remote control signal that signal receiver is received is conveyed directly to onboard flight control system, controls nothing by ground remote control signal Man-machine flight；When threatening factors are more than or equal to threshold value set in advance, fusion avoidance processor resolves avoidance mode passageway Signal, and the offline mode replaced with avoidance mode passageway signal in the ground remote control signal that remote control signal receiver is received is logical Road signal, carries out avoidance by avoidance mode passageway signal control unmanned plane.

2. the Multi-sensor Fusion obstruction-avoiding control system of unmanned plane according to claim 1, it is characterised in that the remote control letter Number receiver, ultrasonic distance-measuring sensor, airborne vision processor are connected with avoidance processor is merged by UART, and fusion keeps away Barrier processor is connected with onboard flight control system by UART.

3. the Multi-sensor Fusion obstruction-avoiding control system of unmanned plane according to claim 1, it is characterised in that the binocular is surveyed Away from sensor using USB2.0 bus transfer agreements variable baseline configuration, and initial baseline is away from for 63mm.

4. the Multi-sensor Fusion obstruction-avoiding control system of unmanned plane according to claim 1, it is characterised in that the fusion keeps away Chip of the barrier processor using STM32F407 models.

5. the Multi-sensor Fusion obstruction-avoiding control system of unmanned plane according to claim 1, it is characterised in that the ultrasound wave Distance measuring sensor adopts bitubular receiving type structure, model HC-SR04.

6. a kind of Multi-sensor Fusion avoidance obstacle method of unmanned plane, it is characterised in that comprise the steps：

Step 1, using the range information of ultrasonic wave real-time measurement unmanned plane preceding object thing；

Step 2, obtains the image information of unmanned plane preceding object thing in real time using binocular camera, and is calculated according to image information The depth information of unmanned plane preceding object thing；

Step 3, adjusts the distance information by filtering algorithm and depth information is filtered fusion, and resolve the threat of barrier because Son, according to threatening factors disturbance in judgement thing with the presence or absence of threatening, when barrier is without threatening, the ground that unmanned plane is received is distant Control signal is conveyed directly to onboard flight control system；When barrier has threat, avoidance mode passageway signal is resolved, and with keeping away Barrier mode passageway signal replaces the offline mode channel signal in the remote signal of ground, and control unmanned plane carries out avoidance action.

7. the Multi-sensor Fusion avoidance obstacle method of unmanned plane according to claim 6, it is characterised in that described in step 2 According to the detailed process of the depth information of image information calculating unmanned plane preceding object thing it is：

The binocular baseline distance of setting binocular distance measuring sensor, and offline vision calibration is carried out to binocular distance measuring sensor, according to double Image correction in mesh intersection visual field to three-dimensional scaling target, calculates the inner and outer system parameter and focal length of binocular distance measuring sensor, profit Image information corrected in real time with inner and outer system parameter and focal length, matched and three-dimensional reconstruction, so as to obtain in front of unmanned plane The depth information of barrier.