WO2021174466A1

WO2021174466A1 - Self-adaptive load balancing method and system, and storage medium

Info

Publication number: WO2021174466A1
Application number: PCT/CN2020/077845
Authority: WO
Inventors: 周游; 刘洁; 叶长春
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-03-04
Filing date: 2020-03-04
Publication date: 2021-09-10
Also published as: CN112753018B; CN112753018A

Abstract

A self-adaptive load balancing method and system, and a storage medium. The method comprises: obtaining the number of damaged chips (S101); determining, according to the number of damaged chips and an average number of data types borne by each chip, the remaining number of data types carried by the remaining chips (S102); selecting data of the remaining number of data types from data of N data types to obtain data of the selected data types (S103); and evenly running the data of the selected data types as data of an algorithm system on the remaining chips (S104).

Description

Method, system and storage medium for self-adaptive load balancing

Technical field

This application relates to the field of computer technology, and in particular to a method, system and storage medium for adaptive load balancing.

Background technique

Artificial intelligence related fields need to use algorithm systems in practical applications, which require a large amount of data, and the calculation and processing of these data are undertaken by the chip.

For example, in the application of a vision-based algorithm system in an unmanned aerial vehicle, the hovering performance of the unmanned aerial vehicle adopts the visual positioning algorithm system, and the obstacle avoidance performance of the unmanned aerial vehicle adopts the obstacle avoidance algorithm system. To ensure the hovering and obstacle avoidance performance of the UAV, it is necessary to ensure that it can process more image frames in real time. However, the single chip processing capacity of the UAV is limited. Therefore, when the vision sensor is connected to multiple directions, each algorithm system Image frames transmitted from the vision sensor in multiple directions are required, and at this time, multi-chip collaboration is required.

However, when the chip is damaged and cannot be replaced in time in the multi-chip collaboration, the remaining chip cannot handle the original image frames in multiple directions, which will seriously affect the relevant performance of the UAV.

Summary of the invention

Based on this, the present application provides a method, system and storage medium for adaptive load balancing.

In the first aspect, this application provides a method for adaptive load balancing, which is suitable for an algorithm system. The algorithm system runs on multiple chips, and the multiple chips carry N data types of the algorithm system. Calculation of the data, the method includes:

Get the number of bad chips;

Determining the remaining number of data types carried by the remaining chips except for the bad chip according to the number of the bad chips and the average number of the data types carried by each chip;

Select the remaining number of data types from the N data types to obtain the selected data type data, where N is a positive integer greater than or equal to two;

The data of the selected data type is run on the remaining chips in a balanced manner as the data of the algorithm system.

In a second aspect, the present application provides an adaptive load balancing system, which is suitable for an algorithm system, and the algorithm system runs on multiple chips, and the multiple chips carry the algorithm System for calculating data of N data types, the system includes: a memory and a processor;

The memory is used to store a computer program;

The processor is used to execute the computer program and when executing the computer program, implement the following steps:

Get the number of bad chips;

Selecting the remaining number of data types from the N data types to obtain data of the selected data type, where N is a positive integer greater than or equal to two;

In a third aspect, the present application provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor realizes the adaptive load as described above. Balanced approach.

The embodiments of the present application provide a method, system and storage medium for adaptive load balancing to obtain the number of bad chips; according to the number of bad chips and the average number of data types carried by each chip, determine the other than the bad chips The remaining number of data types carried by the remaining chip; select the remaining number of data types from the N data types to obtain the data of the selected data type; use the data of the selected data type as the data of the algorithm system to operate in a balanced manner On the remaining chips. Since the remaining number of data types carried by the remaining chips is determined according to the number of bad chips and the average number of data types carried by each chip, and the remaining number of data types is selected from the N data types, the selected data type will be As the data of the algorithm system, the data runs on the remaining chips in a balanced manner. In this way, the selected data types that can withstand the capabilities of the remaining chips can be run on the remaining chips in a balanced manner. The impact of the bad chip on the algorithm system is minimized, and the impact of the bad chip on the performance of the specific application of the algorithm system is minimized.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the application.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. Ordinary technicians can obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic flowchart of an embodiment of an adaptive load balancing method according to the present application;

Figure 2 is a schematic diagram of load balancing of the dual-chip unmanned aerial vehicle;

Fig. 3 is a schematic structural diagram of an embodiment of an adaptive load balancing system of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

The flowchart shown in the drawings is only an example, and does not necessarily include all contents and operations/steps, nor does it have to be executed in the described order. For example, some operations/steps can also be decomposed, combined or partially combined, so the actual execution order may be changed according to actual conditions.

Artificial intelligence related fields need to use algorithm systems in practical applications, which require a large amount of data, and the calculation and processing of these data are undertaken by the chip. The processing capacity of a single chip is limited. When a large amount of data of different data types needs to be processed in real time, multi-chip collaboration is required. However, when the chip is damaged and cannot be replaced in time in the multi-chip collaborative cooperation, the remaining chip is unable to process the original data of multiple different data types, which will seriously affect the relevant performance of the specific application of the algorithm system.

In the embodiment of this application, the number of bad chips can be obtained; according to the number of bad chips and the average number of data types carried by each chip, determine the remaining number of data types carried by the remaining chips except for the bad chips; from N Select the data of the remaining number of data types from the data of the data types to obtain the data of the selected data type; use the data of the selected data type as the data of the algorithm system to run on the remaining chips in a balanced manner. Since the remaining number of data types carried by the remaining chips is determined according to the number of bad chips and the average number of data types carried by each chip, and the remaining number of data types is selected from the N data types, the selected data type will be As the data of the algorithm system, the data runs on the remaining chips in a balanced manner. In this way, the selected data types that can withstand the capabilities of the remaining chips can be run on the remaining chips in a balanced manner. The impact of the bad chip on the algorithm system is minimized, and the impact of the bad chip on the performance of the specific application of the algorithm system is minimized.

Hereinafter, some embodiments of the present application will be described in detail with reference to the accompanying drawings. In the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

Referring to Figure 1, Figure 1 is a schematic flowchart of an embodiment of an adaptive load balancing method according to the present application. The method in this embodiment is applicable to an algorithm system that runs on multiple chips, and multiple chips carry the algorithm system N Calculation of data of each data type.

In the embodiment of the present application, the multiple chips may be more than two chips, for example, two chips, three chips, or more than four chips. N is a positive integer greater than or equal to two, that is, the data type can be more than two data types.

The data of N data types can be required by the algorithm system, and all the data currently available are allocated to N groups. For example, the data of N data types can be data from N different data sources, or it can be the same data source. Data divided into N groups (for example, the amount of data is too large for a single chip to process in real time). Among them, the N different data sources can be data detected by N same detection units (for example, image data of vision sensors in different directions), or data detected by N different detection units (for example, image data of vision sensors). Data, laser sensor data, ultrasonic sensor data, infrared sensor data, etc.), or part of the data detected by the same detection unit, and another part of the data detected by a different detection unit.

It should be noted that the algorithm systems on multiple chips can be one algorithm system, or two or more different algorithm systems. The data type of each algorithm system can be different or the same.

In one application, the algorithm system includes an algorithm system based on artificial intelligence. For example: artificial neural network algorithm system, deep learning algorithm system, dimensionality reduction algorithm system, integrated algorithm system, etc.

In another application, the algorithm system includes a vision-based algorithm system. For example: vision-based 3D reconstruction algorithm system, vision-based path planning algorithm system, vision-based classification algorithm system, vision-based clustering algorithm system, vision-based positioning algorithm, vision-based obstacle avoidance algorithm, etc.

Under normal circumstances (that is, when there is no bad chip), the algorithm system runs on multiple chips, and the multiple chips carry calculations of data of N data types in the algorithm system. Among them, the specific carrying situation of multiple chips (for example, n chips) can be: the calculation of the average number of data (N/n) carried by each chip, or the non-average number of data carried by each chip ( Calculation of data of data types that are greater than the average number or less than the average number); the specific carrying situation of multiple chips is determined according to the specific actual situation (for example, the data size of different data types, the processing capacity of each chip, etc.) , It is not limited here.

For example, there are two chips under normal circumstances, and the algorithm system has 6 data types of data. The specific carrying situation can be: each chip carries the calculation of the algorithm system's 3 data types, or one chip carries the algorithm system Calculation of data of 4 data types, another chip carries the calculation of data of 2 data types of the algorithm system, or one chip carries the calculation of data of 6 data types of the algorithm system, the other chip does not carry the calculation of data of the algorithm system under normal circumstances The calculation of the algorithm system data, and so on.

For another example, under normal circumstances, there are two chips, and two algorithm systems run on the two chips. Each algorithm system has 6 data types of data. The specific carrying situation can be: an algorithm system is that each chip carries the algorithm The system calculates the data of three data types; the other algorithm system is the calculation of the data of the six data types of the algorithm system on a chip, and the calculation of the data of the algorithm system is not carried by the chip under normal circumstances; and so on.

For another example, under normal circumstances, there are three chips, and the algorithm system has 6 data types of data. The specific carrying situation can be: each chip carries the calculation of 2 data types of the algorithm system, or the specific carrying situation can be: One chip carries the calculation of the four data types of the algorithm system, the other chip carries the calculation of the two data types of the algorithm system, the third chip does not carry the calculation of the algorithm system data, and so on.

For another example, there are three chips under normal circumstances, and two algorithm systems run on the two chips. One algorithm system has data of 6 data types, and the other algorithm system has data of 4 data types. The specific carrying situation can be: One algorithm system is the calculation of 2 data types of the algorithm system on each chip; the other algorithm system is the calculation of the 2 data types of data of the algorithm system on one chip, and the other chip carries 1 data of the algorithm system. The calculation of data of different types, the third chip carries the calculation of data of 1 data type of the algorithm system, and so on.

The method includes:

Step S101: Obtain the number of bad chips.

Step S102: According to the number of bad chips and the average number of data types carried by each chip, determine the remaining number of data types carried by the remaining chips except for the bad chips.

Step S103: Select the data of the remaining number of data types from the data of the N data types to obtain the data of the selected data type, where N is a positive integer greater than or equal to two.

Step S104: Use the data of the selected data type as the data of the algorithm system to run on the remaining chips in a balanced manner.

When a chip is damaged, the data that the remaining chips can carry will inevitably decrease. The embodiment of this application does not cut, single, one-sided, and simply remove the average amount of data that the bad chip can carry, but based on the data. The type of data to determine the reduction of data, so that the performance of the algorithm system can be maximized, and the relevant performance of the algorithm system in specific applications can be guaranteed as much as possible. That is, determine the average number of data types carried by each chip, and determine the number of data types carried by bad chips and the total number of data types according to the number of bad chips and the average number of data types carried by each chip. Subtract the number of data types carried by the bad chip from the number N to obtain the remaining number of data types carried by the remaining chip, and then select the remaining number of data types from the data of the N data types, and select The data of the data type runs on the remaining chips evenly as the data of the algorithm system. Among them, running on the remaining chips in a balanced manner may include the calculation of the average number of data types carried by each remaining chip (the remaining number of data types/the number of remaining chips), or the calculation of the data type carried by each chip. Calculation of the average number of data types (remaining number of data types/number of remaining chips); the specific carrying situation of the remaining chips is determined according to the actual situation (for example, the data size of different data types, the processing capacity of each chip) , Etc.), not limited here.

For example: under normal circumstances, there are two chips, the algorithm system has 6 data types of data, the specific carrying situation can be: each chip carries the calculation of 3 data types of the algorithm system; after one chip is damaged, the other The chip carries the calculation of data of 3 data types selected by the algorithm system from 6 data types.

Or, under normal circumstances, there are two chips, and the algorithm system has 6 data types of data. The specific carrying situation can be: one chip carries the calculation of the 6 data types of the algorithm system, and the other chip does not carry the data of the algorithm system under normal conditions. The calculation of the data of the algorithm system; after one chip is damaged, the other chip carries the calculation of the data of the 3 data types selected by the algorithm system from the 6 data types.

For another example, under normal circumstances, there are two chips, and two algorithm systems run on the two chips. Each algorithm system has 6 data types of data. The specific carrying situation can be: an algorithm system is that each chip carries the algorithm The system calculates the data of 3 data types; the other algorithm system is the calculation of the data of the 6 data types of the algorithm system on one chip, and the calculation of the data of the algorithm system is not carried by the chip under normal circumstances. After a chip is damaged, one algorithm system is left with one chip to carry the calculation of the data of the three data types selected by the algorithm system from the six data types; the other algorithm system is to leave one chip to carry the algorithm from the 6 data types. Calculation of data of 3 data types selected from 3 data types.

For another example, there are three chips under normal circumstances, and the algorithm system has 6 data types. The specific carrying situation can be: each chip carries the calculation of 2 data types of the algorithm system; after one chip is damaged, the other two A chip carries the calculation of the data of the 4 data types selected by the algorithm system from the 6 data types (one of the remaining chips carries the calculation of the data of the selected 2 data types, and the other remaining chip carries the selected other Calculation of data of 2 data types).

For another example, under normal circumstances, there are three chips, the algorithm system has 6 data types of data, and the specific carrying situation can be: one chip carries the calculation of the algorithm system's 4 data types, and the other chip carries the algorithm system 2 For the calculation of data of one data type, the third chip does not carry the calculation of the algorithm system data; after one chip is damaged, the other two chips carry the data of 4 data types selected by the algorithm system from the 6 data types Calculation (one of the remaining chips carries the calculation of the data of the selected 3 data types, and the other remaining chip carries the calculation of the data of the other selected data type).

The embodiment of the application obtains the number of bad chips; according to the number of bad chips and the average number of data types carried by each chip, the remaining number of data types carried by the remaining chips except for the bad chips is determined; from N data types Select the data of the remaining number of data types from the data to obtain the data of the selected data type; use the data of the selected data type as the data of the algorithm system to run on the remaining chips in a balanced manner. Since the remaining number of data types carried by the remaining chips is determined according to the number of bad chips and the average number of data types carried by each chip, and the remaining number of data types is selected from the N data types, the selected data type will be As the data of the algorithm system, the data runs on the remaining chips in a balanced manner. In this way, the selected data types that can withstand the capabilities of the remaining chips can be run on the remaining chips in a balanced manner. The impact of the bad chip on the algorithm system is minimized, and the impact of the bad chip on the performance of the specific application of the algorithm system is minimized.

The following specifically describes the details of the data of the remaining number of data types selected from the data of the N data types in step S103.

In an embodiment, selecting the remaining number of data types from the N data types in step S103 may include: based on an algorithm system, selecting the remaining number of data types from the N data types data.

Different algorithm systems have different uses or functions of the results obtained through calculation and processing, and have different requirements for the selection of data types. For example, one is the positioning algorithm, the other is the obstacle avoidance algorithm, image data in different directions; for the positioning algorithm, as long as there is a reference object, it is also possible to select two opposite directions (for example, select both up and down directions, or select both left and right directions. ); For the obstacle avoidance algorithm, if you cannot observe the six directions, it is best to select one direction for each pair of relative directions (for example, select one direction from the up and down direction, or select one direction from the left and right directions). In this way, the data of the selected data type can meet the requirements of the algorithm system as much as possible, and the performance of the algorithm system can be exerted as much as possible.

In another embodiment, selecting the remaining number of data types from the N data types in step S103 may also include: based on the specific application of the algorithm system, selecting the remaining data types from the N data types Number of data types of data.

In different specific applications of the algorithm system, the specific purpose or specific function of the results obtained through calculation and processing by the algorithm system will be different, and the requirements for the selection of data types are also different. For example, positioning algorithms applied to unmanned aerial vehicles and robots (robots with different service types may have different requirements) have different choices of image data in different directions. Unmanned aerial vehicles have restrictions on flying height. It belongs to near-ground flight, so the positioning algorithm must select the downward direction. The robot moves directly on the ground, so the positioning algorithm may not choose the downward direction, but the forward direction can be the direction that must be selected. In this way, the data of the selected data type can meet the requirements of the algorithm system as much as possible, and while the performance of the algorithm system is exerted as much as possible, the relevant performance of the specific application of the algorithm system can be ensured as much as possible.

In another embodiment, selecting the remaining number of data types from the N data types in step S103 may also include: dynamically selecting the remaining number of data types from the N data types. .

In this embodiment, after selecting the remaining number of data types from the N data types, the selected remaining number of data types will not change after being selected, but will change dynamically. , That is, dynamically select the remaining number of data types of data. Although the data of the remaining number of data types is not as comprehensive as the data of the N data types, by dynamically selecting the data of the remaining number of data types, the algorithm system can fully and flexibly apply the data of each data type, so that the algorithm system The performance is partially supplemented and partially compensated, which can prevent the performance of the algorithm system from tilting to a certain aspect, maximize the utilization of system resources, enhance the robustness and stability of the system, and ensure the specific application of the algorithm system as much as possible. Related performance.

For example: image data in 6 directions up, down, left, and right, you need to select image data in 3 directions, you can dynamically select from the image data in the 6 directions, instead of fixedly selecting 3 directions.

It should be noted that the specific dynamic selection can be determined according to the specific algorithm system and specific practical applications. The selection of the remaining number of data types from the N data types in step S103 can combine the above-mentioned multiple implementation manners. For example: based on the algorithm system, dynamically select the remaining number of data types from the N data types of data; or, based on the specific application of the algorithm system, dynamically select the remaining number of data types from the N data types of data Data, etc.

The following takes the vision-based algorithm system as an example to illustrate the specific details.

When the algorithm system includes a vision-based algorithm system, the data is basically image data, which can be image data in different directions, image data from different sources, or image data with different resolutions, and so on.

In an embodiment, the data of N data types may include image data of N directions. At this time, selecting the data of the remaining number of data types from the data of the N data types in step S103 may also include: selecting the image data of the remaining number of directions from the image data of the N directions. Image data in different directions has many uses for vision-based algorithm systems, such as positioning, obstacle avoidance, path planning, navigation, and so on. Among them, positioning and obstacle avoidance are the basic uses and the basis for other uses.

In one application, the algorithm system includes a visual positioning algorithm system. Vision-based positioning uses a vision sensor to obtain an image of an object, and then uses a vision positioning algorithm system to process and calculate the image data, and then obtain the object's pose (position and posture) information. The visual positioning algorithm system includes, but is not limited to, the visual odometry (VO) algorithm system, the visual inertial odometry (Visual Inertial Odometry, VIO) algorithm system, and so on.

If there are vision sensors in multiple directions, the VO algorithm system can calculate the results of each direction separately, select the reference keyframe according to the respective camera pose, and calculate the respective results, and finally Consider how to use it together. The VIO algorithm system uniformly estimates the pose of the aircraft. The advantage is the dimension of the state quantity, which is convenient for multi-channel expansion. Each additional direction of observation only needs to increase the external parameter state quantity of the camera and the inertial measurement unit (IMU), but Due to the large number of multi-directional feature points, the amount of calculation is relatively large.

When the algorithm system includes a visual positioning algorithm system, selecting image data in the remaining number directions from the image data in the N directions in step S103 may specifically include: selecting image data in the remaining number directions from the image data in the N directions , The remaining number direction is the best direction among the N directions, and the best direction refers to the direction most helpful for positioning among the N directions. Further, the best direction may change. For example, in a certain period of time, a certain direction is one of the best directions. In another period of time, this direction cannot be regarded as one of the best directions, so the remaining ones can also be dynamically selected. Number and best direction image data.

The purpose of the visual positioning algorithm system is positioning. Therefore, when image data in N directions cannot be used, image data in the remaining directions that are most helpful for positioning are selected as much as possible to improve positioning accuracy as much as possible.

In a practical application, the basis for selecting the "best direction" from multiple directions can be: the number of feature points successfully matched by the triangulation measurement method, N, divided by the average depth d0 of the feature points, that is, N/d0 is more The bigger, the better the direction. The N directions are arranged from large to small according to the value of N/d0, and the direction represented by the value of N/d0 of the remaining number in the front is selected, which is the best direction.

Specifically, the average depth of feature points is calculated in each direction, which can be divided into three situations:

(1) If the number of successful binocular stereo matching (Stereo Match, different from inter-frame matching Frame Match) on the latest Keyframe is greater than a certain threshold, such as 100 points, then use the depth calculated by binocular stereo matching to find the average value;

(2) If (1) is not satisfied, then the histogram statistics of the feature points successfully matched by the triangulation measurement method on the latest Keyframe will be used as the average depth;

(3) If the above two fail, then assign a larger value to the average depth, such as 500m.

Among them, the visual positioning algorithm system is applied to unmanned aerial vehicles, and the best direction includes the downward direction. The flying height of unmanned aerial vehicles is limited and belongs to close-to-ground flight. Generally, the closer the object is, the clearer the image, and the more feature points that can be matched, the more helpful it is for positioning. Objects in the downward direction are usually away from the unmanned aerial vehicle. It is relatively close, so when the visual positioning algorithm system is applied to an unmanned aerial vehicle, the best direction includes the downward direction. The visual positioning algorithm system is related to the hovering performance of the unmanned aerial vehicle.

For another example, if the visual positioning algorithm system is applied to a ground robot that directly moves on the ground, it is clear that the best direction includes the forward direction.

In another application, the algorithm system includes an obstacle avoidance (OA) algorithm system. For example: Semi-global matching (SGM) algorithm system, etc.

When the algorithm system includes an obstacle avoidance algorithm system, selecting image data in the remaining number directions from the image data in N directions in step S103 may also include: selecting image data in the remaining number directions from the image data in the N directions , Wherein the N directions include a pair of opposite directions, and the directions in the remaining number directions include one of the opposite directions. In the specific selection, the image data of one of the relative directions may be dynamically selected from the pair of relative directions. For example: select the image data in the left direction in a certain period of time, select the image data in the right direction in another period of time; select the image data in the downward direction in a certain period of time, and select the image data in the upper direction in another period of time; in a certain period of time Select the image data in the front direction, and select the image data in the back direction in another period of time; and so on.

The best way to avoid obstacles is the image data in all directions around. The surrounding directions are usually selected in pairs, that is, N directions include pairs of relative directions. When image data in N directions cannot be used, it is best to select the relative direction. In one direction, at least there are obstacles in the opposite direction for reference, so as to avoid blindness in the opposite direction.

The following takes the application of the VIO algorithm system and the obstacle avoidance algorithm system in the unmanned aerial vehicle as an example to illustrate the method of this application in detail.

Refer to Figure 2, which is a schematic diagram of load balancing of the dual-chip UAV. Under normal circumstances, the chip on the left is used as the master chip, connected to the front, down, and rear vision sensors, and is normally responsible for the calculation and processing of the image data of the obstacle avoidance algorithm system in these three directions; the chip on the right is used as the slave The chip is connected with vision sensors in the upper, left, and right directions, and is responsible for the calculation and processing of the image data of the obstacle avoidance algorithm system in these three directions. The image data of the UAV's VIO algorithm system in the above six directions runs on the master chip (or slave chip).

When one of the chips is damaged, the load balancing strategy of the VIO algorithm system and the obstacle avoidance algorithm system on the remaining chip can be:

For the VIO algorithm system, when a chip is detected to be damaged, the logic is switched urgently. While retaining the image data in the downward direction for the VIO algorithm, select two from the five directions of front/rear/left/right/up (down + Choose two from five). If the image data in the downward direction is not available, the image data in three directions is selected from the image data in the remaining five directions instead.

Here it is best to refer to the direction index G=N/d0. The larger the G, the better the direction.

First of all, the downward direction is generally reliable, so when the downward direction is available (the downward direction is available, there are enough feature points in the image in the downward direction, and the feature points of the front and back images can match the upper), the image data in the downward direction is always available. Participate in the calculation of the VIO algorithm system. If the downward direction is not available, you need to choose the image data of the 3 best directions from the remaining 5 directions.

When initializing, you need to calculate the G value of the direction once for the top, front, back, left, and right directions, and arrange the G values in order from high to low, such as: front 30, right 28, back 20, left 16 , Top 5, select the front and right directions with the most forward G value.

During the flight, the G value in the forward and backward directions can be calculated at a certain time interval. If there is no matching feature point in a certain direction or far away from the scene during the flight, the G value will gradually decrease. When the G value is lower than a certain value, for example, when the threshold Gth is 8, the calculation is obtained at this time The G value is: the first 7 and the right 20. Then the next calculation will not use the information in the front direction to update the state quantity, but can try to calculate the G value in the right direction and the back direction in turn (that is, in this process, only calculate the front, back, and left G values, and Do not participate in the calculation of the VIO algorithm system, do not update the state amount, the calculation amount is relatively small), and re-select the best direction as right and back according to the G value (right 20, back 17, left 10, front 7, and up 5). And so on. Only when the G value is lower than the threshold, the alternative directions are calculated, which can save computing resources. If the five directions of front, back, left, right, and up are unqualified, then only the G value will be calculated next time and will not participate in the calculation of the algorithm system.

For the obstacle avoidance algorithm system, when a chip is detected to be damaged, the logic is urgently switched, according to the flight direction (IMU+GPS+VIO combined navigation gives speed observation), according to the flight direction up/down+left/right+forward/ Then, the image data of three directions are selected to participate in the calculation of the obstacle avoidance algorithm system in the manner of selecting one direction from each pair of relative directions among the six directions.

Among them, in the process of making the depth map, the disparity of different resolutions can be merged to improve the accuracy of the depth map. Specifically: the high-precision depth map can be obtained efficiently according to the pair of low-resolution depth map and high-resolution stereo image with lower resolution; the disparity value can be obtained from the high-resolution image by optimizing the energy function; The difference value, in order to selectively replace the low-precision value of the depth map with a high-precision value; the energy function can be selectively optimized on the long-distance small parallax pixel to improve efficiency; according to the lower resolution of the input depth map , The energy function is sampled at horizontal and vertical intervals to further improve efficiency.

Refer to FIG. 3, which is a schematic structural diagram of an embodiment of an adaptive load balancing system of the present application. The adaptive load balancing system is suitable for an algorithm system that runs on multiple chips, and multiple chips carry the algorithm System calculation of data of N data types. It should be noted that the adaptive load balancing system can implement the steps in the above-mentioned adaptive load balancing method. For detailed descriptions of related content, please refer to the above method part, which will not be repeated here.

The adaptive load balancing system 100 includes: a memory 1 and a processor 2; the memory 1 and the processor 2 are connected by a bus.

Among them, the processor 2 may be a micro control unit, a central processing unit, or a digital signal processor, and so on.

Among them, the memory 1 can be a Flash chip, a read-only memory, a magnetic disk, an optical disk, a U disk or a mobile hard disk, etc.

The memory 1 is used to store a computer program; the processor 2 is used to execute the computer program and when the computer program is executed, the following steps are implemented:

Get the number of bad chips; according to the number of bad chips and the average number of data types carried by each chip, determine the remaining number of data types carried by the remaining chips except for the bad chips; select from the data of N data types The data of the remaining number of data types are selected to obtain the data of the selected data type, and N is a positive integer greater than or equal to two; the data of the selected data type is run on the remaining chips in a balanced manner as the data of the algorithm system.

Wherein, when the processor executes the computer program, it implements the following steps: based on the algorithm system, select the remaining number of data types from the N data types.

Wherein, when the processor executes the computer program, it implements the following steps: based on the specific application of the algorithm system, select the remaining number of data types from the N data types.

Among them, when the processor executes the computer program, the following steps are implemented: dynamically selecting the remaining number of data types from the N data types.

Among them, the algorithm system includes an algorithm system based on artificial intelligence.

Among them, the algorithm system includes a vision-based algorithm system; N data types include image data in N directions.

Wherein, when the processor executes the computer program, it implements the following steps: select the image data of the remaining number of directions from the image data of the N directions.

Among them, the algorithm system includes a visual positioning algorithm system.

Among them, when the processor executes the computer program, it implements the following steps: select the image data of the remaining number directions from the image data of the N directions, the remaining number direction is the best direction among the N directions, and the best direction refers to The direction most helpful for positioning among the N directions.

Among them, the visual positioning algorithm system is applied to unmanned aerial vehicles, and the best direction includes the downward direction.

Among them, the visual positioning algorithm system includes the visual inertial mileage calculation system.

Among them, the algorithm system includes obstacle avoidance algorithm system.

Wherein, when the processor executes the computer program, it implements the following steps: select image data in the remaining number directions from the image data in N directions, where the N directions include pairs of relative directions, and the directions in the remaining number directions Including one of the opposite directions.

The present application also provides a computer-readable storage medium that stores a computer program, and when the computer program is executed by a processor, the processor realizes the method for adaptive load balancing as described above. For a detailed description of the relevant content, please refer to the content section of the above adaptive load balancing method, which will not be repeated here.

Wherein, the computer-readable storage medium may be an internal storage unit of the aforementioned adaptive load balancing system, such as a hard disk or a memory. The computer-readable storage medium may also be an external storage device of the above-mentioned adaptive load balancing system, such as an equipped plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, and so on.

It should be understood that the terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit the application.

It should also be understood that the term "and/or" used in the specification and appended claims of this application refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

The above are only specific embodiments of this application, but the scope of protection of this application is not limited to this. Any person skilled in the art can easily think of various equivalents within the technical scope disclosed in this application. Modifications or replacements, these modifications or replacements shall be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

An adaptive load balancing method, characterized in that the method is suitable for an algorithm system, the algorithm system runs on multiple chips, and the multiple chips carry calculations of data of N data types in the algorithm system , The method includes:

Get the number of bad chips;

Determining the remaining number of data types carried by the remaining chips except the bad chip according to the number of the bad chips and the average number of data types carried by each chip;

Selecting the remaining number of data types from the N data types to obtain data of the selected data type, where N is a positive integer greater than or equal to two;

The data of the selected data type is run on the remaining chips in a balanced manner as the data of the algorithm system.
The method according to claim 1, wherein the selecting the remaining number of data types from the N data types includes:

Based on the algorithm system, data of the remaining number of data types are selected from the data of the N data types.
The method according to claim 1, wherein the selecting the remaining number of data types from the N data types includes:

Based on the specific application of the algorithm system, data of the remaining number of data types are selected from the data of the N data types.
The method according to claim 1, wherein the selecting the remaining number of data types from the N data types includes:

The remaining number of data types are dynamically selected from the N data types of data.
The method according to claim 1, wherein the algorithm system comprises an algorithm system based on artificial intelligence.
The method according to claim 1, wherein the algorithm system comprises a vision-based algorithm system; the data of the N data types comprises image data of N directions.
The method according to claim 6, wherein the selecting the remaining number of data types of data from the N data types of data comprises:

Select the image data of the remaining number of directions from the image data of the N directions.
The method according to claim 7, wherein the algorithm system comprises a visual positioning algorithm system.
The method according to claim 8, wherein the selecting image data in the remaining number directions from the image data in N directions comprises:

Select the image data of the remaining number directions from the image data of the N directions, the remaining number direction is the best direction among the N directions, and the best direction refers to the N directions The direction most helpful for positioning.
The method according to claim 9, wherein the visual positioning algorithm system is applied to an unmanned aerial vehicle, and the best direction includes a downward direction.
The method according to claim 8, wherein the visual positioning algorithm system comprises a visual inertial mileage calculation system.
The method according to claim 7, wherein the algorithm system comprises an obstacle avoidance algorithm system.
The method according to claim 12, wherein the selecting the image data of the remaining number of directions from the image data of the N directions comprises:

The image data of the remaining number directions are selected from the image data of N directions, wherein the N directions include a pair of relative directions, and the direction in the remaining number directions includes one of the relative directions.
An adaptive load balancing system, characterized in that the adaptive load balancing system is suitable for an algorithm system, the algorithm system runs on multiple chips, and the multiple chips carry N data of the algorithm system For calculation of various types of data, the system includes: a memory and a processor;

The memory is used to store a computer program;

The processor is used to execute the computer program and when executing the computer program, implement the following steps:

Get the number of bad chips;

Determining the remaining number of data types carried by the remaining chips except the bad chip according to the number of the bad chips and the average number of data types carried by each chip;

Select the remaining number of data types from the N data types to obtain the selected data type data, where N is a positive integer greater than or equal to two;

The data of the selected data type is run on the remaining chips in a balanced manner as the data of the algorithm system.
The system according to claim 14, wherein the processor implements the following steps when executing the computer program:

Based on the algorithm system, data of the remaining number of data types are selected from the data of the N data types.
The system according to claim 14, wherein the processor implements the following steps when executing the computer program:

Based on the specific application of the algorithm system, data of the remaining number of data types are selected from the data of the N data types.
The system according to claim 14, wherein the processor implements the following steps when executing the computer program:

The remaining number of data types are dynamically selected from the N data types of data.
The system according to claim 14, wherein the algorithm system comprises an algorithm system based on artificial intelligence.
The system according to claim 14, wherein the algorithm system comprises a vision-based algorithm system; the data of the N data types comprises image data of N directions.
The system according to claim 19, wherein the processor implements the following steps when executing the computer program:

Select the image data of the remaining number of directions from the image data of the N directions.
The system according to claim 20, wherein the algorithm system comprises a visual positioning algorithm system.
The system according to claim 21, wherein the processor implements the following steps when executing the computer program:

Select the image data of the remaining number directions from the image data of the N directions, the remaining number direction is the best direction among the N directions, and the best direction refers to the N directions The direction most helpful for positioning.
The system according to claim 22, wherein the visual positioning algorithm system is applied to an unmanned aerial vehicle, and the best direction includes a downward direction.
The system according to claim 21, wherein the visual positioning algorithm system comprises a visual inertial mileage calculation system.
The system according to claim 20, wherein the algorithm system comprises an obstacle avoidance algorithm system.
The system according to claim 25, wherein the processor implements the following steps when executing the computer program:

The image data of the remaining number directions are selected from the image data of N directions, wherein the N directions include a pair of relative directions, and the direction in the remaining number directions includes one of the relative directions.
A computer-readable storage medium, characterized in that, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor realizes any one of claims 1-13. The method of adaptive load balancing.