CN116028234B - Distributed database load balancing method, device, equipment and storage medium - Google Patents
Distributed database load balancing method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN116028234B CN116028234B CN202310333844.7A CN202310333844A CN116028234B CN 116028234 B CN116028234 B CN 116028234B CN 202310333844 A CN202310333844 A CN 202310333844A CN 116028234 B CN116028234 B CN 116028234B
- Authority
- CN
- China
- Prior art keywords
- node
- database
- data
- target
- fpga
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a distributed database load balancing method, a device, equipment and a storage medium, relates to the field of FPGA application, and is applied to an FPGA on a preset load balancing board card of a target database node, wherein the target database node is a management node in a distributed database system; the method comprises the following steps: sending a monitoring request to a CPU of the target database node based on a preset regular monitoring time point so that the CPU monitors the average daily access number of the target database node based on the monitoring request; summarizing the average daily access numbers of other database nodes in the system, and determining a target balance plan based on the average daily access numbers of all database nodes in the system; and distributing the target balance plan so that the FPGA of each database node can execute corresponding data migration operation based on the target balance plan. According to the method and the device, the FPGA is used for carrying out periodic monitoring so as to determine and complete corresponding equalization operation, and the overall performance of the database is effectively improved.
Description
Technical Field
The present invention relates to the field of FPGA applications, and in particular, to a method, apparatus, device, and storage medium for load balancing of a distributed database.
Background
Currently, in some application systems with large data volume, massive data needs to be distributed in a plurality of databases according to a certain algorithm. As time goes on and data increases, the data distribution is not balanced, so that some databases are loaded with large pressure, some databases are loaded with small pressure, the excessive pressure can greatly affect the resources such as a CPU (central processing unit) of a database server, a memory and the like and the network bandwidth, and the server resources cannot be fully utilized due to the too small pressure, so that the resource waste is caused.
Disclosure of Invention
Accordingly, the present invention aims to provide a distributed database load balancing method, device, equipment and storage medium, which can realize database load balancing through FPGA, thereby effectively improving overall performance of database. The specific scheme is as follows:
in a first aspect, the present application provides a distributed database load balancing method, which is applied to an FPGA on a preset load balancing board of a target database node, where the target database node is a management node in a distributed database system; wherein the method comprises the following steps:
based on a preset regular monitoring time point, sending a corresponding monitoring request to a CPU of the target database node through a preset interface, so that the CPU monitors the average daily access number of the target database node based on the monitoring request;
summarizing the average daily access numbers of other database nodes in the current distributed database system, and determining a corresponding target balance plan based on a preset balance strategy and the average daily access numbers corresponding to the database nodes in the distributed database system;
and distributing the target balance plan in the distributed database system so that the FPGA of each database node executes corresponding data migration operation based on the data to be migrated corresponding to the node in the target balance plan and corresponding migration paths and/or data to be migrated.
Optionally, the CPU monitors the average daily access number of the target database node based on the monitoring request, including:
and after receiving the monitoring request, the CPU counts the user access amount of the target database node in a preset time period to obtain the average daily access number corresponding to the target database node.
Optionally, the summarizing the average daily access number of other database nodes in the distributed database system includes:
and obtaining the average daily access number of other database nodes in the current distributed database system by communicating with the FPGA on the preset load balancing board card of the other database nodes in the current distributed database system.
Optionally, the distributed database load balancing method further includes:
when the FPGA of each database node in the distributed database system completes corresponding data migration operation, determining that all data migration operations corresponding to the target balance plan are completed, and controlling the FPGA of each database node in the distributed database system to enter a dormant state.
Optionally, the FPGA of each database node performs a corresponding data migration operation based on the data to be migrated corresponding to the node of the FPGA and the corresponding migration path and/or the data to be migrated in the target balancing plan, including:
the FPGA of each database node acquires data to be migrated corresponding to the node of the FPGA based on the target balance plan, and sends the data to be migrated according to an migration path corresponding to the data to be migrated in the target balance plan so as to complete corresponding data migration operation;
and/or, the FPGA of each database node determines corresponding data to be migrated based on the target balance plan, and migrates the data to be migrated into the DDR4 on the preset load balance board card corresponding to the self node so as to complete corresponding data migration operation.
Optionally, the FPGA of each database node obtains data to be migrated corresponding to the own node by using a target balancing plan, including:
the FPGA of each database node reads corresponding data to be migrated from the memory of the CPU of the own node based on the target balance plan, and stores the data to be migrated into the DDR4 of the own node according to a preset communication strategy so as to finish the sending operation of the data to be migrated stored in the DDR4 based on the corresponding migration path.
Optionally, after determining, by the FPGA of each database node, the corresponding data to be migrated based on the target balancing plan, and migrating the data to be migrated into the DDR4 on the preset load balancing board card corresponding to the node of the FPGA, so as to complete the corresponding data migration operation, the method further includes:
the FPGA of each database node stores the data to be migrated in the DDR4 of the self node into the memory of the CPU of the self node based on the preset communication strategy.
In a second aspect, the present application provides a distributed database load balancing device, which is applied to an FPGA on a preset load balancing board of a target database node, where the target database node is a management node in a distributed database system; wherein the device comprises:
the request sending module is used for sending a corresponding monitoring request to the CPU of the target database node based on a preset regular monitoring time point through a preset interface so that the CPU monitors the average daily access number of the target database node based on the monitoring request;
the plan determining module is used for summarizing the average daily access numbers of other database nodes in the distributed database system at present and determining a corresponding target balance plan based on a preset balance strategy and the average daily access numbers corresponding to the database nodes in the distributed database system;
and the data migration module is used for distributing the target balance plan in the distributed database system so that the FPGA of each database node executes corresponding data migration operation based on the data to be migrated corresponding to the self node in the target balance plan and the corresponding migration path and/or the data to be migrated.
In a third aspect, the present application provides an electronic device, including:
a memory for storing a computer program;
and the processor is used for executing the computer program to realize the steps of the distributed database load balancing method.
In a fourth aspect, the present application provides a computer readable storage medium storing a computer program which, when executed by a processor, implements the steps of the distributed database load balancing method described above.
In the application, a corresponding monitoring request is sent to a CPU of the target database node based on a preset regular monitoring time point through a preset interface, so that the CPU monitors the average daily access number of the target database node based on the monitoring request; summarizing the average daily access numbers of other database nodes in the current distributed database system, and determining a corresponding target balance plan based on a preset balance strategy and the average daily access numbers corresponding to the database nodes in the distributed database system; and distributing the target balance plan in the distributed database system so that the FPGA of each database node executes corresponding data migration operation based on the data to be migrated corresponding to the node in the target balance plan and corresponding migration paths and/or data to be migrated. The average daily access number of the target database nodes is monitored regularly through the FPGA, and the average daily access number of each database node in the current distributed database system is comprehensively distributed, so that corresponding balanced operation is determined and executed by the FPGA. Therefore, the whole database balancing process is completed by the FPGA, memory resources of a database CPU are not occupied, the load capacity of each database node in the current distributed database system can be effectively balanced, the maximum utilization of server resources of each database node is realized, and the overall performance of the database is effectively improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a distributed database load balancing method provided in the present application;
fig. 2 is a schematic diagram of a load balancing board card structure of a distributed database node provided in the present application;
FIG. 3 is a block diagram of the internal logic structure of an FPGA provided in the present application;
fig. 4 is a schematic flow chart of a load balancing method provided in the present application;
fig. 5 is a schematic structural diagram of a distributed database load balancing device provided in the present application;
fig. 6 is a block diagram of an electronic device provided in the present application.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Currently, in some application systems with large data volume, massive data needs to be distributed in a plurality of databases according to a certain algorithm. As time goes on and data increases, some databases are loaded with large pressure, some databases are loaded with small pressure, and too much pressure can greatly affect resources such as a CPU, a memory and the like of a database server and network bandwidth, and too little pressure can not fully utilize server resources, so that resource waste is caused. Therefore, the distributed database load balancing scheme can realize the database load balancing through the FPGA, so that the overall performance of the database can be effectively improved.
Referring to fig. 1, the embodiment of the invention discloses a distributed database load balancing method, which is applied to an FPGA on a preset load balancing board of a target database node, wherein the target database node is a management node in a distributed database system; wherein the method comprises the following steps:
and step S11, based on a preset regular monitoring time point, sending a corresponding monitoring request to the CPU of the target database node through a preset interface so that the CPU monitors the average daily access number of the target database node based on the monitoring request.
In this embodiment, the monitoring, by the CPU, the average daily access number of the target database node based on the monitoring request may specifically include: and after receiving the monitoring request, the CPU counts the user access amount of the target database node in a preset time period to obtain the average daily access number corresponding to the target database node. The preset periodic detection time point can be set independently by a user based on actual requirements, and can also be set automatically by a system. The preset interface may be a PCIE (peripheral component interconnect express, a high-speed serial computer expansion bus standard) interface. Further, the preset time period may be set independently by the user based on the actual requirement, or may default to a time period between two adjacent monitoring.
And step S12, summarizing the average daily access numbers of other database nodes in the distributed database system, and determining a corresponding target balance plan based on a preset balance strategy and the average daily access number corresponding to each database node in the distributed database system.
In this embodiment, it should be understood that, referring to fig. 3, each database node in the distributed database system currently has a preset load balancing card corresponding to the database node, where the preset load balancing card is used as a physical implementation basis, and the preset load balancing card includes an FPGA (Field Programmable Gate Array ), a storage device, an input/output interface, a peripheral electronic device, and a PCB (Printed Circuit Board ) board. The storage device includes, but is not limited to, flash and DDR4 (Dual Date Rate-4, a new generation of memory specification), where the Flash is used to store FPGA logic program to implement power-on solidification of the FPGA, and the DDR4 may be used to store equalization plan and migration data. The input/output interface includes, but is not limited to, a network interface, a PCIE interface, and a JTAG (Joint Test Action Group, joint test engineering group) interface, where the PCIE interface is responsible for high-speed communication between the FPGA and the database, and the JTAG interface is responsible for burning and debugging the FPGA logic program. The peripheral electronic device comprises a power chip, a clock chip and necessary resistance and capacitance, and the PCB is responsible for providing carriers for the FPGA, the storage equipment, the input and output interface and the peripheral electronic device.
Further, referring to fig. 2, the FPGA logic includes a PCIE module (pcie_inst), a LOAD monitoring module (load_moni), a LOAD balancing module (load_bala), a DATA migration module (data_tran), a DATA transmission module (data_tx), and a DATA reception module (data_rx). The PCIE module is used for realizing high-speed communication with a database CPU, realizing data transmission between a CPU memory and the DDR4 through an XDMA, the load monitoring module is used for initiating the monitoring request to the CPU and receiving monitoring data sent by the CPU, and the load balancing module is used for comprehensively planning load information of all database nodes and balancing a plan according to a formulated target. The data migration module is used for executing corresponding data migration according to the target balance plan; the data sending module is used for sending load information of the database node of the data sending module or reading data in the DDR4 to carry out data migration; the data receiving module is used for receiving load information of other database nodes or receiving migration data sent by other nodes and writing the migration data into the DDR4.
Specifically, the summarizing the average daily access number of other database nodes in the distributed database system may specifically include: and obtaining the average daily access number of other database nodes in the current distributed database system by communicating with the FPGA on the preset load balancing board card of the other database nodes in the current distributed database system.
And step S13, distributing the target balance plan in the distributed database system so that the FPGA of each database node executes corresponding data migration operation based on the data to be migrated corresponding to the self node in the target balance plan and the corresponding migration path and/or data to be migrated.
In this embodiment, when the FPGAs of each database node in the distributed database system complete the corresponding data migration operation, it is determined that all the data migration operations corresponding to the target balancing plan are completed, and the FPGAs of each database node in the distributed database system are controlled to enter a dormant state. Specifically, the performing, by the FPGA of each database node, a corresponding data migration operation based on the data to be migrated corresponding to the node of the FPGA of the target balancing plan and the corresponding migration path and/or the data to be migrated may specifically include: the FPGA of each database node acquires data to be migrated corresponding to the node of the FPGA based on the target balance plan, and sends the data to be migrated according to an migration path corresponding to the data to be migrated in the target balance plan so as to complete corresponding data migration operation; and/or, the FPGA of each database node determines corresponding data to be migrated based on the target balance plan, and migrates the data to be migrated into the DDR4 on the preset load balance board card corresponding to the self node so as to complete corresponding data migration operation.
Further, in this embodiment, the obtaining, by the FPGA of each database node, data to be migrated corresponding to the own node based on the target balancing plan may specifically include: the FPGA of each database node reads corresponding data to be migrated from the memory of the CPU of the own node based on the target balance plan, and stores the data to be migrated into the DDR4 of the own node according to a preset communication strategy so as to finish the sending operation of the data to be migrated stored in the DDR4 based on the corresponding migration path. And, the FPGA of each database node determines corresponding data to be migrated based on the target balancing plan, and migrates the data to be migrated into the DDR4 on the preset load balancing board card corresponding to the node thereof, so as to complete corresponding data migration operation, which specifically may further include: the FPGA of each database node stores the data to be migrated in the DDR4 of the self node into the memory of the CPU of the self node based on the preset communication strategy. The preset communication policy may specifically be a communication policy for performing data transmission through PCIE XDMA.
Therefore, in the embodiment of the application, based on a preset regular monitoring time point, a corresponding monitoring request is sent to the CPU of the target database node through a preset interface, so that the CPU monitors the average daily access number of the target database node based on the monitoring request; summarizing the average daily access numbers of other database nodes in the current distributed database system, and determining a corresponding target balance plan based on a preset balance strategy and the average daily access numbers corresponding to the database nodes in the distributed database system; and distributing the target balance plan in the distributed database system so that the FPGA of each database node executes corresponding data migration operation based on the data to be migrated corresponding to the node in the target balance plan and corresponding migration paths and/or data to be migrated. The average daily access number of the target database nodes is monitored regularly through the FPGA, and the average daily access number of each database node in the current distributed database system is comprehensively distributed, so that corresponding balanced operation is determined and executed by the FPGA. Therefore, the whole database balancing process is completed by the FPGA, memory resources of a database CPU are not occupied, the load capacity of each database node in the current distributed database system can be effectively balanced, the maximum utilization of server resources of each database node is realized, and the overall performance of the database is effectively improved.
The following describes a specific implementation procedure of a distributed database load balancing method disclosed in the present application with reference to fig. 4.
Referring to the balancing method flowchart shown in fig. 4, the FPGA on the preset load balancing board of the target database node periodically sends a monitoring request (fig. 1) to the target database node CPU through the preset PCIE interface; after receiving the monitoring request, the CPU counts the number of users accessing the target database node within a certain time (the time can be defined by a user or can be defaulted to be a regular monitoring time interval) to calculate a corresponding average daily access number, and then the CPU sends the average daily access number to the FPGA (fig. 2) through the preset PCIE interface, and the FPGA records the average daily access number corresponding to the target database node and communicates with FPGAs of other database nodes in the current distributed database network to collect the average daily access numbers of all database nodes.
And then, the FPGA establishes a target balance plan according to the average daily access number of each database node and a balance strategy preset by a user, wherein the target balance plan comprises user data needing to be migrated and/or migrated in each database node and a corresponding migration path. The FPGA of the database node that needs to migrate data reads user data that needs to migrate from the memory of the CPU of the database itself, that is, the server main memory of the database itself, into the DDR4 on board itself (shown in fig. 3) through PCIE XDMA according to the target balancing plan, and then sends the user data that needs to migrate to the corresponding database node FPGA according to the corresponding migration path (shown in fig. 4 and 5). Meanwhile, the FPGA of the database node needing to migrate data receives the user data needing to migrate according to the target balancing plan, stores the user data in DDR4 (shown in (6) and (7)) on the self-board, and then writes the user data in the DDR4 into the memory of the self-database CPU (shown in (8)) through the PCIE XDMA. And repeatedly executing corresponding data migration and/or migration by all the database nodes FPGA until all the data migration operations are completed. Finally, when all data migration operations corresponding to the target balancing plan are completed, the FPGA on the preset load balancing board card of the target database node controls the FPGA of each database node in the distributed data system to enter a dormant state currently, and waits for the next load balancing.
Referring to fig. 5, the embodiment of the application also correspondingly discloses a distributed database load balancing device, which is applied to an FPGA on a preset load balancing board of a target database node, wherein the target database node is a management node in a distributed database system; wherein the device comprises:
a request sending module 11, configured to send a corresponding monitoring request to a CPU of the target database node through a preset interface based on a preset periodic monitoring time point, so that the CPU monitors an average daily access number of the target database node based on the monitoring request;
the plan determining module 12 is configured to aggregate the average daily access numbers of other database nodes in the distributed database system, and determine a corresponding target balance plan based on a preset balance policy and the average daily access number corresponding to each database node in the distributed database system;
and the data migration module 13 is configured to distribute the target balancing plan in the distributed database system, so that the FPGA of each database node performs a corresponding data migration operation based on the data to be migrated corresponding to the node of the target balancing plan and the corresponding migration path and/or the data to be migrated.
The more specific working process of each module may refer to the corresponding content disclosed in the foregoing embodiment, and will not be described herein.
Therefore, in the application, based on a preset regular monitoring time point, a corresponding monitoring request is sent to the CPU of the target database node through a preset interface, so that the CPU monitors the average daily access number of the target database node based on the monitoring request; summarizing the average daily access numbers of other database nodes in the current distributed database system, and determining a corresponding target balance plan based on a preset balance strategy and the average daily access numbers corresponding to the database nodes in the distributed database system; and distributing the target balance plan in the distributed database system so that the FPGA of each database node executes corresponding data migration operation based on the data to be migrated corresponding to the node in the target balance plan and corresponding migration paths and/or data to be migrated. The average daily access number of the target database nodes is monitored regularly through the FPGA, and the average daily access number of each database node in the current distributed database system is comprehensively distributed, so that corresponding balanced operation is determined and executed by the FPGA. Therefore, the whole database balancing process is completed by the FPGA, memory resources of a database CPU are not occupied, the load capacity of each database node in the current distributed database system can be effectively balanced, the maximum utilization of server resources of each database node is realized, and the overall performance of the database is effectively improved.
In some specific embodiments, the request sending module 11 may specifically include:
and the first average daily access number acquisition unit is used for counting the user access amount of the target database node in a preset time period after the CPU receives the monitoring request so as to obtain the average daily access number corresponding to the target database node.
In some embodiments, the plan determining module 12 may specifically include:
the second average daily access number obtaining unit is used for obtaining the average daily access number of other database nodes in the distributed database system through communication with the FPGA on the preset load balancing board card of the other database nodes in the distributed database system.
In some specific embodiments, the distributed database load balancing apparatus may specifically further include:
and the FPGA dormancy unit is used for determining that all data migration operations corresponding to the target balance plan are finished when the FPGA of each database node in the distributed database system finishes corresponding data migration operations, and controlling the FPGA of each database node in the distributed database system to enter a dormancy state.
In some specific embodiments, the data migration module 13 may specifically include:
the data migration unit is used for the FPGA of each database node to acquire data to be migrated corresponding to the node of the FPGA based on the target balance plan, and sends the data to be migrated according to an migration path corresponding to the data to be migrated in the target balance plan so as to complete corresponding data migration operation;
and the data migration unit is used for and/or the FPGA of each database node determines corresponding data to be migrated based on the target balance plan, and migrates the data to be migrated into the DDR4 on the preset load balance board card corresponding to the self node so as to complete corresponding data migration operation.
In some specific embodiments, the distributed database load balancing apparatus may specifically include:
the FPGA of each database node reads corresponding data to be migrated from the memory of the CPU of the own node based on the target balance plan, and stores the data to be migrated into the DDR4 of the own node according to a preset communication strategy so as to finish the sending operation of the data to be migrated stored in the DDR4 based on the corresponding migration path.
In some specific embodiments, the distributed database load balancing apparatus may specifically further include:
and the data storage unit to be migrated is used for storing the data to be migrated in the DDR4 of the self node into the memory of the CPU of the self node based on the preset communication strategy by the FPGA of each database node.
Further, the embodiment of the present application further discloses an electronic device, and fig. 6 is a structural diagram of the electronic device 20 according to an exemplary embodiment, where the content of the drawing is not to be considered as any limitation on the scope of use of the present application.
Fig. 6 is a schematic structural diagram of an electronic device 20 according to an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input output interface 25, and a communication bus 26. Wherein the memory 22 is configured to store a computer program that is loaded and executed by the processor 21 to implement the relevant steps of the distributed database load balancing method disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in the present embodiment may be specifically an electronic computer.
In this embodiment, the power supply 23 is configured to provide an operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and an external device, and the communication protocol in which the communication interface is in compliance is any communication protocol applicable to the technical solution of the present application, which is not specifically limited herein; the input/output interface 25 is used for acquiring external input data or outputting external output data, and the specific interface type thereof may be selected according to the specific application requirement, which is not limited herein.
The memory 22 may be a carrier for storing resources, such as a read-only memory, a random access memory, a magnetic disk, or an optical disk, and the resources stored thereon may include an operating system 221, a computer program 222, and the like, and the storage may be temporary storage or permanent storage.
The operating system 221 is used for managing and controlling various hardware devices on the electronic device 20 and computer programs 222, which may be Windows Server, netware, unix, linux, etc. The computer program 222 may further comprise a computer program capable of performing other specific tasks in addition to the computer program capable of performing the distributed database load balancing method performed by the electronic device 20 as disclosed in any of the previous embodiments.
Further, the application also discloses a computer readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the distributed database load balancing method disclosed previously. For specific steps of the method, reference may be made to the corresponding contents disclosed in the foregoing embodiments, and no further description is given here.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, so that the same or similar parts between the embodiments are referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing has outlined the detailed description of the preferred embodiment of the present application, and the detailed description of the principles and embodiments of the present application has been provided herein by way of example only to facilitate the understanding of the method and core concepts of the present application; meanwhile, as those skilled in the art will have modifications in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.
Claims (9)
1. The distributed database load balancing method is characterized by being applied to an FPGA (field programmable gate array) on a preset load balancing board of a target database node, wherein the target database node is a management node in a distributed database system; wherein the method comprises the following steps:
based on a preset regular monitoring time point, sending a corresponding monitoring request to a CPU of the target database node through a preset interface, so that the CPU monitors the average daily access number of the target database node based on the monitoring request;
summarizing the average daily access numbers of other database nodes in the current distributed database system, and determining a corresponding target balance plan based on a preset balance strategy and the average daily access numbers corresponding to the database nodes in the distributed database system;
distributing the target balance plan in the distributed database system so that the FPGA of each database node executes corresponding data migration operation based on data to be migrated corresponding to the node of the target balance plan and corresponding migration paths and/or data to be migrated;
the FPGA of each database node executes a corresponding data migration operation based on the data to be migrated corresponding to the node of the FPGA and a corresponding migration path and/or data to be migrated in the target balancing plan, including:
the FPGA of each database node acquires data to be migrated corresponding to the node of the FPGA based on the target balance plan, and sends the data to be migrated according to an migration path corresponding to the data to be migrated in the target balance plan so as to complete corresponding data migration operation;
and/or, the FPGA of each database node determines corresponding data to be migrated based on the target balance plan, and migrates the data to be migrated into the DDR4 on the preset load balance board card corresponding to the self node so as to complete corresponding data migration operation.
2. The distributed database load balancing method according to claim 1, wherein the CPU monitors an average daily access number of the target database node based on the monitoring request, comprising:
and after receiving the monitoring request, the CPU counts the user access amount of the target database node in a preset time period to obtain the average daily access number corresponding to the target database node.
3. The distributed database load balancing method according to claim 1, wherein the summarizing the average daily access number of other database nodes in the current distributed database system comprises:
and obtaining the average daily access number of other database nodes in the current distributed database system by communicating with the FPGA on the preset load balancing board card of the other database nodes in the current distributed database system.
4. The distributed database load balancing method of claim 1, further comprising:
when the FPGA of each database node in the distributed database system completes corresponding data migration operation, determining that all data migration operations corresponding to the target balance plan are completed, and controlling the FPGA of each database node in the distributed database system to enter a dormant state.
5. The distributed database load balancing method according to claim 1, wherein the FPGA of each database node obtains data to be migrated corresponding to the own node by using the target balancing plan, including:
the FPGA of each database node reads corresponding data to be migrated from the memory of the CPU of the own node based on the target balance plan, and stores the data to be migrated into the DDR4 of the own node according to a preset communication strategy so as to finish the sending operation of the data to be migrated stored in the DDR4 based on the corresponding migration path.
6. The method for balancing load of distributed database according to claim 5, wherein after the FPGA of each database node determines the corresponding data to be migrated based on the target balancing plan and migrates the data to be migrated into the DDR4 on the preset load balancing board corresponding to the node of the FPGA of each database node to complete the corresponding data migration operation, the method further comprises:
the FPGA of each database node stores the data to be migrated in the DDR4 of the self node into the memory of the CPU of the self node based on the preset communication strategy.
7. The distributed database load balancing device is characterized by being applied to an FPGA (field programmable gate array) on a preset load balancing board of a target database node, wherein the target database node is a management node in a distributed database system; wherein the device comprises:
the request sending module is used for sending a corresponding monitoring request to the CPU of the target database node based on a preset regular monitoring time point through a preset interface so that the CPU monitors the average daily access number of the target database node based on the monitoring request;
the plan determining module is used for summarizing the average daily access numbers of other database nodes in the distributed database system at present and determining a corresponding target balance plan based on a preset balance strategy and the average daily access numbers corresponding to the database nodes in the distributed database system;
the data migration module is used for distributing the target balance plan in the distributed database system so that the FPGA of each database node executes corresponding data migration operation based on data to be migrated corresponding to the node in the target balance plan and corresponding migration paths and/or data to be migrated;
wherein, the data migration module includes:
the data migration unit is used for the FPGA of each database node to acquire data to be migrated corresponding to the node of the FPGA based on the target balance plan, and sends the data to be migrated according to an migration path corresponding to the data to be migrated in the target balance plan so as to complete corresponding data migration operation;
and/or the data migration unit is used for determining corresponding data to be migrated by the FPGA of each database node based on the target balance plan, and migrating the data to be migrated into the DDR4 on the preset load balance board card corresponding to the self node so as to complete corresponding data migration operation.
8. An electronic device, comprising:
a memory for storing a computer program;
a processor for executing the computer program to implement the distributed database load balancing method of any one of claims 1 to 6.
9. A computer readable storage medium for storing a computer program which when executed by a processor implements a distributed database load balancing method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310333844.7A CN116028234B (en) | 2023-03-31 | 2023-03-31 | Distributed database load balancing method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310333844.7A CN116028234B (en) | 2023-03-31 | 2023-03-31 | Distributed database load balancing method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116028234A CN116028234A (en) | 2023-04-28 |
| CN116028234B true CN116028234B (en) | 2023-07-21 |
Family
ID=86089788
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310333844.7A Active CN116028234B (en) | 2023-03-31 | 2023-03-31 | Distributed database load balancing method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116028234B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107577530A (en) * | 2016-07-04 | 2018-01-12 | 中兴通讯股份有限公司 | Board, the method and system of balanced board memory usage |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE112011103497T5 (en) * | 2010-11-22 | 2013-08-14 | International Business Machines Corp. | Information processing system, information processing device, load balancing method, database provisioning scheduling method, and load balancing connection distribution program in a distributed database |
| CN102244685B (en) * | 2011-08-11 | 2013-09-18 | 中国科学院软件研究所 | Distributed type dynamic cache expanding method and system for supporting load balancing |
| CN102929715B (en) * | 2012-10-31 | 2015-05-06 | 曙光云计算技术有限公司 | Method and system for scheduling network resources based on virtual machine migration |
| CN107315637B (en) * | 2016-04-26 | 2020-07-31 | 南京中兴新软件有限责任公司 | Load balancing method and device of signal processing module |
| CN107480254B (en) * | 2017-08-14 | 2021-05-11 | 上海交通大学 | Online load balancing method suitable for distributed memory database |
| CN112578997B (en) * | 2019-09-30 | 2022-07-22 | 华为云计算技术有限公司 | Data migration method, system and related equipment |
-
2023
- 2023-03-31 CN CN202310333844.7A patent/CN116028234B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107577530A (en) * | 2016-07-04 | 2018-01-12 | 中兴通讯股份有限公司 | Board, the method and system of balanced board memory usage |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116028234A (en) | 2023-04-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108829352B (en) | User quota method and system for distributed storage system | |
| CN108205541B (en) | Method and device for scheduling distributed web crawler tasks | |
| CN104077212A (en) | Pressure test system and method | |
| CN111666159A (en) | Load balancing control method and device, storage medium and electronic equipment | |
| CN111767144B (en) | Transaction route determination method, device, equipment and system for transaction data | |
| CN111966289A (en) | Partition optimization method and system based on Kafka cluster | |
| CN109788027A (en) | Method of data synchronization, device, server and computer storage medium | |
| CN105468619B (en) | Resource allocation methods and device for database connection pool | |
| CN107135279A (en) | It is a kind of to handle the method and apparatus that request is set up in long connection | |
| CN106302640A (en) | Data request processing method and device | |
| CN107343023A (en) | Resource allocation methods, device and electronic equipment in a kind of Mesos management cluster | |
| CN114185675A (en) | Resource management method, device, electronic equipment and storage medium | |
| CN112311585A (en) | Method and device for selecting management node, equipment and storage medium thereof | |
| CN113965587B (en) | Data acquisition method, device, equipment and medium of artificial intelligence platform | |
| CN116028234B (en) | Distributed database load balancing method, device, equipment and storage medium | |
| CN110795041A (en) | Quota method, quota device, server and storage medium of distributed file system | |
| CN114442952A (en) | Cold data migration method and device, storage medium and electronic device | |
| CN109002381A (en) | Process communication monitoring method, electronic device and computer readable storage medium | |
| CN109257256A (en) | Apparatus monitoring method, device, computer equipment and storage medium | |
| CN110188258B (en) | Method and device for acquiring external data by using crawler | |
| WO2015067051A1 (en) | Test agent method and device thereof | |
| CN117667358A (en) | Container group scheduling method, system, electronic device, cluster and readable storage medium | |
| CN104468674B (en) | Data migration method and device | |
| CN115580618B (en) | Load balancing method, device, equipment and medium | |
| CN111813621B (en) | Data processing method, device, equipment and medium based on Flume data center |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |