CN114448845A - Cluster management system based on dispatching center - Google Patents

Abstract

The invention relates to the technical field of block chain application, and particularly discloses a cluster management system based on a scheduling center, which comprises a service database, an execution node group, a configuration database and a task scheduling center, wherein the execution node group is associated with the service database and is used for executing a service, the configuration database is associated with the execution node group and is used for storing and configuring node data, the task scheduling center is associated with the configuration database and is used for calling the execution node, and a plurality of single execution nodes are arranged in the execution node group.

Description

Cluster management system based on dispatching center

Technical Field

The application relates to the technical field of block chain application, and particularly discloses a cluster management system based on a dispatching center.

Background

In the conventional cluster management system, a node state monitoring management program is lacked, the running state of each node cannot be monitored in real time, and the information of all running nodes cannot be checked;

the node expansion mode is manual expansion, and after the mirror image is deployed to the corresponding node, the project configuration in the node is manually changed, so that the expansion is realized, and the method is complex;

and the up-down flow is complex, the project inside the node needs to be started or stopped manually, and the usability of the node is difficult to check.

Therefore, the present invention provides a cluster management system based on a dispatch center to solve the above problems.

Disclosure of Invention

The invention aims to solve the problem that the running state of each node cannot be monitored in real time due to the fact that the traditional cluster management system lacks the node state monitoring management process.

In order to achieve the above object, a basic solution of the present invention provides a cluster management system based on a dispatching center, including a service database: the service database contains a specific service list;

executing the node group: is associated with a service database and is used for executing services;

configuring a database: associated with the executing node group and used for storing and configuring node data;

the task scheduling center: is associated with the configuration database and is used for calling the execution node;

and a plurality of single execution nodes are arranged in the execution node group.

Further, the single node refers to a virtual machine or a physical machine that actually executes a specific service logic. Each virtual machine and physical machine is a separate execution node.

Further, the service database contains a specific task list, and the execution node can be controlled by the task scheduling center to pull the task from the service database for execution. The task scheduling center controls the execution state of the execution node to perform the task, and the state of the node can be directly observed through the task scheduling center.

Further, the specific process of the executing node pulling the task from the service database is as follows:

step A001: sending a calling request to an execution node by a task scheduling center;

step A002: after receiving the calling request, the execution node inquires the longest task which is not executed in the service database, and modifies the state of the task into the execution state;

step A003: the executing node executes its corresponding service function.

Further, the strategy for invoking the execution node by the task scheduling center comprises a continuously executed routing strategy and a blocking processing strategy.

Further, the routing policy is: when the dispatching center detects that the executing node has the task to execute or has the fault, the task is distributed to the next executing node.

Further, the blocking handling policy is: when a routing strategy has a problem, the executing node receives a new task when executing the task, the executing node waits for the current task to be executed and then executes the new task, the new task is not discarded, and the old task is not covered.

Further, the invention can also detect the newly added execution node as follows:

step S001: starting an execution node, observing whether configuration information of the execution node appears in a task scheduling center, if so, indicating that the node is successfully online, and if not, indicating that the node is failed to be online;

step S002: and calling the service content of the node through the scheduling center, wherein if the task can be pulled from the service database and executed, the node is available, and if the task cannot be pulled from the service database or the task cannot be executed, the node is unavailable.

Compared with the prior art, the invention has the beneficial effects that:

1. the cluster maintenance is convenient, all the node information is displayed in one list, and the fault node can be quickly positioned.

2. The invention has convenient on-line and off-line processes, and can realize on-line and off-line of one node only by modifying the database.

3. The node availability of the invention is convenient to check, and the availability can be verified by directly calling the service of the corresponding node in the task scheduling center.

4. Compared with the prior art, the method and the system have the advantages that the corresponding node can be directly called in the task scheduling center through the cooperation of the task scheduling center and the configuration database, so that the information and the availability of the node can be monitored, and the problem that the running state of each node cannot be monitored in real time due to the fact that the traditional cluster management system lacks a node state monitoring and managing process is solved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic diagram illustrating a framework of a cluster management system based on a scheduling center according to an embodiment of the present application;

fig. 2 is a schematic flowchart illustrating a cluster management system based on a dispatch center according to an embodiment of the present application;

fig. 3 shows a schematic flowchart of a cluster management system based on a scheduling center according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The following is further detailed by the specific embodiments:

a cluster management system based on a dispatch center, the embodiment is shown in fig. 1: the system comprises a configuration database, a task scheduling center, an execution node cluster and a service database, wherein the service database is associated with an execution node group, the execution node group is used for executing a service to be processed, the execution node group is associated with the configuration database, the configuration database is used for storing and configuring node data, the configuration database is also associated with the task scheduling center, the task scheduling center is used for calling the execution nodes, and a plurality of single execution nodes are arranged in the execution node group;

the task scheduling center has the following specific functions: acquiring information of a timing task from a configuration database, calling a proper execution node according to the requirement of task information, wherein the execution node needs to be in an idle state and is not in a fault state, and detecting that the execution node needs to be in the idle state and is not in the fault state by a task scheduling center by sending a heartbeat detection request;

the specific functions of the executing node cluster are as follows: the method comprises the following steps that a plurality of execution nodes capable of being scheduled by a task scheduling center are included, the execution nodes are virtual machines or physical machines which actually execute automation scripts for evidence obtaining, the execution node cluster deployment is carried out, and the task scheduling center creates a list according to all perceived online execution nodes, such as 127.0.0.1:9997,127.0.0.1:9998,127.0.0.1:9999, and the ordering rule is as follows: the method comprises the steps of ascending, firstly dividing into four sections according to ip, sequentially comparing from left to right, ascending according to port numbers when the ip is the same, and sending heartbeat detection requests to an actuator according to the sequence when a scheduling center initiates scheduling requests each time;

the task scheduling center firstly carries out heartbeat detection on a first address, heartbeat failure is automatically skipped, a second still heartbeat detection fails … … until a third address 127.0.0.1:9999 of heartbeat detection is successful, and the third address is selected as a target actuator; then, sending a scheduling request to a target executor;

the heartbeat mechanism is as follows: the execution node sends data of the ip address, the end number and the survival state of the execution node to the dispatching center every 10 seconds, if the dispatching center does not receive the survival data sent by the execution node within 1 minute, the node is considered to be in fault, and the node is not called any more. If the dispatching center receives the data sent by the execution node within 1 minute, the function is normal, and the timing of the dispatching center is reset for one minute;

heartbeat detection request: the scheduling center sends a heartbeat request to the execution node, the execution node does not wait for 10 seconds any more, but directly sends survival state information to the scheduling center, if the survival state information is not received in the scheduling center 20s, the execution node is considered to be in a fault, if the survival state information is received in the scheduling center 20s, the state of the execution node is normal, and a scheduling request is sent to a target actuator;

as can be known from fig. 1, the execution node group includes a plurality of individual execution nodes, the individual execution nodes can be associated with each other, each execution node is a minimum service execution unit, the individual execution nodes in the execution node group can be called by the task scheduling center, and the configuration information of each execution node in the execution node group is stored in the configuration database, the execution node group is further associated with the service database, the service database includes a specific task list, the execution nodes can be controlled by the task scheduling center to pull tasks from the service database for execution, and then write the executed task information into the database to complete the feedback of the information, the configuration database is further associated with the task scheduling center, the task scheduling center can display all the node information that normally runs and call the nodes to execute the tasks, and the user can determine the node through the node information displayed by the task scheduling center, and simultaneously checking whether the node works normally, wherein the method for judging whether the node works normally comprises the following steps:

firstly, a scheduling center sends a scheduling request to an execution node, the execution node immediately sends a response to the scheduling center after receiving the scheduling request, and if the scheduling center does not receive the response after sending the scheduling request, the execution node is in a fault state;

when the execution node works normally, the execution node pulls the task from the service database, and the specific flow of the task pulled by the execution node is as follows:

step A001, a task scheduling center sends a calling request to an execution node;

step A002, the executing node receives the calling request and then queries the longest task which is not executed in the service database;

step A003, the executive node modifies the state of the selected task into execution, and then the executive node executes the corresponding service function;

the strategy for the task scheduling center to call the execution node comprises two parts of continuous execution, which are respectively and sequentially as follows: executing strategy and error correction strategy, and when the first part fails, replacing the first part by the second part to complete the work of the first part;

the first part is an execution policy, also called a routing policy:

when the dispatching center detects that the execution node has a task to execute or has a fault, the dispatching center distributes the task to the next execution node, and the distribution step is to firstly send a calling request to the execution node and then the execution node calls the task to be executed or the task which is not completed due to the fault to execute;

the specific routing strategy is as follows:

a. the task scheduling center initiates a heartbeat request to a corresponding execution node;

b. after receiving a heartbeat request initiated by a task scheduling center, an execution node feeds back state information to the task scheduling center within 10 s;

C. if the task scheduling center does not receive the state information fed back by the execution node within 20s, judging that the execution node has a fault, and performing the step a on the next execution node;

the second part is an error correction strategy, also called a blocking processing strategy: if the routing strategy has problems, the execution node receives a new task when executing the task, and executes the new task after waiting for the completion of the execution of the current task, so that the new task cannot be discarded, and the old task cannot be covered;

the specific blocking handling strategy is as follows:

d. the task scheduling center initiates a heartbeat request to the execution node marked in a normal state;

e. after receiving a heartbeat request initiated by a task scheduling center, an execution node marked as a normal state feeds back state information to the task scheduling center within 10 s;

f. if the task scheduling center receives the status information fed back by the execution node marked as the normal state within 10s, the execution node is judged to be idle, if the task scheduling center receives the status information fed back by the execution node marked as the normal state within 10 s-20 s, the execution node is judged to be busy, if the task scheduling center does not receive the status information fed back by the execution node marked as the normal state within 20s, the execution node is judged to be in fault, the information is fed back to the task scheduling center, after receiving the feedback information of the busy or fault of the execution node, the task scheduling center sequentially detects whether the next execution node is in fault or not, selects one execution node in the normal state, marks the execution node as the normal state, and performs the step d on the execution node marked as the normal state according to the number, until an execution node which is not in a fault state and is in an idle state is selected and marked as a target execution node; then sending a scheduling request to a target execution node, and scheduling the execution node to execute an automatic evidence obtaining task;

the heartbeat detection feedback time in the invention is not limited to 10s or 20s, and can be set for 2s, 5s and the like according to the use requirement and the calculation effectiveness;

as shown in fig. 2 and fig. 3, the present invention can also detect an executing node, and the detecting method includes:

starting an execution node, observing in a task scheduling center, and when the configuration information of the execution node can be observed in the task scheduling center, indicating that the node is successfully online;

if the configuration information of the execution node is not seen in the task scheduling center, indicating that the node is not successfully online, and feeding back the information;

meanwhile, the service content of the node is called in the dispatching center, the task can be pulled from the service database and completely executed, namely, the availability of the node can be verified;

if the node cannot pull the task from the service database or cannot completely execute the task after pulling the task, indicating that the node is unavailable;

when the nodes need to be expanded, deploying the mirror images to new nodes, automatically adding the new nodes to an execution node group, starting the execution nodes, checking whether the newly added nodes are successfully online through a task scheduling center, and continuously verifying the usability of the nodes if the newly added nodes are successfully online, so that the newly added nodes can complete the work to be completed and can be put into task allocation;

the mirror image deployment is mainly realized through the function of Alice cloud, the configuration information of a new node is written into a configuration database for storage, the configuration information mainly comprises a scheduling center address, a port number, a packet name and the like, the configuration information is used for directly connecting the node to a task scheduling center, and the configuration information is stored in a key value pair method, so that the expansion of subsequent configuration attributes is facilitated;

in summary, compared with the prior art, the invention can directly call the corresponding node in the task scheduling center through the cooperation of the task scheduling center and the configuration database, so as to monitor the information and the availability of the node, and solve the problem that the running state of each node cannot be monitored in real time due to the lack of the process of monitoring and managing the node state in the traditional cluster management system.

While, for purposes of simplicity of explanation, the methodologies are shown and described as a series of acts, it is to be understood and appreciated that the methodologies are not limited by the order of acts. In accordance with one or more embodiments, some acts may occur in different orders and/or concurrently with other acts from that shown and described herein or not shown and described herein but understood by those of ordinary skill in the art. To further enable those of skill in the art, the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention. The various illustrative logical blocks, modules, and circuits described in connection with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal. In one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a web site, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk (disk) and disc (disc), as used herein, includes Compact Disc (CD), laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk and blu-ray disc where disks (disks) usually reproduce data magnetically, while discs (discs) reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

Claims (8)

1. A cluster management system based on a dispatching center is characterized by comprising a service database: the service database contains a specific service list;

executing the node group: is associated with the service database and is used for executing the service;

configuring a database: associated with the executing node group and used for storing and configuring node data;

the task scheduling center: is associated with the configuration database and is used for calling the execution node;

and a plurality of single execution nodes are arranged in the execution node group.

2. The dispatching center-based cluster management system of claim 1, wherein the single node refers to a virtual machine or a physical machine actually executing a specific service logic.

3. The system according to claim 2, wherein the service database contains a specific task list, and the executing node is controlled by the task scheduling center to pull the task from the service database for execution.

4. The system of claim 2, wherein the specific flow of the executing node pulling the task from the service database is as follows:

step A001: sending a calling request to an execution node by a task scheduling center;

step A002: after receiving the calling request, the execution node inquires the longest task which is not executed in the service database, and modifies the state of the task into the execution state;

step A003: the executing node executes its corresponding service function.

5. The dispatching center-based cluster management system of claim 4, wherein the task dispatching center invokes policies of the executing node comprising a continuously executing routing policy and a blocking handling policy.

6. The dispatch center-based cluster management system of claim 5, wherein the routing policy is: when the dispatching center detects that the executing node has the task to execute or has the fault, the task is distributed to the next executing node.

7. The dispatch center-based cluster management system of claim 6, wherein the blocking handling policy is: when a routing strategy has a problem, the executing node receives a new task when executing the task, the executing node waits for the current task to be executed and then executes the new task, the new task is not discarded, and the old task is not covered.

8. The cluster management system based on the dispatching center as claimed in claim 2, wherein the invention further performs the following detection on the newly added executing node:

step S001: starting an execution node, observing whether configuration information of the execution node appears in a task scheduling center, if so, indicating that the node is successfully online, and if not, indicating that the node is failed to be online;

step S002: and calling the service content of the node through the scheduling center, wherein if the task can be pulled from the service database and executed, the node is available, and if the task cannot be pulled from the service database or the task cannot be executed, the node is unavailable.

Images