CN112068954B - Method and system for scheduling network computing resources - Google Patents
Method and system for scheduling network computing resources Download PDFInfo
- Publication number
- CN112068954B CN112068954B CN202010833765.9A CN202010833765A CN112068954B CN 112068954 B CN112068954 B CN 112068954B CN 202010833765 A CN202010833765 A CN 202010833765A CN 112068954 B CN112068954 B CN 112068954B
- Authority
- CN
- China
- Prior art keywords
- task
- personal computing
- computer cluster
- computing resources
- computer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000004364 calculation method Methods 0.000 claims abstract description 54
- 238000012545 processing Methods 0.000 claims abstract description 25
- 238000004891 communication Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 238000004590 computer program Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013515 script Methods 0.000 description 2
- 238000011960 computer-aided design Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
- G06F9/5088—Techniques for rebalancing the load in a distributed system involving task migration
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multi Processors (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention relates to a method and a system for scheduling network computing resources, wherein the method comprises the following steps: acquiring tasks and task sets on at least one client, and performing task scheduling on the tasks and task sets on all the clients to acquire a system total task schedule; according to the arrangement sequence of the tasks to be calculated in the system total task scheduling, aiming at each task to be calculated in the system total task scheduling, judging whether each task in the tasks to be calculated is a calculation type larger than a preset scale, and acquiring a judging result of the tasks; according to the task, the judging result of the task, the information of the hardware and software environment provided by each computer cluster in the network and the information of the hardware and software environment provided by each personal computing resource, matching the task with the computer clusters or the personal computing resources in the network, determining the computer clusters or the personal computing resources matched with the task, and calculating the task processing result by the computer clusters or the personal computing resources matched with the task.
Description
Technical Field
The present invention relates to the field of network computing resource management technologies, and in particular, to a method and a system for scheduling network computing resources.
Background
Computer aided design, computer aided engineering simulation and scientific calculation are mainly performed by local resources or computing resources in a network, and the network computing resources comprise not only computer clusters specially used for large-scale computing, but also desktops, small-sized workstations, notebook computers, other mobile office terminals and the like used by engineers in a personal way. Typically, small-scale computing tasks are placed on local computers to complete the computing work, and large-scale computing tasks are submitted to large computing clusters on the network to complete the computing work.
However, the traditional network computing resource scheduling method is designed aiming at a medium-large computer cluster at a far end of a network, and a computing task can only complete computation at a local computer or on a medium-large computer cluster fixed on the network, so that computing resources used by various design developers in the network, including a desktop computer, a small workstation, a notebook computer, other mobile office terminals and the like cannot be called. The method has the defects that on one hand, a large number of calculation tasks are queued up on a medium-large computer cluster, and time is wasted; on the other hand, a large amount of computing resources on the local area network are idle, and resources are wasted.
Disclosure of Invention
First, the technical problem to be solved
In view of the above-mentioned drawbacks and shortcomings of the prior art, the present invention provides a method and system for scheduling network computing resources, which solve the technical problem of idle computing resources.
(II) technical scheme
In order to achieve the above purpose, the main technical scheme adopted by the invention comprises the following steps:
In a first aspect, an embodiment of the present invention provides a method for scheduling network computing resources, including:
S1, acquiring tasks and task sets on at least one client, and performing task scheduling on the tasks and task sets on all the clients to acquire a system total task schedule;
the system total task scheduling comprises tasks to be calculated which are sequentially arranged, wherein the tasks to be calculated are one task or a plurality of tasks;
the task set comprises a plurality of tasks, and the tasks comprise information of a hard-software environment required by computing the tasks;
S2, judging whether each task in the tasks to be calculated is a calculation type larger than a preset scale or not according to the arrangement sequence of the tasks to be calculated in the system total task scheduling and aiming at each task to be calculated in the system total task scheduling, and acquiring a judgment result of the tasks;
Judging whether each task in the tasks to be calculated is a calculation type larger than a preset scale, specifically judging whether each task in the tasks to be calculated is a calculation type which needs to be processed by processor cores with the number larger than or equal to a preset number of cores;
S3, matching the task with the computer clusters or the personal computing resources in the network according to the task, the judging result of the task and the obtained information of the hard software environment provided by each computer cluster in the network and the obtained information of the hard software environment provided by each personal computing resource, determining the computer clusters or the personal computing resources matched with the task, and calculating the task by the computer clusters or the personal computing resources matched with the task to obtain the processing result of the task.
Preferably, the step S2 specifically includes:
aiming at each task to be calculated in the total task scheduling of the system, when the task to be calculated is one task, judging whether the task is a calculation type larger than a preset scale, and acquiring a judgment result of the task;
Aiming at each task to be calculated in the total task scheduling of the system, when the task to be calculated has a plurality of tasks, sequentially judging whether each task in the tasks to be calculated is a calculation type larger than a preset scale according to a preset first sequence, and acquiring a judgment result of each task;
Preferably, the step S3 includes:
If the judging result of the task is a calculation type larger than a preset scale, matching the task according to the hardware and software environment information required in the task and the hardware and software environment information provided by each computer cluster in the network, determining the computer cluster matched with the task, and calculating the task by the computer cluster matched with the task to obtain the processing result of the task;
If the judgment result of the task is not a calculation type larger than a preset scale, matching the task according to the hardware and software environment information required in the task and the hardware and software environment information provided by each personal calculation resource in the network, determining the personal calculation resource matched with the task, and calculating the task by the personal calculation resource matched with the task to obtain the processing result of the task;
The information of the hardware and software environment provided by the computer cluster comprises information provided by the computer cluster: the method comprises the steps of processing CPU core number, main frequency of the processor CPU, memory RAM space, storage space, network communication information and information of software tools installed in a computer cluster;
The information of the hardware and software environment provided by the personal computing resource comprises information provided by the personal computing resource: processor CPU core number, processor CPU main frequency, memory RAM space, storage space, network communication information and information of installed software-like tools of personal computing resources.
Preferably, the step S3 includes:
If the judging result of the task is a calculation type larger than a preset scale, matching the task according to the hardware and software environment information required in the task and the hardware and software environment information provided by each computer cluster in the network, and determining the computer cluster matched with the task;
Submitting the task to a computer cluster matched with the task; the computer cluster matched with the task receives the task, and performs task scheduling according to a preset rule aiming at the task and the pre-received task to obtain a local task scheduling of the computer cluster;
the local task scheduling of the computer cluster comprises the tasks and the tasks which are received in advance and are arranged according to a preset second sequence;
According to the local task scheduling of the computer cluster, the computer cluster drives hardware and software corresponding to the computer cluster to calculate the task and the task received in advance, and a processing result of each task in the local task scheduling of the computer cluster is obtained;
the processing result of the task comprises: the calculation result of the task or the error information of the task.
Preferably, in the step S3, the task is matched according to the hardware and software environment information required in the task and the hardware and software environment information provided by each computer cluster in the network, and the determining the computer cluster matched with the task specifically includes:
acquiring a first computer cluster according to the hardware and software environment information required in the task and the hardware and software environment information provided by each computer cluster in the network;
the first computer cluster is a computer cluster in which the hardware and software environment information provided in the network meets the hardware and software environment information required in the task;
When the number of the first computer clusters is one, determining the first computer clusters as computer clusters matched with the task;
When the number of the first computer clusters is a plurality of, determining the computer clusters matched with the task according to the hardware environment information provided by the first computer clusters;
The hardware environment information provided by the first computer cluster includes that provided by the first computer cluster: processor CPU core number, processor CPU main frequency, memory RAM space, storage space.
Preferably, when the number of the first computer clusters is a plurality of, determining a computer cluster matched with the task according to the hardware environment information provided by the first computer cluster specifically includes:
When the number of the first computer clusters is a plurality of, acquiring the first computer cluster with the largest number of provided processor CPU cores;
when the number of the first computer clusters with the largest number of the provided processor CPU cores is one, determining the first computer clusters with the largest number of the provided processor CPU cores as the computer clusters matched with the task;
When the number of the first computer clusters with the largest number of the provided processor CPU cores is a plurality of, acquiring a second computer cluster with the highest main frequency of the provided processor CPU;
The second computer cluster is a first computer cluster with the largest number of CPU cores of the provided processor;
when the number of the second computer clusters with the highest main frequency of the provided processor CPU is one, determining the second computer cluster with the highest main frequency of the provided processor CPU as a computer cluster matched with the task;
when the number of the second computer clusters with the highest main frequency of the CPU is a plurality of, acquiring a third computer cluster with the largest memory RAM space;
the third computer cluster is a second computer cluster with highest main frequency for providing a CPU of the processor;
when the number of the third computer clusters with the largest memory RAM space is one, determining the third computer cluster with the largest memory RAM space as the computer cluster matched with the task;
When the number of the third computer clusters with the largest memory RAM space is a plurality of, acquiring a fourth computer cluster with the largest memory space;
the fourth computer cluster is a third computer cluster with the largest memory RAM space;
When the number of the fourth computer clusters with the largest provided storage space is one, determining the fourth computer cluster with the largest provided storage space as a computer cluster matched with the task;
when the number of the fourth computer clusters with the largest provided storage space is a plurality of, determining one fourth computer cluster with the largest provided storage space from the fourth computer clusters with the largest provided storage space as the computer cluster matched with the task.
Preferably, the step S3 includes:
If the judgment result of the task is not a calculation type larger than a preset scale, matching the task according to the hardware and software environment information required in the task and the hardware and software environment information provided by each personal calculation resource in the network, and determining the personal calculation resource matched with the task;
submitting the task to a personal computing resource matched with the task; the personal computing resource matched with the task receives the task, and performs task scheduling according to preset rules aiming at the task and the pre-received task to obtain local task scheduling of the personal computing resource;
The local task scheduling of the personal computing resource comprises the tasks and the tasks which are received in advance and arranged according to a preset third sequence;
According to the local task scheduling of the personal computing resource, the personal computing resource drives hardware and software corresponding to the personal computing resource to calculate the task and the task received in advance, and a processing result of each task in the local task scheduling of the personal computing resource is obtained;
the processing result of the task comprises: the calculation result of the task or the error information of the task.
Preferably, in the step S3, the task is matched according to the hardware and software environment information required in the task and the hardware and software environment information provided by each personal computing resource in the network, and the personal computing resource matched with the task is determined, which specifically includes:
Acquiring a first personal computing resource according to the hardware and software environment information required in the task and the hardware and software environment information provided by each personal computing resource in the network;
the first personal computing resource is a personal computing resource in a network which meets the hardware and software environment information required in the task;
when the number of the first personal computing resources is one, determining the first personal computing resources to be personal computing resources matched with the task;
when the number of the first personal computing resources is a plurality of, determining the personal computing resources matched with the task according to the hardware environment information provided by the first personal computing resources;
the hardware environment information provided by the first personal computing resource includes that provided by the first personal computing resource: processor CPU core number, processor CPU main frequency, memory RAM space, storage space.
Preferably, when the number of the first personal computing resources is a plurality of, determining the personal computing resources matched with the task according to the hardware environment information provided by the first personal computing resources specifically includes:
when the number of the first personal computing resources is a plurality of, acquiring the first personal computing resource with the largest processor CPU core number;
When the number of the first personal computing resources with the largest number of the provided processor CPU cores is one, determining the first personal computing resources with the largest number of the provided processor CPU cores as the personal computing resources matched with the task;
When the number of the first personal computing resources with the largest number of available processor CPU cores is a plurality of, acquiring a second personal computing resource with the highest main frequency of the processor CPU;
The second personal computing resource is a first personal computing resource with the maximum number of CPU cores of the processor;
when the number of the second personal computing resources with the highest main frequency of the CPU provided by the processor is one, determining the second personal computing resources with the highest main frequency of the CPU provided by the processor as the personal computing resources matched with the task;
when the number of the second personal computing resources with the highest main frequency of the CPU of the providing processor is a plurality of, acquiring a third personal computing resource with the largest memory RAM space;
The third personal computing resource is a second personal computing resource with highest main frequency for providing a CPU of the processor;
when the number of the third personal computing resources with the largest memory RAM space is one, determining the third personal computing resources with the largest memory RAM space as the personal computing resources matched with the task;
When the number of the third personal computing resources with the largest available memory RAM space is a plurality of, acquiring a fourth personal computing resource with the largest storage space;
the fourth personal computing resource is a third personal computing resource with the largest memory RAM space;
When the number of the fourth personal computing resources with the largest provided storage space is one, determining the fourth personal computing resources with the largest provided storage space as personal computing resources matched with the task;
When the number of the fourth personal computing resources with the largest provided storage space is a plurality, determining one fourth personal computing resource with the largest provided storage space as the personal computing resource matched with the task according to the fourth personal computing resources with the largest provided storage space.
On the other hand, the embodiment also provides a system for scheduling network computing resources, which comprises:
At least one processor; and
At least one memory communicatively coupled to the processor, wherein the memory stores program instructions executable by the processor, the processor invoking the program instructions capable of performing any of the methods of network computing resource scheduling described above.
(III) beneficial effects
The beneficial effects of the invention are as follows: the invention relates to a method for scheduling network computing resources, which is characterized in that on one hand, tasks and task sets input by various clients are received, and then the tasks are scheduled, namely, the scheduling of the total tasks of a system based on the tasks input by users is performed; on the other hand, the hardware and software environment of each computing resource is detected and matched and compared with the current task sent by the total task scheduling system, and compared with the prior art, the network computing resource scheduling method can realize the balanced utilization of all computing resource loads in the network system and realize the maximization of the utilization efficiency.
Drawings
FIG. 1 is a flow chart of a method for scheduling network computing resources according to the present invention;
FIG. 2 is a schematic diagram of a method for scheduling network computing resources according to an embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating task scheduling in a method for scheduling network computing resources according to an embodiment of the present invention;
Fig. 4 is a task matching diagram in a method for scheduling network computing resources according to an embodiment of the present invention.
Detailed Description
The invention will be better explained by the following detailed description of the embodiments with reference to the drawings.
In order that the above-described aspects may be better understood, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
Referring to fig. 1 and 2, the present embodiment provides a method for scheduling network computing resources, including:
S1, acquiring tasks and task sets on at least one client, and performing task scheduling on all the tasks and task sets on the client according to preset rules to acquire a system total task schedule.
The system total task schedule comprises tasks to be calculated which are sequentially arranged.
The task to be calculated is one task or a plurality of tasks.
The set of tasks includes a plurality of tasks.
The tasks include information of the hardware and software environment required for calculating the tasks.
The client in this embodiment includes: intelligent terminal equipment such as personal workstation, notebook computer, cell-phone.
Boundary of task in this embodiment: including input conditions and output result definitions.
Task set in this embodiment: the tasks are arranged into a serial task set, a parallel task set or a serial-parallel mixed task set according to a certain logic sequence.
In this embodiment, task scheduling is performed on all tasks and task sets on the client according to a preset rule to obtain a system total task schedule, which may be that all tasks and task sets on the client are scheduled according to a preset logic relationship such as a time sequence, a priority sequence, and the like, to obtain the system total task schedule.
S2, according to the arrangement sequence of the tasks to be calculated in the system total task scheduling, judging whether each task in the tasks to be calculated is a calculation type larger than a preset scale or not according to each task to be calculated in the system total task scheduling, and acquiring a judgment result of the tasks.
In the invention, whether each task in the tasks to be calculated is a calculation type larger than a preset scale is judged, and specifically, whether each task in the tasks to be calculated is a calculation type which needs to be processed by processor cores with the number larger than or equal to a preset number of cores is judged.
For example, if the preset scale calculation type is a calculation type with a preset core number of 64 cores, judging whether each task in the tasks to be calculated is a calculation type larger than the preset scale, specifically judging whether the task needs 64 cores or more than 64 cores for processing, if the minimum processor core number needed by the task is larger than or equal to 64 cores, the system can define the task as being larger than the preset scale calculation type; in this embodiment, the number of processor cores required by the calculation type of the preset scale is not limited, that is, if the calculation type of the preset scale is a calculation type with 128 cores, it is determined whether each task in the tasks to be calculated is a calculation type greater than the preset scale, specifically, whether the task needs 128 cores or more than 128 cores for processing, and if the minimum number of processor cores required by the task is greater than or equal to 128 cores, the system may define the task as being greater than the preset scale calculation type.
In this embodiment, the task to be calculated may be a single task; in this embodiment, the task to be calculated may be a plurality of tasks satisfying parallel processing.
S3, matching the task with the computer clusters or the personal computing resources in the network according to the task, the judging result of the task, the obtained information of the hard software environment provided by each computer cluster in the network and the obtained information of the hard software environment provided by each personal computing resource, determining the computer clusters or the personal computing resources matched with the task, and calculating the task by the computer clusters or the personal computing resources matched with the task to obtain the processing result of the task.
Preferably, the step S2 specifically includes:
and aiming at each task to be calculated in the total task scheduling of the system, when the task to be calculated is one task, judging whether the task is a calculation type larger than a preset scale, and acquiring a task judgment result.
Aiming at each task to be calculated in the total task scheduling of the system, when the task to be calculated has a plurality of tasks, sequentially judging whether each task in the tasks to be calculated is a calculation type larger than a preset scale according to a preset first sequence, and acquiring a judgment result of each task.
The first order preset in this embodiment is a predetermined default order.
Preferably, the method comprises the steps of,
The information of the hardware and software environment provided by the computer cluster comprises information provided by the computer cluster: processor CPU core number, processor CPU main frequency, memory RAM space, storage space, network communication information and information of software tools installed in the computer cluster.
The information of the hardware and software environment provided by the personal computing resource comprises information provided by the personal computing resource: processor CPU core number, processor CPU main frequency, memory RAM space, storage space, network communication information and information of installed software-like tools of personal computing resources.
Preferably, the step S3 includes:
and if the judging result of the task is a calculation type larger than a preset scale, matching the task according to the hardware and software environment information required by the task and the hardware and software environment information provided by each computer cluster in the network, and determining the computer cluster matched with the task.
In this embodiment, the types and hardware performances of the personal computing resources in the network and the software included in the computer cluster are different, and the tasks submitted by the user are more various, as shown in fig. 4, by detecting the hardware-software environment of each computing resource and performing matching comparison with the current task sent by the overall task scheduling system, if matching is successful, the computing resource is used as one of the task and the candidate computing resource, and then the upper software of the computing resource is automatically driven to solve and calculate the task received in the host.
Submitting the task to a computer cluster matched with the task; and receiving the task by the computer cluster matched with the task, and aiming at the task and the task which is received in advance, performing task scheduling according to a preset rule to obtain the local task scheduling of the computer cluster.
Referring to fig. 3, in this embodiment, not only task scheduling is performed on all users submitting tasks and task sets, but also task scheduling is performed on tasks received by a computer cluster in a network, so that local task scheduling of the computer cluster and a system total scheduling system are unified, and therefore, loads of all computing resources in the network system are balanced and utilized, and the utilization efficiency is maximized.
The local task scheduling of the computer cluster comprises the tasks and the tasks which are received in advance and are arranged according to a preset second sequence.
According to the local task scheduling of the computer cluster, the computer cluster drives hardware and software corresponding to the computer cluster to calculate the task and the task received in advance, and a processing result of each task in the local task scheduling of the computer cluster is obtained.
The processing result of the task comprises: the calculation result of the task or the error information of the task.
In this embodiment, according to a preset second order arrangement of tasks received by the computer clusters in the network, the hardware and the software corresponding to the computing resources are sequentially driven to complete the computing tasks successively, and when the solving fails, error information corresponding to the tasks is returned. The calculation result or error information is stored in a file server appointed by the user, and corresponding feedback information is sent back to the client.
Preferably, in the step S3, the task is matched according to the hardware and software environment information required in the task and the hardware and software environment information provided by each computer cluster in the network, and the determining the computer cluster matched with the task specifically includes:
And acquiring a first computer cluster according to the hardware and software environment information required in the task and the hardware and software environment information provided by each computer cluster in the network.
The first computer cluster is a computer cluster in which the hardware and software environment information provided in the network meets the hardware and software environment information required in the task.
When the number of the first computer clusters is one, the first computer clusters are determined to be the computer clusters matched with the task.
When the number of the first computer clusters is a plurality of, determining the computer clusters matched with the task according to the hardware environment information provided by the first computer clusters.
The hardware environment information provided by the first computer cluster includes that provided by the first computer cluster: processor CPU core number, processor CPU main frequency, memory RAM space, storage space.
Preferably, when the number of the first computer clusters is a plurality of, determining a computer cluster matched with the task according to the hardware environment information provided by the first computer cluster specifically includes:
When the number of the first computer clusters is a plurality of, the first computer cluster with the largest number of the provided processor CPU cores is obtained.
When the number of the first computer clusters with the largest number of the provided processor CPU cores is one, the first computer clusters with the largest number of the provided processor CPU cores are determined to be the computer clusters matched with the task.
When the number of the first computer clusters with the largest number of the provided processor CPU cores is a plurality of, a second computer cluster with the highest main frequency of the provided processor CPU is obtained.
The second computer cluster is the first computer cluster with the largest number of provided processor CPU cores.
When the number of the second computer clusters with the highest main frequency of the provided processor CPU is one, the second computer cluster with the highest main frequency of the provided processor CPU is determined to be the computer cluster matched with the task.
And when the number of the second computer clusters with the highest main frequency of the provided processor CPU is a plurality of, acquiring a third computer cluster with the largest provided memory RAM space.
The third computer cluster is a second computer cluster with highest main frequency for providing a CPU of the processor.
When the number of the third computer clusters with the largest provided memory RAM space is one, determining the third computer cluster with the largest provided memory RAM space as the computer cluster matched with the task.
And when the number of the third computer clusters with the largest provided memory RAM space is a plurality of, acquiring a fourth computer cluster with the largest provided memory space.
The fourth computer cluster is the third computer cluster with the largest memory RAM space.
And when the number of the fourth computer clusters with the largest provided storage space is one, determining the fourth computer cluster with the largest provided storage space as the computer cluster matched with the task.
When the number of the fourth computer clusters with the largest provided storage space is a plurality of, determining one fourth computer cluster with the largest provided storage space from the fourth computer clusters with the largest provided storage space as the computer cluster matched with the task.
Preferably, the step S3 includes:
And if the judgment result is that the calculation type is not larger than the preset scale, matching the task according to the hardware and software environment information required by the task and the hardware and software environment information provided by each personal calculation resource in the network, and determining the personal calculation resource matched with the task.
In this embodiment, the personal computing resources and the computer clusters in the network include different types of software and hardware performances, and the tasks submitted by the users are more various, as shown in fig. 4. The hardware and software environment of each computing resource is detected, matching comparison is carried out between the hardware and software environment of each computing resource and the current task sent by the total task scheduling system, the computing resource is used as one of the task and the candidate computing resource when matching is successful, and then the upper software of the computing resource is automatically driven to carry out solving calculation on the task received in the host.
Submitting the task to a personal computing resource matched with the task; and the personal computing resource matched with the task receives the task, and performs task scheduling according to a preset rule aiming at the task and the pre-received task to obtain the local task scheduling of the personal computing resource.
Referring to fig. 3, in this embodiment, not only task scheduling is performed on all users submitting tasks and task sets, but also task scheduling is performed on tasks received by personal computing resources in a network, so that local task scheduling of the personal computing resources and unification of a system overall scheduling system are realized, and therefore, loads of all computing resources in the network system are balanced and utilized, and utilization efficiency is maximized.
The local task schedule of the personal computing resource includes the tasks and pre-received tasks arranged in a pre-set third order.
According to the local task scheduling of the personal computing resource, the personal computing resource drives hardware and software corresponding to the personal computing resource to calculate the task and the task received in advance, and a processing result of each task in the local task scheduling of the personal computing resource is obtained.
The processing result of the task comprises: the calculation result of the task or the error information of the task.
In this embodiment, according to a preset third order sequence of tasks received by personal computing resources in the network, hardware and software corresponding to the computing resources are sequentially driven to complete computing tasks successively, and when solving fails, error information corresponding to the tasks is returned. The calculation result or error information is stored in a file server appointed by the user, and corresponding feedback information is sent back to the client.
Preferably, in the step S3, the task is matched according to the hardware and software environment information required in the task and the hardware and software environment information provided by each personal computing resource in the network, and the personal computing resource matched with the task is determined, which specifically includes:
and acquiring a first personal computing resource according to the hard software environment information required in the task and the hard software environment information provided by each personal computing resource in the network.
The first personal computing resource is a personal computing resource in the network that satisfies the hardware and software environment information required in the task.
When the number of first personal computing resources is one, then the first personal computing resource is determined to be a personal computing resource that matches the task.
When the number of the first personal computing resources is a plurality of, the personal computing resources matched with the task are determined according to the hardware environment information provided by the first personal computing resources.
The hardware environment information provided by the first personal computing resource includes that provided by the first personal computing resource: processor CPU core number, processor CPU main frequency, memory RAM space, storage space.
Preferably, when the number of the first personal computing resources is a plurality of, determining the personal computing resources matched with the task according to the hardware environment information provided by the first personal computing resources specifically includes:
When the number of the first personal computing resources is a plurality, the first personal computing resource with the largest number of processor CPU cores provided is acquired.
When the number of the first personal computing resources with the largest number of the provided processor CPU cores is one, the first personal computing resources with the largest number of the provided processor CPU cores are determined to be the personal computing resources matched with the task.
When the number of the first personal computing resources with the largest number of available processor CPU cores is a plurality of, the second personal computing resource with the highest main frequency of the processor CPU is obtained.
The second personal computing resource is the first personal computing resource providing the greatest number of processor CPU cores.
When the number of the second personal computing resources with the highest main frequency of the CPU is one, the second personal computing resources with the highest main frequency of the CPU is determined to be the personal computing resources matched with the task.
And when the number of the second personal computing resources with the highest main frequency of the CPU of the providing processor is a plurality of, acquiring a third personal computing resource with the largest memory RAM space.
The third personal computing resource is a second personal computing resource with the highest main frequency of the CPU.
When the number of the third personal computing resources with the largest provided memory RAM space is one, determining the third personal computing resources with the largest provided memory RAM space as the personal computing resources matched with the task.
And when the number of the third personal computing resources with the largest available memory RAM space is a plurality of, acquiring the fourth personal computing resources with the largest provided storage space.
The fourth personal computing resource is a third personal computing resource with the largest memory RAM space.
When the number of the fourth personal computing resources with the largest provided storage space is one, the fourth personal computing resources with the largest provided storage space are determined as the personal computing resources matched with the task.
When the number of the fourth personal computing resources with the largest provided storage space is a plurality, determining one fourth personal computing resource with the largest provided storage space as the personal computing resource matched with the task according to the fourth personal computing resources with the largest provided storage space.
According to the embodiment, the computing tasks submitted by the users are automatically distributed to the available computing resources on the network, solving tasks and result calling are completed, and the maximization of network computing resource utilization is achieved.
In the embodiment, not only the task and task set scheduling is submitted to all users, but also the task scheduling is performed to the computer clusters in the network, so that the unification of the local task scheduling of the computer clusters and the total system scheduling system is realized, and the balanced utilization of all computing resource loads in the network system is realized, and the maximization of the utilization efficiency is realized.
All engineering software supporting script driving, scientific computing software, data analysis software, physical and chemical simulation software, office document software and other software supporting scripts can be driven in the embodiment.
On the other hand, the embodiment also provides a system for scheduling network computing resources, which comprises:
At least one processor; and
At least one memory communicatively coupled to the processor, wherein the memory stores program instructions executable by the processor, the processor invoking the program instructions capable of performing any of the methods of network computing resource scheduling described above.
Since the system described in the foregoing embodiments of the present invention is a system for implementing the method of the foregoing embodiments of the present invention, those skilled in the art will be able to understand the specific structure and modification of the system based on the method of the foregoing embodiments of the present invention, and thus will not be described in detail herein. All systems used in the methods of the above embodiments of the present invention are within the scope of the present invention.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions.
It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the terms first, second, third, etc. are for convenience of description only and do not denote any order. These terms may be understood as part of the component name.
Furthermore, it should be noted that in the description of the present specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to a specific feature, structure, material, or characteristic described in connection with the embodiment or example being included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art upon learning the basic inventive concepts. Therefore, the appended claims should be construed to include preferred embodiments and all such variations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, the present invention should also include such modifications and variations provided that they come within the scope of the following claims and their equivalents.
Claims (3)
1. A method for scheduling network computing resources, comprising:
S1, acquiring tasks and task sets on at least one client, and performing task scheduling on the tasks and task sets on all the clients to acquire a system total task schedule;
the system total task scheduling comprises tasks to be calculated which are sequentially arranged, wherein the tasks to be calculated are one task or a plurality of tasks;
the task set comprises a plurality of tasks, and the tasks comprise information of a hard-software environment required by computing the tasks;
s2, judging whether each task in the tasks to be calculated is a calculation type larger than a preset scale or not according to the arrangement sequence of the tasks to be calculated in the system total task scheduling and aiming at each task to be calculated in the system total task scheduling, and acquiring a judgment result of the tasks;
S3, matching the task with the computer clusters or the personal computing resources in the network according to the task, the judging result of the task and the obtained information of the hard software environment provided by each computer cluster in the network and the obtained information of the hard software environment provided by each personal computing resource, determining the computer clusters or the personal computing resources matched with the task, and calculating the task by the computer clusters or the personal computing resources matched with the task to obtain the processing result of the task;
the step S2 specifically includes:
aiming at each task to be calculated in the total task scheduling of the system, when the task to be calculated is one task, judging whether the task is a calculation type larger than a preset scale, and acquiring a judgment result of the task;
Aiming at each task to be calculated in the total task scheduling of the system, when the task to be calculated has a plurality of tasks, sequentially judging whether each task in the tasks to be calculated is a calculation type larger than a preset scale according to a preset first sequence, and acquiring a judgment result of each task;
The step S3 includes:
If the judging result of the task is a calculation type larger than a preset scale, matching the task according to the hardware and software environment information required in the task and the hardware and software environment information provided by each computer cluster in the network, determining the computer cluster matched with the task, and calculating the task by the computer cluster matched with the task to obtain the processing result of the task;
If the judgment result of the task is not a calculation type larger than a preset scale, matching the task according to the hardware and software environment information required in the task and the hardware and software environment information provided by each personal calculation resource in the network, determining the personal calculation resource matched with the task, and calculating the task by the personal calculation resource matched with the task to obtain the processing result of the task;
The information of the hardware and software environment provided by the computer cluster comprises information provided by the computer cluster: the method comprises the steps of processing CPU core number, main frequency of the processor CPU, memory RAM space, storage space, network communication information and information of software tools installed in a computer cluster;
the information of the hardware and software environment provided by the personal computing resource comprises information provided by the personal computing resource: processor CPU core number, processor CPU main frequency, memory RAM space, storage space, network communication information and information of installed software tools of personal computing resources;
The step S3 includes:
If the judging result of the task is a calculation type larger than a preset scale, matching the task according to the hardware and software environment information required in the task and the hardware and software environment information provided by each computer cluster in the network, and determining the computer cluster matched with the task;
Submitting the task to a computer cluster matched with the task; the computer cluster matched with the task receives the task, and performs task scheduling according to a preset rule aiming at the task and the pre-received task to obtain a local task scheduling of the computer cluster;
the local task scheduling of the computer cluster comprises the tasks and the tasks which are received in advance and are arranged according to a preset second sequence;
According to the local task scheduling of the computer cluster, the computer cluster drives hardware and software corresponding to the computer cluster to calculate the task and the task received in advance, and a processing result of each task in the local task scheduling of the computer cluster is obtained;
the processing result of the task comprises: calculating a task or error information of the task;
In the step S3, the task is matched according to the hardware and software environment information required in the task and the hardware and software environment information provided by each computer cluster in the network, and the computer cluster matched with the task is determined, which specifically includes:
acquiring a first computer cluster according to the hardware and software environment information required in the task and the hardware and software environment information provided by each computer cluster in the network;
the first computer cluster is a computer cluster in which the hardware and software environment information provided in the network meets the hardware and software environment information required in the task;
When the number of the first computer clusters is one, determining the first computer clusters as computer clusters matched with the task;
When the number of the first computer clusters is a plurality of, determining the computer clusters matched with the task according to the hardware environment information provided by the first computer clusters;
the hardware environment information provided by the first computer cluster includes that provided by the first computer cluster: processor CPU core number, processor CPU main frequency, memory RAM space and storage space;
when the number of the first computer clusters is a plurality of, determining the computer clusters matched with the task according to the hardware environment information provided by the first computer clusters, specifically including:
When the number of the first computer clusters is a plurality of, acquiring the first computer cluster with the largest number of provided processor CPU cores;
when the number of the first computer clusters with the largest number of the provided processor CPU cores is one, determining the first computer clusters with the largest number of the provided processor CPU cores as the computer clusters matched with the task;
When the number of the first computer clusters with the largest number of the provided processor CPU cores is a plurality of, acquiring a second computer cluster with the highest main frequency of the provided processor CPU;
The second computer cluster is a first computer cluster with the largest number of CPU cores of the provided processor;
when the number of the second computer clusters with the highest main frequency of the provided processor CPU is one, determining the second computer cluster with the highest main frequency of the provided processor CPU as a computer cluster matched with the task;
when the number of the second computer clusters with the highest main frequency of the CPU is a plurality of, acquiring a third computer cluster with the largest memory RAM space;
the third computer cluster is a second computer cluster with highest main frequency for providing a CPU of the processor;
when the number of the third computer clusters with the largest memory RAM space is one, determining the third computer cluster with the largest memory RAM space as the computer cluster matched with the task;
When the number of the third computer clusters with the largest memory RAM space is a plurality of, acquiring a fourth computer cluster with the largest memory space;
the fourth computer cluster is a third computer cluster with the largest memory RAM space;
When the number of the fourth computer clusters with the largest provided storage space is one, determining the fourth computer cluster with the largest provided storage space as a computer cluster matched with the task;
When the number of the fourth computer clusters with the largest provided storage space is a plurality of, determining one fourth computer cluster with the largest provided storage space from the fourth computer clusters with the largest provided storage space as a computer cluster matched with the task;
The step S3 includes:
If the judgment result of the task is not a calculation type larger than a preset scale, matching the task according to the hardware and software environment information required in the task and the hardware and software environment information provided by each personal calculation resource in the network, and determining the personal calculation resource matched with the task;
submitting the task to a personal computing resource matched with the task; the personal computing resource matched with the task receives the task, and performs task scheduling according to preset rules aiming at the task and the pre-received task to obtain local task scheduling of the personal computing resource;
The local task scheduling of the personal computing resource comprises the tasks and the tasks which are received in advance and arranged according to a preset third sequence;
According to the local task scheduling of the personal computing resource, the personal computing resource drives hardware and software corresponding to the personal computing resource to calculate the task and the task received in advance, and a processing result of each task in the local task scheduling of the personal computing resource is obtained;
the processing result of the task comprises: calculating a task or error information of the task;
In the step S3, the task is matched according to the hardware and software environment information required in the task and the hardware and software environment information provided by each personal computing resource in the network, and the personal computing resource matched with the task is determined, which specifically includes:
Acquiring a first personal computing resource according to the hardware and software environment information required in the task and the hardware and software environment information provided by each personal computing resource in the network;
the first personal computing resource is a personal computing resource in a network which meets the hardware and software environment information required in the task;
when the number of the first personal computing resources is one, determining the first personal computing resources to be personal computing resources matched with the task;
when the number of the first personal computing resources is a plurality of, determining the personal computing resources matched with the task according to the hardware environment information provided by the first personal computing resources;
the hardware environment information provided by the first personal computing resource includes that provided by the first personal computing resource: processor CPU core number, processor CPU main frequency, memory RAM space, storage space.
2. The method according to claim 1, wherein when the number of the first personal computing resources is plural, determining the personal computing resources matching the task according to the hardware environment information provided by the first personal computing resources specifically includes:
when the number of the first personal computing resources is a plurality of, acquiring the first personal computing resource with the largest processor CPU core number;
When the number of the first personal computing resources with the largest number of the provided processor CPU cores is one, determining the first personal computing resources with the largest number of the provided processor CPU cores as the personal computing resources matched with the task;
When the number of the first personal computing resources with the largest number of available processor CPU cores is a plurality of, acquiring a second personal computing resource with the highest main frequency of the processor CPU;
The second personal computing resource is a first personal computing resource with the maximum number of CPU cores of the processor;
when the number of the second personal computing resources with the highest main frequency of the CPU provided by the processor is one, determining the second personal computing resources with the highest main frequency of the CPU provided by the processor as the personal computing resources matched with the task;
when the number of the second personal computing resources with the highest main frequency of the CPU of the providing processor is a plurality of, acquiring a third personal computing resource with the largest memory RAM space;
The third personal computing resource is a second personal computing resource with highest main frequency for providing a CPU of the processor;
when the number of the third personal computing resources with the largest memory RAM space is one, determining the third personal computing resources with the largest memory RAM space as the personal computing resources matched with the task;
When the number of the third personal computing resources with the largest available memory RAM space is a plurality of, acquiring a fourth personal computing resource with the largest storage space;
the fourth personal computing resource is a third personal computing resource with the largest memory RAM space;
When the number of the fourth personal computing resources with the largest provided storage space is one, determining the fourth personal computing resources with the largest provided storage space as personal computing resources matched with the task;
When the number of the fourth personal computing resources with the largest provided storage space is a plurality, determining one fourth personal computing resource with the largest provided storage space as the personal computing resource matched with the task according to the fourth personal computing resources with the largest provided storage space.
3. A system for scheduling network computing resources, comprising:
At least one processor; and
At least one memory communicatively coupled to the processor, wherein the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the method of network computing resource scheduling of any of claims 1-2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010833765.9A CN112068954B (en) | 2020-08-18 | 2020-08-18 | Method and system for scheduling network computing resources |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010833765.9A CN112068954B (en) | 2020-08-18 | 2020-08-18 | Method and system for scheduling network computing resources |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112068954A CN112068954A (en) | 2020-12-11 |
| CN112068954B true CN112068954B (en) | 2024-08-16 |
Family
ID=73662059
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010833765.9A Active CN112068954B (en) | 2020-08-18 | 2020-08-18 | Method and system for scheduling network computing resources |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112068954B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107450982A (en) * | 2017-06-07 | 2017-12-08 | 上海交通大学 | A kind of method for scheduling task based on system mode |
| CN110362392A (en) * | 2019-07-15 | 2019-10-22 | 深圳乐信软件技术有限公司 | A kind of ETL method for scheduling task, system, equipment and storage medium |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101169725A (en) * | 2006-10-23 | 2008-04-30 | 国际商业机器公司 | Personal computer supply system on demand and method |
| US9229781B2 (en) * | 2007-06-12 | 2016-01-05 | Broadcom Corporation | System and method for allocating spare system resources |
| CN101819540B (en) * | 2009-02-27 | 2013-03-20 | 国际商业机器公司 | Method and system for scheduling task in cluster |
| CN103544064B (en) * | 2013-10-28 | 2018-03-13 | 华为数字技术（苏州）有限公司 | Cloud computing method, cloud management platform and client |
| CN104461740B (en) * | 2014-12-12 | 2018-03-20 | 国家电网公司 | A kind of cross-domain PC cluster resource polymerization and the method for distribution |
| US10387179B1 (en) * | 2014-12-16 | 2019-08-20 | Amazon Technologies, Inc. | Environment aware scheduling |
| CN107942958B (en) * | 2017-10-13 | 2019-06-14 | 北京邮电大学 | A kind of industrial control system and method for internet of things oriented |
| CN108509276B (en) * | 2018-03-30 | 2021-11-30 | 南京工业大学 | Video task dynamic migration method in edge computing environment |
| CN109634748A (en) * | 2018-12-12 | 2019-04-16 | 深圳前海微众银行股份有限公司 | Cluster resource dispatching method, device, equipment and computer readable storage medium |
| CN109688222B (en) * | 2018-12-26 | 2020-12-25 | 深圳市网心科技有限公司 | Shared computing resource scheduling method, shared computing system, server and storage medium |
| CN111176846B (en) * | 2019-12-30 | 2023-06-13 | 云知声智能科技股份有限公司 | Task allocation method and device |
-
2020
- 2020-08-18 CN CN202010833765.9A patent/CN112068954B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107450982A (en) * | 2017-06-07 | 2017-12-08 | 上海交通大学 | A kind of method for scheduling task based on system mode |
| CN110362392A (en) * | 2019-07-15 | 2019-10-22 | 深圳乐信软件技术有限公司 | A kind of ETL method for scheduling task, system, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112068954A (en) | 2020-12-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9996593B1 (en) | Parallel processing framework | |
| US20150195344A1 (en) | Method and System for a Scheduled Map Executor | |
| Kaleeswaran et al. | Dynamic scheduling of data using genetic algorithm in cloud computing | |
| CN109614227A (en) | Task resource concocting method, device, electronic equipment and computer-readable medium | |
| CN113760488A (en) | Method, device, equipment and computer readable medium for scheduling task | |
| Robertsen et al. | Meta-pipe-pipeline annotation, analysis and visualization of marine metagenomic sequence data | |
| Draghmi et al. | An Improved dynamic Load Balancing Algorithm applied to a Cafeteria System in a University Campus | |
| CN114816730A (en) | Robot process automation cloud service system and implementation method | |
| CN116820714A (en) | Scheduling method, device, equipment and storage medium of computing equipment | |
| CN112068954B (en) | Method and system for scheduling network computing resources | |
| Gabaldon et al. | Multi-criteria genetic algorithm applied to scheduling in multi-cluster environments | |
| CN108537006A (en) | A kind of gene sequence data processing method, apparatus and system | |
| CN112463334A (en) | Training task queuing reason analysis method, system, equipment and medium | |
| Luo et al. | Hierarchical MapReduce: towards simplified cross‐domain data processing | |
| Yang et al. | G‐BLAST: a Grid‐based solution for mpiBLAST on computational Grids | |
| CN111159355A (en) | Customer complaint order processing method and device | |
| Giersch et al. | Scheduling tasks sharing files on heterogeneous clusters | |
| Abbasi et al. | SWSA: A Hybrid Scientific Workflow Scheduling Algorithm Based on Metaheuristic Approach in Cloud Computing Environment | |
| CN107247621B (en) | Scheduling method and device for batch processing tasks | |
| CN112711588A (en) | Multi-table connection method and device | |
| Kumar et al. | Grid Computing Model to Solve Job Shop Scheduling and Flow Shop Scheduling by Fuzzy C-Mean Algorithm | |
| Kang et al. | A hybrid genetic scheduling algorithm to heterogeneous distributed system | |
| Liu et al. | A two-stage AIS approach for grid scheduling problems | |
| Chen et al. | Scheduling of job combination and dispatching strategy for grid and cloud system | |
| Singh | Genetic-variable neighborhood search with thread replication for mobile cloud computing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |