CN108446176B - Task allocation method, computer readable storage medium and terminal device - Google Patents
Task allocation method, computer readable storage medium and terminal device Download PDFInfo
- Publication number
- CN108446176B CN108446176B CN201810121441.5A CN201810121441A CN108446176B CN 108446176 B CN108446176 B CN 108446176B CN 201810121441 A CN201810121441 A CN 201810121441A CN 108446176 B CN108446176 B CN 108446176B
- Authority
- CN
- China
- Prior art keywords
- task
- task processing
- processing server
- utilization rate
- current
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5044—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering hardware capabilities
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer And Data Communications (AREA)
- Hardware Redundancy (AREA)
Abstract
The invention belongs to the technical field of computers, and particularly relates to a task allocation method, a computer-readable storage medium and a terminal device. The method comprises the steps of obtaining a current target task to be processed; respectively acquiring the current CPU utilization rate and the memory utilization rate of each task processing server; respectively calculating the current comprehensive utilization rate of each task processing server according to the current CPU utilization rate and memory utilization rate of each task processing server and the prestored historical CPU utilization rate and memory utilization rate; respectively acquiring historical task processing records of each task processing server in a preset statistical time period; respectively calculating the reliability of each task processing server according to the historical task processing records; respectively calculating the task allocation ratio of each task processing server according to the comprehensive utilization rate and the reliability; and distributing the target tasks to be processed to each task processing server according to the task distribution ratio.
Description
Technical Field
The invention belongs to the technical field of computers, and particularly relates to a task allocation method, a computer-readable storage medium and a terminal device.
Background
When a large number of tasks need to be processed, the requirement often cannot be met by only using one server. At present, many systems adopt a multi-thread processing mode, and distribute tasks waiting for processing evenly to a plurality of servers for parallel processing. However, the performance difference between the servers is often very large, the way of even distribution may cause imbalance of server loads, the task processing pressure may be concentrated on some servers, and other servers are in an idle state, so the comprehensive efficiency is low.
Disclosure of Invention
In view of this, embodiments of the present invention provide a task allocation method, a computer-readable storage medium, and a terminal device, so as to solve the problem in the prior art that the overall efficiency is low due to the fact that tasks waiting for processing are evenly distributed to multiple servers for parallel processing.
A first aspect of an embodiment of the present invention provides a task allocation method, which may include:
acquiring a current target task to be processed;
respectively acquiring the current CPU utilization rate and the memory utilization rate of each task processing server;
respectively calculating the current comprehensive utilization rate of each task processing server according to the current CPU utilization rate and memory utilization rate of each task processing server and the prestored historical CPU utilization rate and memory utilization rate;
respectively acquiring historical task processing records of each task processing server in a preset statistical time period;
respectively calculating the reliability of each task processing server according to the historical task processing records;
respectively calculating the task allocation ratio of each task processing server according to the comprehensive utilization rate and the reliability;
and distributing the target tasks to be processed to each task processing server according to the task distribution ratio, wherein the number of the target tasks distributed to the task processing servers is positively correlated with the task distribution ratio.
A second aspect of embodiments of the present invention provides a computer-readable storage medium storing computer-readable instructions, which when executed by a processor implement the steps of:
acquiring a current target task to be processed;
respectively acquiring the current CPU utilization rate and the memory utilization rate of each task processing server;
respectively calculating the current comprehensive utilization rate of each task processing server according to the current CPU utilization rate and memory utilization rate of each task processing server and the prestored historical CPU utilization rate and memory utilization rate;
respectively acquiring historical task processing records of each task processing server in a preset statistical time period;
respectively calculating the reliability of each task processing server according to the historical task processing records;
respectively calculating the task allocation ratio of each task processing server according to the comprehensive utilization rate and the reliability;
and distributing the target tasks to be processed to each task processing server according to the task distribution ratio, wherein the number of the target tasks distributed to the task processing servers is positively correlated with the task distribution ratio.
A third aspect of the embodiments of the present invention provides a task allocation terminal device, including a memory, a processor, and computer readable instructions stored in the memory and executable on the processor, where the processor executes the computer readable instructions to implement the following steps:
acquiring a current target task to be processed;
respectively acquiring the current CPU utilization rate and the memory utilization rate of each task processing server;
respectively calculating the current comprehensive utilization rate of each task processing server according to the current CPU utilization rate and memory utilization rate of each task processing server and the prestored historical CPU utilization rate and memory utilization rate;
respectively acquiring historical task processing records of each task processing server in a preset statistical time period;
respectively calculating the reliability of each task processing server according to the historical task processing records;
respectively calculating the task allocation ratio of each task processing server according to the comprehensive utilization rate and the reliability;
and distributing the target tasks to be processed to each task processing server according to the task distribution ratio, wherein the number of the target tasks distributed to the task processing servers is positively correlated with the task distribution ratio.
Compared with the prior art, the embodiment of the invention has the following beneficial effects: the embodiment of the invention obtains a current target task to be processed; respectively acquiring the current CPU utilization rate and the memory utilization rate of each task processing server; respectively calculating the current comprehensive utilization rate of each task processing server according to the current CPU utilization rate and memory utilization rate of each task processing server and the prestored historical CPU utilization rate and memory utilization rate; respectively acquiring historical task processing records of each task processing server in a preset statistical time period; respectively calculating the reliability of each task processing server according to the historical task processing records; respectively calculating the task allocation ratio of each task processing server according to the comprehensive utilization rate and the reliability; and distributing the target tasks to be processed to each task processing server according to the task distribution ratio. According to the invention, the current comprehensive utilization rate of the task processing servers is calculated according to the CPU utilization rate and the memory utilization rate, the reliability of the task processing servers is calculated according to the historical task processing records, and the task allocation ratio of each task processing server is calculated according to the comprehensive utilization rate and the reliability respectively, and is related to the number of the target tasks allocated by the task processing servers, namely, the sufficient task allocation is combined with the processing capacity of the servers, so that more target tasks are allocated to the servers with strong processing capacity, less target tasks are allocated to the servers with weak processing capacity, and the comprehensive efficiency is improved to the maximum extent.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
FIG. 1 is a schematic diagram of an implementation environment in an embodiment of the invention;
FIG. 2 is a flowchart of an embodiment of a task allocation method according to an embodiment of the present invention;
FIG. 3 is a block diagram of an embodiment of a task assigning apparatus according to an embodiment of the present invention;
fig. 4 is a schematic block diagram of a task allocation terminal device in an embodiment of the present invention.
Detailed Description
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the embodiments described below are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, an execution subject in the embodiment of the present invention is a task allocation terminal device, which receives tasks sent from a specified task source and allocates the tasks to a plurality of task processing servers.
Referring to fig. 2, an embodiment of a task allocation method according to an embodiment of the present invention may include:
step S201, obtaining a current target task to be processed.
Preferably, in this embodiment, the acquired target tasks may be sequentially buffered in a preset FIFO queue for subsequent processing.
Step S202, respectively obtaining the current CPU utilization rate and the memory utilization rate of each task processing server;
the specific process of obtaining the current CPU utilization rate of a certain task processing server is as follows:
acquiring a file which is preset in the task processing server and records CPU information at a preset first sampling moment, and extracting CPU running state data at the first sampling moment from the file which records the CPU information; acquiring the file for recording the CPU information at a preset second sampling moment, wherein the second sampling moment is separated from the first sampling moment by a preset time length, and extracting CPU running state data at the second sampling moment from the file for recording the CPU information; and calculating the CPU utilization rate according to the CPU running state data at the first sampling moment and the CPU running state data at the second sampling moment.
For example, the CPU information recorded in the/proc/cpuinfo file of the task processing server may be acquired, and the following CPU running state data may be extracted:
user: the running time in the user state does not contain a process with a negative nice value;
nice: the nice value is the CPU time occupied by the negative process, the nice value is a numerical value representing the static priority in the system, each process has the own static priority, and the process with high priority can be preferentially operated;
system: run time in kernel mode;
idle: other latencies than IO latency;
iowait: IO latency;
irq: service interruption time;
softirq: soft interrupt time;
steelstolen: time spent on other operating systems while the virtual environment is running;
guest: time spent by the operating system virtual CPU;
guest _ nice: the operating system virtual CPU spends time on the nice process.
Successively sampling CPU running state data of two sufficiently short time intervals, which are respectively marked as t1 and t2, wherein the structures of t1 and t2 are 10 tuples (user, nice, system, idle, iowait, irq, softirq, steelsulen, gut and gut _ nice); summing all the 10-tuple data for the first time to obtain totalCPUTime 1; summing all the 10-tuple data of the second time to obtain totalc CPUTime 2; obtaining first-time idle data, which is marked as idle 1; obtaining a second-time idle data, denoted idle 2; the current CPU utilization λ of the task processing serverCPUCan be calculated by the following formula:
the specific process of obtaining the current memory utilization rate of a certain task processing server is as follows:
acquiring a preset file for recording memory information in the task processing server; extracting the total amount of the memory space and the available allowance of the memory space from the file for recording the memory information; and calculating the memory utilization rate according to the total amount of the memory space and the available allowance of the memory space.
For example, the memory information recorded in the/proc/meminfo file of the task processing server may be obtained, where the file contains much information, and if the memory usage rate is to be calculated, only two variable values of the file are needed, namely MemTotal memory space amount representing MemTotal memory space and MemFree representing available memory space remaining amount, and the current memory usage rate λ of the task processing serverMemoryCan be calculated by the following formula:
step S203, respectively calculating the current comprehensive utilization rate of each task processing server according to the current CPU utilization rate and memory utilization rate of each task processing server and the pre-stored historical CPU utilization rate and memory utilization rate.
Specifically, the CPU utilization is smoothed according to the following equation:
y(tn)＝k1*u(tn)+k2*u(tn-1)+k3*u(tn-2)，n≥3
y(tn)＝k′1*u(tn)+k′2*u(tn-1)，n＝2
y(tn)＝u(tn)，n＝1
wherein u (t)n) For the current CPU usage, u (t)n-1) Is u (t)n) The CPU usage rate at the previous sampling time of (d), u (t)n-2) Is u (t)n-1) The CPU utilization rate of the previous sampling time，y(tn) For CPU usage after smoothing, k1、k2、k3、k′1、k′2Is a predetermined proportionality coefficient, and k1+k2+k3＝1，k1>k2>k3，k′1+k′2＝1，k′1>k′2，k′1>k1，k′2>k2。
For example, if k is as follows1＝0.7、k2＝0.2、k3＝0.1、k′1＝0.75、k′2At t ═ 0.251The CPU utilization rate collected at that moment is 0.5, i.e., u (t)1) At t ═ 0.52The CPU utilization rate collected at that moment is 0.2, i.e., u (t)2) At t ═ 0.23The CPU utilization rate collected at that moment is 0.1, i.e., u (t)3) If the value is 0.1, the low-pass smoothing processing is carried out on the value, and the smooth value of the CPU utilization rate is obtained as follows:
y(t1)＝u(t1)＝0.5
y(t2)＝k′1*u(t2)+k′2*u(t1)＝0.75*0.2+0.25*0.5＝0.275
y(t3)＝k1*u(t3)+k2*u(t2)+k3*u(t1)＝0.7*0.1+0.2*0.2+0.1*0.5＝0.16
the memory usage is smoothed according to the following equation:
x(tn)＝k1*v(tn)+k2*v(tn-1)+k3*v(tn-2)，n≥3
x(tn)＝k′1*v(tn)+k′2*v(tn-1)，n＝2
x(tn)＝v(tn)，n＝1
wherein, v (t)n) For the current memory usage, v (t)n-1) Is v (t)n) The memory usage at the previous sampling instant of, v (t)n-2) Is v (t)n-1) The memory usage at the previous sampling instant of, x (t)n) After being smoothedThe memory usage rate;
calculating the current integrated utilization according to the following formula:
z(tn)＝k″1*x(tn)+k″2*y(tn)，n≥1
wherein, z (t)n) Is the current comprehensive utilization rate, k ″)1、k″2Is a predetermined proportionality coefficient, and k ″)1+k″2＝1。
And step S204, respectively acquiring historical task processing records of each task processing server in a preset statistical time period.
The statistical time period may be set according to actual conditions, for example, historical task processing records of the previous month, the previous week or the previous day may be taken for statistics.
And step S205, respectively calculating the reliability of each task processing server according to the historical task processing records.
Specifically, the duration that the processing state of each task processing server is normal and the duration that the processing state is abnormal are respectively counted according to the historical task processing records, wherein the condition that the processing state is abnormal refers to the condition that the task processing server sends abnormal feedback or has no response after time out.
Respectively calculating the reliability of each task processing server according to the time length of each task processing server with the normal processing state and the time length of each task processing server with the abnormal processing state, wherein the calculation formula is as follows:
where η is the reliability, h is the total duration of the processing state being normal, h' is the total duration of the processing state being abnormal, m1M is the total number of segments of the time period in which the processing state is normal2For the total number of segments of the time period in which the processing state is abnormal, hnIs the time length of the time period, h ', of which the nth section of processing state is normal'nThe length of the time period in which the nth processing state is abnormal.
And S206, respectively calculating the task allocation ratio of each task processing server according to the comprehensive utilization rate and the reliability.
Specifically, the task allocation ratio of each task processing server is calculated according to the following formula:
wherein λ ispAssigning a ratio, z, to tasks of the pth task processing serverpComprehensive utilization of servers for the pth task processing, ηpThe reliability of the pth task processing server, M is the total number of task processing servers, l1、l2Is a predetermined proportionality coefficient, and1+l2＝1。
step S207, the target tasks to be processed are distributed to each of the task processing servers according to the task distribution ratio, and the number of the target tasks distributed by the task processing servers is positively correlated with the task distribution ratio.
The target tasks to be processed can be divided into M task groups according to the task allocation ratio according to the data characteristics of the target tasks to be processed, such as contract number tail number, application number tail number, and client number tail number, and the task groups respectively correspond to M task processing servers.
For example, if there are 4 task processing servers, the task allocation ratios are 0.1, 0.2, 0.3, and 0.4, respectively. The target task to be processed can be divided into 4 task groups according to the contract number and the tail number, wherein the tail number is 0 to form the task group 1, the tail numbers are 1 and 2 to form the task group 2, the tail numbers are 3, 4 and 5 to form the task group 3, and the tail numbers are 6, 7, 8 and 9 to form the task group 4. And then distributing each task group to a corresponding task processing server for processing, namely distributing the task group 1 to the task processing server 1, distributing the task group 2 to the task processing server 2, distributing the task group 3 to the task processing server 3, and distributing the task group 4 to the task processing server 4.
Further, the task allocation method further includes acquiring a current processing state of each task processing server in real time, and if a task processing server with an abnormal current processing state exists, transferring the task which is not processed by the task processing server with the abnormal current processing state to the task processing server with a normal current processing state for continuous processing.
In summary, the embodiment of the present invention obtains the current target task to be processed; respectively acquiring the current CPU utilization rate and the memory utilization rate of each task processing server; respectively calculating the current comprehensive utilization rate of each task processing server according to the current CPU utilization rate and memory utilization rate of each task processing server and the prestored historical CPU utilization rate and memory utilization rate; respectively acquiring historical task processing records of each task processing server in a preset statistical time period; respectively calculating the reliability of each task processing server according to the historical task processing records; respectively calculating the task allocation ratio of each task processing server according to the comprehensive utilization rate and the reliability; and distributing the target tasks to be processed to each task processing server according to the task distribution ratio. According to the invention, the current comprehensive utilization rate of the task processing servers is calculated according to the CPU utilization rate and the memory utilization rate, the reliability of the task processing servers is calculated according to the historical task processing records, and the task allocation ratio of each task processing server is calculated according to the comprehensive utilization rate and the reliability respectively, and is related to the number of the target tasks allocated by the task processing servers, namely, the sufficient task allocation is combined with the processing capacity of the servers, so that more target tasks are allocated to the servers with strong processing capacity, less target tasks are allocated to the servers with weak processing capacity, and the comprehensive efficiency is improved to the maximum extent.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
Fig. 3 is a structural diagram of an embodiment of a task allocation apparatus according to an embodiment of the present invention, which corresponds to a task allocation method described in the foregoing embodiment.
In this embodiment, a task allocation apparatus may include:
a target task obtaining module 301, configured to obtain a current target task to be processed;
a utilization index obtaining module 302, configured to obtain a current CPU utilization and a current memory utilization of each task processing server respectively;
a comprehensive utilization rate calculation module 303, configured to calculate, according to the current CPU utilization rate and memory utilization rate of each task processing server and a pre-stored historical CPU utilization rate and memory utilization rate, the current comprehensive utilization rate of each task processing server respectively;
a history record obtaining module 304, configured to obtain history task processing records of each task processing server within a preset statistical time period;
a reliability calculation module 305, configured to calculate reliability of each task processing server according to the historical task processing records;
a distribution ratio calculation module 306, configured to calculate a task distribution ratio of each task processing server according to the comprehensive utilization rate and the reliability;
and a task allocation module 307, configured to allocate the target tasks to be processed to each of the task processing servers according to the task allocation ratio, where the number of the target tasks allocated by the task processing servers is positively correlated to the task allocation ratio.
Further, the comprehensive utilization rate calculation module may include:
a first smoothing unit for smoothing the CPU usage according to the following formula:
y(tn)＝k1*u(tn)+k2*u(tn-1)+k3*u(tn-2)，n≥3
y(tn)＝k′1*u(tn)+k′2*u(tn-1)，n＝2
y(tn)＝u(tn)，n＝1
wherein u (t)n) For the current CPU usage, u (t)n-1) Is u (t)n) The CPU usage rate at the previous sampling time of (d), u (t)n-2) Is u (t)n-1) The CPU usage at the previous sampling instant of, y (t)n) For CPU usage after smoothing, k1、k2、k3、k′1、k′2Is a predetermined proportionality coefficient, and k1+k2+k3＝1，k1>k2>k3，k′1+k′2＝1，k′1>k′2，k′1>k1，k′2>k2；
A second smoothing unit configured to smooth the memory usage according to:
x(tn)＝k1*v(tn)+k2*v(tn-1)+k3*v(tn-2)，n≥3
x(tn)＝k′1*v(tn)+k′2*v(tn-1)，n＝2
x(tn)＝v(tn)，n＝1
wherein, v (t)n) For the current memory usage, v (t)n-1) Is v (t)n) The memory usage at the previous sampling instant of, v (t)n-2) Is v (t)n-1) The memory usage at the previous sampling instant of, x (t)n) The memory usage rate after smoothing processing;
a comprehensive utilization rate calculation unit for calculating a current comprehensive utilization rate according to the following formula:
z(tn)＝k″1*x(tn)+k″2*y(tn)，n≥1
wherein, z (t)n) Is the current comprehensive utilization rate, k ″)1、k″2Is a predetermined proportionality coefficient, and k ″)1+k″2＝1。
Further, the reliability calculation module may include:
the duration counting unit is used for respectively counting the duration of which the processing state of each task processing server is normal and the duration of which the processing state is abnormal according to the historical task processing records, wherein the processing state is abnormal, which means that the task processing servers send abnormal feedback or overtime no response condition;
the reliability calculation unit is used for calculating the reliability of each task processing server according to the time length when the processing state of each task processing server is normal and the time length when the processing state of each task processing server is abnormal, and the calculation formula is as follows:
where η is the reliability, h is the total duration of the processing state being normal, h' is the total duration of the processing state being abnormal, m1M is the total number of segments of the time period in which the processing state is normal2For the total number of segments of the time period in which the processing state is abnormal, hnIs the duration of the time period in which the nth processing state is normal, hn' is the duration of the period of time during which the nth processing state is abnormal.
Further, the allocation ratio calculation module may include:
an allocation ratio calculation unit, configured to calculate a task allocation ratio of each of the task processing servers according to the following formula:
wherein λ ispAssigning a ratio, z, to tasks of the pth task processing serverpComprehensive utilization of servers for the pth task processing, ηpThe reliability of the pth task processing server, M is the total number of task processing servers, l1、l2Is a predetermined proportionality coefficient, and1+l2＝1。
further, the task allocation method may further include:
the processing state acquisition module is used for acquiring the current processing state of each task processing server;
and the task transferring module is used for transferring the task which is not processed by the task processing server with the abnormal current processing state to the task processing server with the normal current processing state for continuous processing if the task processing server with the abnormal current processing state exists.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses, modules and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Fig. 4 shows a schematic block diagram of a task allocation terminal device according to an embodiment of the present invention, and for convenience of description, only the parts related to the embodiment of the present invention are shown.
In this embodiment, the task distribution terminal device 4 may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The task distribution terminal device 4 may include: a processor 40, a memory 41, and computer readable instructions 42 stored in the memory 41 and executable on the processor 40, such as computer readable instructions to perform the task assignment method described above. The processor 40, when executing the computer readable instructions 42, implements the steps in the various task allocation method embodiments described above, such as steps S201-S207 shown in fig. 2. Alternatively, the processor 40, when executing the computer readable instructions 42, implements the functions of the modules/units in the above device embodiments, such as the functions of the modules 301 to 307 shown in fig. 3.
Illustratively, the computer readable instructions 42 may be partitioned into one or more modules/units that are stored in the memory 41 and executed by the processor 40 to implement the present invention. The one or more modules/units may be a series of computer-readable instruction segments capable of performing specific functions, which are used for describing the execution process of the computer-readable instructions 42 in the task distribution terminal device 4.
The Processor 40 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The storage 41 may be an internal storage unit of the task allocation terminal 4, such as a hard disk or a memory of the task allocation terminal 4. The memory 41 may also be an external storage device of the task distribution terminal device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are equipped on the task distribution terminal device 4. Further, the memory 41 may also include both an internal storage unit and an external storage device of the task allocation terminal device 4. The memory 41 is used for storing the computer readable instructions and other instructions and data required by the task distribution terminal 4. The memory 41 may also be used to temporarily store data that has been output or is to be output.
Each functional unit in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of computer readable instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and the like, which can store computer readable instructions.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (4)
1. A task allocation method, comprising:
acquiring a current target task to be processed;
respectively acquiring the current CPU utilization rate and the memory utilization rate of each task processing server;
respectively calculating the current comprehensive utilization rate of each task processing server according to the current CPU utilization rate and memory utilization rate of each task processing server and the prestored historical CPU utilization rate and memory utilization rate;
respectively acquiring historical task processing records of each task processing server in a preset statistical time period;
respectively calculating the reliability of each task processing server according to the historical task processing records;
respectively calculating the task allocation ratio of each task processing server according to the comprehensive utilization rate and the reliability;
distributing the target tasks to be processed to each task processing server according to the task distribution ratio, wherein the number of the target tasks distributed to each task processing server is positively correlated with the task distribution ratio;
the step of respectively calculating the current comprehensive utilization rate of each task processing server according to the current CPU utilization rate and memory utilization rate of each task processing server and the pre-stored historical CPU utilization rate and memory utilization rate comprises the following steps:
smoothing the CPU utilization according to the following equation:
y(tn)＝k1*u(tn)+k2*u(tn-1)+k3*u(tn-2)，n≥3
y(tn)＝k′1*u(tn)+k′2*u(tn-1)，n＝2
y(tn)＝u(tn)，n＝1
wherein u (t)n) For the current CPU usage, u (t)n-1) Is u (t)n) The CPU usage rate at the previous sampling time of (d), u (t)n-2) Is u (t)n-1) The CPU usage at the previous sampling instant of, y (t)n) For CPU usage after smoothing, k1、k2、k3、k′1、k′2Is a predetermined proportionality coefficient, and k1+k2+k3＝1，k1>k2>k3，k′1+k′2＝1，k′1>k′2，k′1>k1，k′2>k2；
The memory usage is smoothed according to the following equation:
x(tn)＝k1*v(tn)+k2*v(tn-1)+k3*v(tn-2)，n≥3
x(tn)＝k′1*v(tn)+k′2*v(tn-1)，n＝2
x(tn)＝v(tn)，n＝1
wherein, v (t)n) For the current memory usage, v (t)n-1) Is v (t)n) The memory usage at the previous sampling instant of, v (t)n-2) Is v (t)n-1) The memory usage at the previous sampling instant of, x (t)n) The memory usage rate after smoothing processing;
calculating the current integrated utilization according to the following formula:
z(tn)＝k″1*x(tn)+k″2*y(tn)，n≥1
wherein, z (t)n) Is the current comprehensive utilization rate, k ″)1、k″2Is a predetermined proportionality coefficient, and k ″)1+k″2＝1；
The respectively calculating the reliability of each task processing server according to the historical task processing records comprises:
respectively counting the time length of each task processing server with a normal processing state and the time length of each task processing server with an abnormal processing state according to the historical task processing records, wherein the abnormal processing state refers to the condition that the task processing servers send abnormal feedback or overtime and no response exists;
respectively calculating the reliability of each task processing server according to the time length of each task processing server with the normal processing state and the time length of each task processing server with the abnormal processing state, wherein the calculation formula is as follows:
where η is the reliability, h is the total duration of the processing state being normal, h' is the total duration of the processing state being abnormal, m1M is the total number of segments of the time period in which the processing state is normal2For the total number of segments of the time period in which the processing state is abnormal, hnIs the duration of the time period in which the nth processing state is normal, hnIs the nth stage treatmentThe duration of the time period in which the state is abnormal;
the respectively calculating the task allocation ratio of each task processing server according to the comprehensive utilization rate and the reliability includes:
calculating the task allocation ratio of each task processing server according to the following formula:
wherein λ ispAssigning a ratio, z, to tasks of the pth task processing serverpComprehensive utilization of servers for the pth task processing, ηpThe reliability of the pth task processing server, M is the total number of task processing servers, l1、l2Is a predetermined proportionality coefficient, and1+l2＝1。
2. the task allocation method according to claim 1, further comprising:
acquiring the current processing state of each task processing server;
if the task processing server with the abnormal current processing state exists, transferring the task which is not processed by the task processing server with the abnormal current processing state to the task processing server with the normal current processing state for continuous processing.
3. A computer readable storage medium storing computer readable instructions, which when executed by a processor implement the steps of the task assignment method of any one of claims 1 to 2.
4. A task distribution terminal device comprising a memory, a processor, and computer readable instructions stored in the memory and executable on the processor, wherein the processor when executing the computer readable instructions performs the steps of:
acquiring a current target task to be processed;
respectively acquiring the current CPU utilization rate and the memory utilization rate of each task processing server;
respectively calculating the current comprehensive utilization rate of each task processing server according to the current CPU utilization rate and memory utilization rate of each task processing server and the prestored historical CPU utilization rate and memory utilization rate;
respectively acquiring historical task processing records of each task processing server in a preset statistical time period;
respectively calculating the reliability of each task processing server according to the historical task processing records;
respectively calculating the task allocation ratio of each task processing server according to the comprehensive utilization rate and the reliability;
distributing the target tasks to be processed to each task processing server according to the task distribution ratio, wherein the number of the target tasks distributed to each task processing server is positively correlated with the task distribution ratio;
the step of respectively calculating the current comprehensive utilization rate of each task processing server according to the current CPU utilization rate and memory utilization rate of each task processing server and the pre-stored historical CPU utilization rate and memory utilization rate comprises the following steps:
smoothing the CPU utilization according to the following equation:
y(tn)＝k1*u(tn)+k2*u(tn-1)+k3*u(tn-2)，n≥3
y(tn)＝k′1*u(tn)+k′2*u(tn-1)，n＝2
y(tn)＝u(tn)，n＝1
wherein u (t)n) For the current CPU usage, u (t)n-1) Is u (t)n) The CPU usage rate at the previous sampling time of (d), u (t)n-2) Is u (t)n-1) The CPU usage at the previous sampling instant of, y (t)n) After being smoothedCPU utilization, k1、k2、k3、k′1、k′2Is a predetermined proportionality coefficient, and k1+k2+k3＝1，k1>k2>k3，k′1+k′2＝1，k′1>k′2，k′1>k1，k′2>k2；
The memory usage is smoothed according to the following equation:
x(tn)＝k1*v(tn)+k2*v(tn-1)+k3*v(tn-2)，n≥3
x(tn)＝k′1*v(tn)+k′2*v(tn-1)，n＝2
x(tn)＝v(tn)，n＝1
wherein, v (t)n) For the current memory usage, v (t)n-1) Is v (t)n) The memory usage at the previous sampling instant of, v (t)n-2) Is v (t)n-1) The memory usage at the previous sampling instant of, x (t)n) The memory usage rate after smoothing processing;
calculating the current integrated utilization according to the following formula:
z(tn)＝k″1*x(tn)+k″2*y(tn)，n≥1
wherein, z (t)n) Is the current comprehensive utilization rate, k ″)1、k″2Is a predetermined proportionality coefficient, and k ″)1+k″2＝1；
The respectively calculating the reliability of each task processing server according to the historical task processing records comprises:
respectively counting the time length of each task processing server with a normal processing state and the time length of each task processing server with an abnormal processing state according to the historical task processing records, wherein the abnormal processing state refers to the condition that the task processing servers send abnormal feedback or overtime and no response exists;
respectively calculating the reliability of each task processing server according to the time length of each task processing server with the normal processing state and the time length of each task processing server with the abnormal processing state, wherein the calculation formula is as follows:
where η is the reliability, h is the total duration of the processing state being normal, h' is the total duration of the processing state being abnormal, m1M is the total number of segments of the time period in which the processing state is normal2For the total number of segments of the time period in which the processing state is abnormal, hnIs the duration of the time period in which the nth processing state is normal, hn' is the duration of the time period in which the nth processing state is abnormal;
the respectively calculating the task allocation ratio of each task processing server according to the comprehensive utilization rate and the reliability includes:
calculating the task allocation ratio of each task processing server according to the following formula:
wherein λ ispAssigning a ratio, z, to tasks of the pth task processing serverpComprehensive utilization of servers for the pth task processing, ηpThe reliability of the pth task processing server, M is the total number of task processing servers, l1、l2Is a predetermined proportionality coefficient, and1+l2＝1。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810121441.5A CN108446176B (en) | 2018-02-07 | 2018-02-07 | Task allocation method, computer readable storage medium and terminal device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810121441.5A CN108446176B (en) | 2018-02-07 | 2018-02-07 | Task allocation method, computer readable storage medium and terminal device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108446176A CN108446176A (en) | 2018-08-24 |
| CN108446176B true CN108446176B (en) | 2020-07-03 |
Family
ID=63192021
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810121441.5A Active CN108446176B (en) | 2018-02-07 | 2018-02-07 | Task allocation method, computer readable storage medium and terminal device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108446176B (en) |
Families Citing this family (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109144837B (en) * | 2018-09-04 | 2021-04-27 | 南京大学 | User behavior pattern recognition method supporting accurate service push |
| CN109669773B (en) * | 2018-11-12 | 2024-03-08 | 平安科技（深圳）有限公司 | Financial data processing method, device, equipment and storage medium |
| CN109800148A (en) * | 2018-12-14 | 2019-05-24 | 平安普惠企业管理有限公司 | A kind of page adjustment method, device, readable storage medium storing program for executing and terminal device |
| CN111381956B (en) * | 2018-12-28 | 2024-02-27 | 杭州海康威视数字技术股份有限公司 | Task processing method and device and cloud analysis system |
| CN111949384B (en) * | 2019-05-14 | 2023-05-02 | 龙芯中科技术股份有限公司 | Task scheduling method, device, equipment and computer readable storage medium |
| CN110569124A (en) * | 2019-08-15 | 2019-12-13 | 中国平安财产保险股份有限公司 | Task allocation method and device |
| CN111176966A (en) * | 2019-12-26 | 2020-05-19 | 京信通信系统（中国）有限公司 | Method, device and equipment for determining CPU utilization rate and storage medium |
| CN111597085A (en) * | 2020-04-02 | 2020-08-28 | 北京邮电大学 | Computing resource occupation computing method and device |
| CN113742052B (en) * | 2020-05-29 | 2023-09-01 | 北京京东振世信息技术有限公司 | Batch task processing method and device |
| CN113807621B (en) * | 2020-06-12 | 2024-03-19 | 北京四维图新科技股份有限公司 | Data processing method, device and equipment |
| CN113918309A (en) * | 2020-12-31 | 2022-01-11 | 技象科技(浙江)有限公司 | Task queue maintenance method, device, system and medium based on waiting duration |
| CN112764934B (en) * | 2021-01-29 | 2024-01-30 | 上海弘积信息科技有限公司 | Calculation method for CPU utilization rate of load balancing equipment |
| CN113485838A (en) * | 2021-07-26 | 2021-10-08 | 北京沃东天骏信息技术有限公司 | Server distribution method and device, electronic equipment and computer readable storage medium |
| CN113625861B (en) * | 2021-08-27 | 2024-04-19 | 深圳供电局有限公司 | Power consumption saving method and system |
| CN114880120B (en) * | 2022-05-10 | 2024-07-23 | 马上消费金融股份有限公司 | Data processing method, device, equipment and storage medium |
| CN115695432B (en) * | 2023-01-04 | 2023-04-07 | 河北华通科技股份有限公司 | Load balancing method and device, electronic equipment and storage medium |
| CN117331291B (en) * | 2023-12-01 | 2024-03-19 | 上海红岩临芯半导体科技有限公司 | Scheduling method and system for unit balance flow sheet of glue spreading and developing device |
| CN117785485A (en) * | 2024-02-26 | 2024-03-29 | 浪潮计算机科技有限公司 | Cloud computing virtual machine task allocation method, device, equipment and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002259143A (en) * | 2001-03-05 | 2002-09-13 | Nec Soft Ltd | Device and method for automatically controlling number of task, on-like transaction processing system and program |
| CN102317917A (en) * | 2011-06-30 | 2012-01-11 | 华为技术有限公司 | Hot field virtual machine cpu dispatching method and virtual machine system (vms) |
| CN104572286A (en) * | 2015-01-30 | 2015-04-29 | 湖南蚁坊软件有限公司 | Task scheduling method based on distributed memory clusters |
| CN106155802A (en) * | 2015-03-30 | 2016-11-23 | 阿里巴巴集团控股有限公司 | Method for scheduling task, device and control node |
| CN107516158A (en) * | 2017-07-27 | 2017-12-26 | 深圳市佰仟金融服务有限公司 | A kind of method for allocating tasks, device and terminal device |
-
2018
- 2018-02-07 CN CN201810121441.5A patent/CN108446176B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002259143A (en) * | 2001-03-05 | 2002-09-13 | Nec Soft Ltd | Device and method for automatically controlling number of task, on-like transaction processing system and program |
| CN102317917A (en) * | 2011-06-30 | 2012-01-11 | 华为技术有限公司 | Hot field virtual machine cpu dispatching method and virtual machine system (vms) |
| CN104572286A (en) * | 2015-01-30 | 2015-04-29 | 湖南蚁坊软件有限公司 | Task scheduling method based on distributed memory clusters |
| CN106155802A (en) * | 2015-03-30 | 2016-11-23 | 阿里巴巴集团控股有限公司 | Method for scheduling task, device and control node |
| CN107516158A (en) * | 2017-07-27 | 2017-12-26 | 深圳市佰仟金融服务有限公司 | A kind of method for allocating tasks, device and terminal device |
Non-Patent Citations (1)
| Title |
|---|
| 基于节点性能的Hadoop作业调度算法改进;冯兴杰 等;《计算机应用与软件》;20170515;第34卷(第5期);第223-228页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108446176A (en) | 2018-08-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108446176B (en) | Task allocation method, computer readable storage medium and terminal device | |
| US8468251B1 (en) | Dynamic throttling of access to computing resources in multi-tenant systems | |
| US8782224B2 (en) | Systems and methods for time-based dynamic allocation of resource management | |
| US10037230B2 (en) | Managing data processing resources | |
| US20140280970A1 (en) | Systems and methods for time-based dynamic allocation of resource management | |
| US10255114B2 (en) | Abnormality detection apparatus, control method, and program | |
| US10884667B2 (en) | Storage controller and IO request processing method | |
| CN110196767A (en) | Service resource control method, device, equipment and storage medium | |
| CN108491122B (en) | Click event response method, computer readable storage medium and terminal equipment | |
| US20190138354A1 (en) | Method for scheduling jobs with idle resources | |
| CN107704322B (en) | Request distribution method and device | |
| US20150180936A1 (en) | Data transfer device, data transfer method, and program storage medium | |
| CN113495779A (en) | Task scheduling method and device and task execution system | |
| CN111930516B (en) | Load balancing method and related device | |
| CN105045667B (en) | A kind of resource pool management method for virtual machine vCPU scheduling | |
| CN111552547A (en) | Job processing method and device and computer equipment | |
| CN110865877A (en) | Task request response method and device | |
| CN107220166B (en) | A kind of statistical method and device of CPU usage | |
| CN111767188A (en) | Training task monitoring method and device | |
| CN117112222A (en) | Request processing method and device, electronic equipment and storage medium | |
| CN110073321B (en) | Storage controller and IO request processing method | |
| CN108429704B (en) | Node resource allocation method and device | |
| CN112130974B (en) | Cloud computing resource configuration method and device, electronic equipment and storage medium | |
| CN115114005A (en) | Service scheduling control method, device, equipment and computer readable storage medium | |
| CN110955644A (en) | IO control method, device, equipment and storage medium of storage system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |