CN104536811A - HIVE task based task scheduling method and device - Google Patents
HIVE task based task scheduling method and device Download PDFInfo
- Publication number
- CN104536811A CN104536811A CN201410831524.5A CN201410831524A CN104536811A CN 104536811 A CN104536811 A CN 104536811A CN 201410831524 A CN201410831524 A CN 201410831524A CN 104536811 A CN104536811 A CN 104536811A
- Authority
- CN
- China
- Prior art keywords
- task
- dependence
- dispatching
- scheduling
- needs
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a HIVE task based task scheduling method and device and belongs to the technical field of computers. The method comprises the steps that multiple tasks needing scheduling and the dependency between the multiple tasks are determined; a front dependent task and a rear dependent task of each task are determined according to the dependency; tasks currently needing scheduling are selected from all the task needing scheduling, and the selected tasks are obtained; the selected tasks are scheduled according to the scheduling results of the front dependent tasks of the selected tasks; the rear dependent tasks of the selected tasks and all tasks not scheduled are processed in the modes of selecting the selected tasks and scheduling the selected tasks until all the tasks needing scheduling are scheduled. The scheduling results of the front dependent tasks of the selected tasks are determined according to the dependency between the tasks to schedule the selected tasks, so that the task scheduling process is simple, and it can be ensured that the tasks can be executed orderly. The task scheduling efficiency is high.
Description
Technical field
The present invention relates to field of computer technology, particularly a kind of method for scheduling task based on HIVE task and device.
Background technology
Along with the fast development of present computer technology, in order to execute the task, server is not all needing to dispatch task, particularly in the scheduling field based on HIVE task in the same time.Wherein, HIVE is a Tool for Data Warehouse based on Hadoop, and Hadoop is a software architecture can carrying out distributed treatment to mass data, thus structurized data file can be mapped as a database table by HIVE task, and complete SQL (Structured Query Language is provided, Structured Query Language (SQL)) query function, the task that SQL statement can be converted to Map (mapping) and Reduce (abbreviation) is run.Follow-up work execution efficiency is had influence on because server carries out scheduling to task.Therefore, how carrying out task scheduling based on HIVE task is improve the key of tasks carrying efficiency.
Provide a kind of method for scheduling task based on HIVE task in correlation technique, in the method, pre-determine the task that a certain moment needs scheduling, if current time has arrived the moment of this task of scheduling, then this task is dispatched.After this task scheduling completes, need the next task of dispatching after determining this task scheduling by relevant staff according to the scheduling result of this task, and determine the scheduling instance of next task.Thus manually controlled to dispatch each task by relevant staff.
Realizing in process of the present invention, inventor finds that said method at least exists following problem:
Before each scheduler task, after needing to determine according to the scheduling result of the task of a upper scheduling task that next needs are dispatched by relevant staff, could dispatch task.When task is more time, then need manually to control each task, task scheduling process is more loaded down with trivial details.Therefore, the efficiency of scheduler task is lower.
Summary of the invention
In order to solve the problem of prior art, embodiments provide a kind of method for scheduling task based on HIVE task and device.Described technical scheme is as follows:
First aspect, provides a kind of method for scheduling task based on HIVE task, and described method comprises:
Determine the multiple tasks needing to carry out dispatching, and determine the dependence between each task;
The preposition dependence task of each task and rearmounted dependence task is determined according to described dependence;
Carry out at all needs in the task of dispatching, selecting the current task of needing scheduling, obtain selected task;
Scheduling result according to the preposition dependence task of described selected task is dispatched described selected task;
According to selecting the rearmounted dependence task of selected task described in the mode of described selected task and the mode process of the selected task of execution cost and all tasks of not dispatching, until dispatched the task that all needs carry out dispatching.
In conjunction with first aspect, in the first possible implementation of first aspect, described carrying out at all needs selects the current task of needing scheduling in the task of dispatching, comprising:
Task of selecting the task of meeting schedulable condition to dispatch as current needs in the task of dispatching is carried out at all needs; Wherein, described schedulable condition is that all preposition dependence task are all dispatched complete.
In conjunction with the first possible implementation of first aspect, in the implementation that the second of first aspect is possible, described task of carrying out selecting the task of meeting schedulable condition to dispatch as current needs in the task of dispatching at all needs, comprising:
If there is the multiple task of meeting schedulable condition, using the described multiple task of meeting schedulable condition all as the task of current needs scheduling, or, according to priority described multiple meet in the task of schedulable condition select the task task of dispatching as current needs.
In conjunction with first aspect, in the third possible implementation of first aspect, each needs carry out a dispatching cycle of task equal correspondence of dispatching; The scheduling result of the described preposition dependence task according to described selected task is dispatched described selected task, comprising:
The preposition dependence task obtaining described selected task, according to one or more scheduling result before obtaining after being scheduled the dispatching cycle of its correspondence, is dispatched described selected task according to the scheduling result got.
Any one possible implementation in the implementation possible in conjunction with first aspect, the first possible implementation of first aspect, the second and the third possible implementation, in the 4th kind of possible implementation of first aspect, describedly determine the preposition dependence task of each task and rearmounted dependence task according to described dependence, comprising:
To the arbitrary task needing to dispatch, by with the task of described arbitrary task Existence dependency relationship in be defined as the preposition dependence task of described arbitrary task by the task of described arbitrary Task Dependent scheduling result, and the task of the scheduling result relying on described arbitrary task is defined as the rearmounted dependence task of described arbitrary task.
Second aspect, provides a kind of task scheduling apparatus based on HIVE task, and described device comprises:
First determination module, for determining the multiple tasks needing to carry out dispatching, and determines the dependence between each task;
Second determination module, for determining the preposition dependence task of each task and rearmounted dependence task according to described dependence;
Selecting module, for selecting the current task of needing scheduling in the task of carrying out dispatching at all needs, obtaining selected task;
Scheduler module, the scheduling result according to the preposition dependence task of described selected task is dispatched described selected task;
Processing module, for according to selecting the rearmounted dependence task of selected task described in the mode of described selected task and the mode process of the selected task of execution cost and all tasks of not dispatching, until dispatched the task that all needs carry out dispatching.
In conjunction with second aspect, in the first possible implementation of second aspect, described selection module, the task of dispatching as current needs for selecting the task of meeting schedulable condition in the task of carrying out dispatching at all needs; Wherein, described schedulable condition is that all preposition dependence task are all dispatched complete.
In conjunction with the first possible implementation of second aspect, in the implementation that the second of second aspect is possible, described selection module, for when have multiple meet the task of schedulable condition time, using the described multiple task of meeting schedulable condition all as the task of current needs scheduling, or, according to priority described multiple meet in the task of schedulable condition select the task task of dispatching as current needs.
In conjunction with second aspect, in the third possible implementation of second aspect, each needs carry out a dispatching cycle of task equal correspondence of dispatching; Described scheduler module, comprising:
Acquiring unit, for the preposition dependence task that obtains described selected task according to one or more scheduling result before obtaining after being scheduled the dispatching cycle of its correspondence;
Scheduling unit, for dispatching described selected task according to the scheduling result got.
In conjunction with second aspect, the first possible implementation of second aspect, any one possible implementation in the implementation that the second is possible and the third possible implementation, in the 4th kind of possible implementation of second aspect, described second determination module, for arbitrary task of dispatching needs, by with the task of described arbitrary task Existence dependency relationship in be defined as the preposition dependence task of described arbitrary task by the task of described arbitrary Task Dependent scheduling result, and the task of the scheduling result relying on described arbitrary task is defined as the rearmounted dependence task of described arbitrary task.
The beneficial effect that the technical scheme that the embodiment of the present invention provides is brought is:
By determining the dependence between each task of needing to carry out dispatching, and after the preposition dependence task determining each task according to dependence and rearmounted dependence task, the scheduling result of preposition dependence task according to the selected task meeting schedulable condition carries out task scheduling to selected task, and dispatch other tasks of not dispatching according to this, thus make task scheduling process fairly simple, and can ensure that task performs in order.Therefore, the efficiency of scheduler task is higher.
Accompanying drawing explanation
In order to be illustrated more clearly in the technical scheme in the embodiment of the present invention, below the accompanying drawing used required in describing embodiment is briefly described, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.
Fig. 1 is the process flow diagram of a kind of method for scheduling task based on HIVE task that one embodiment of the invention provides;
Fig. 2 is the process flow diagram of a kind of method for scheduling task based on HIVE task that another embodiment of the present invention provides;
Fig. 3 is the dependence schematic diagram between each task a kind of of providing of another embodiment of the present invention;
Fig. 4 is the structural representation of a kind of task scheduling apparatus based on HIVE task that another embodiment of the present invention provides;
Fig. 5 is the structural representation of a kind of scheduler module that another embodiment of the present invention provides;
Fig. 6 is the structural representation of a kind of server that another embodiment of the present invention provides.
Embodiment
For making the object, technical solutions and advantages of the present invention clearly, below in conjunction with accompanying drawing, embodiment of the present invention is described further in detail.
Because correlation technique is before each scheduler task, after needing to determine according to the scheduling result of the task of a upper scheduling task that next needs are dispatched by relevant staff, could dispatch task.When task is more time, manually scheduling controlling is carried out to each task, cause task scheduling process more loaded down with trivial details.Therefore, the efficiency of scheduler task is lower.
In order to prevent above-mentioned situation, improve efficiency during scheduler task.Embodiments provide a kind of method for scheduling task based on HIVE task, see Fig. 1, the method flow that the present embodiment provides comprises:
101: determine the multiple tasks needing to carry out dispatching, and determine the dependence between each task;
102: determine the preposition dependence task of each task and rearmounted dependence task according to dependence;
As a kind of embodiment, determine the preposition dependence task of each task and rearmounted dependence task according to dependence, comprising:
To the arbitrary task needing to dispatch, by with the task of arbitrary task Existence dependency relationship in be defined as the preposition dependence task of arbitrary task by the task of arbitrary Task Dependent scheduling result, and the task of the scheduling result relying on arbitrary task is defined as the rearmounted dependence task of arbitrary task.
103: carry out at all needs in the task of dispatching, selecting the current task of needing scheduling, obtain selected task;
As a kind of embodiment, carry out at all needs in the task of dispatching, selecting the current task of needing scheduling, comprising:
Task of selecting the task of meeting schedulable condition to dispatch as current needs in the task of dispatching is carried out at all needs; Wherein, schedulable condition is that all preposition dependence task are all dispatched complete.
As a kind of embodiment, carry out task of selecting the task of meeting schedulable condition to dispatch as current needs in the task of dispatching at all needs, comprising:
If there is the multiple task of meeting schedulable condition, using the multiple task of meeting schedulable condition all as the task of current needs scheduling, or, meet in the task of schedulable condition task of selecting a task to dispatch as current needs according to priority multiple.
104: the scheduling result according to the preposition dependence task of selected task is dispatched selected task;
As a kind of embodiment, each needs carry out a dispatching cycle of task equal correspondence of dispatching; Scheduling result according to the preposition dependence task of selected task is dispatched selected task, comprising:
The preposition dependence task obtaining selected task, according to one or more scheduling result before obtaining after being scheduled the dispatching cycle of its correspondence, is dispatched selected task according to the scheduling result got.
105: according to selecting the mode of selected task and dispatching the rearmounted dependence task of the selected task of mode process of selected task and all tasks of not dispatching, until the task of having dispatched that all needs carry out dispatching.
The method that the embodiment of the present invention provides, by determining the dependence between each task of needing to carry out dispatching, and after the preposition dependence task determining each task according to dependence and rearmounted dependence task, the scheduling result of preposition dependence task according to the selected task meeting schedulable condition carries out task scheduling to selected task, and dispatch other tasks of not dispatching according to this, thus task scheduling process can be simplified, and can ensure that task performs in order.Therefore, the efficiency of scheduler task is higher.
In conjunction with the content of above-described embodiment, another embodiment of the present invention provides a kind of method for scheduling task based on HIVE task, and see Fig. 2, the method flow that the embodiment of the present invention provides comprises:
201: determine the multiple tasks needing to carry out dispatching, and determine the dependence between each task;
Wherein, the present embodiment is to determining that the mode of the multiple tasks needing to dispatch limits, include but not limited to, manually pre-entering on the basis needing each self-corresponding task parameters of multiple tasks of carrying out dispatching, determine the multiple tasks needing to carry out dispatching according to the task parameters of input.About the task parameters that each task needing to carry out dispatching is corresponding, the present embodiment does not do concrete restriction, includes but not limited to task names, task identification etc.In addition, in order to dispatch the task that all needs carry out dispatching in order, an each task corresponding dispatching cycle, the present embodiment does not limit the length of dispatching cycle corresponding to each task.During concrete enforcement, can be and need each task of carrying out dispatching to pre-set a task class, this task class comprise one dispatching cycle parameter.After determining the multiple tasks needing to carry out dispatching, the dispatching cycle that can comprise at the task class of each task arranges concrete dispatching cycle in parameter.Such as, take dispatching cycle as sky, week, the moon be example, some tasks may need perform once for one day, the dispatching cycle that then this task is corresponding is sky, some tasks may need perform once for one week, the dispatching cycle that then this task is corresponding is week, and some tasks may need perform once for one month, then the dispatching cycle that this task is corresponding is the moon.It should be noted that, after corresponding dispatching cycle is set for each task, if need to revise dispatching cycle corresponding to a certain task in the later stage, then parameter dispatching cycle corresponding to task class of this task reconfigurable, thus making system can flexibly according to the task that scheduling dispatching cycle of different length is corresponding, the present embodiment does not do concrete restriction to this.
Determine the multiple tasks needing to carry out dispatching based on aforesaid way after, in the process of concrete scheduler task, sometimes the output data that obtain after being the some other task of scheduling of the input data of scheduling needed for current task, that is, after scheduling current task needs to wait until that some other task is scheduled, the scheduling result according to other tasks could perform.Thus, according to task processing logic, when dispatching certain task, need the scheduling result depending on being scheduled of task above, to this, the relation between these tasks is called dependence by the present embodiment.Such as, can obtain one after task A scheduling and export data, for the output data obtained for data M, according to task processing logic, data M can as the input data of scheduler task B.Therefore, when can determine scheduler task B, need the scheduling result depending on task A, i.e. Existence dependency relationship between task A and task B.
About the mode of the dependence determined between all tasks, the present embodiment does not do concrete restriction to this, when specifically implementing, includes but not limited to realize by obtaining the dependence parameter pre-entered.Such as, each task has corresponding dependence parameter, by manually arranging this dependence parameter, the dependence of this task and other tasks can be set, and then the dependence between all tasks can be determined by the dependence parameter got.No matter adopt the dependence which kind of mode is determined between each task, this dependence all can include but not limited to represent to the dependence of the scheduling result of task and by the relation relied on, thus logical order when can show that each task is scheduled.The dependence of carrying out with the needs determined between the task of dispatching is illustrated in figure 3 example, equal Existence dependency relationship between two tasks that each arrow connects, and the direction of arrow can represent and to rely on and by the physical relationship relied on.As shown in Figure 3, Existence dependency relationship between the task A that arrow connects and task C, because task C is pointed to by task A in the direction of arrow, then represents that the scheduling result of task A is relied on by task C, or the scheduling result of task C dependence task A.That is, the scheduling result of dependence task A is needed during scheduler task C.
202: determine the preposition dependence task of each task and rearmounted dependence task according to dependence;
As a kind of embodiment, determine the preposition dependence task of each task and the mode of rearmounted dependence task according to dependence, include but not limited to:
To the arbitrary task needing to dispatch, by with the task of arbitrary task Existence dependency relationship in be defined as the preposition dependence task of arbitrary task by the task of arbitrary Task Dependent scheduling result, and the task of the scheduling result relying on arbitrary task is defined as the rearmounted dependence task of arbitrary task.
For the ease of understanding, still for the dependence shown in Fig. 3, for the task D in Fig. 3, owing to being respectively task A and task G with the task of task D Existence dependency relationship, and in task A and task G, as shown in Figure 3, task A is relied on scheduling result by task D, also namely the scheduling result of task A is relied on by task D, thus task A can be defined as the preposition dependence task of task D.In like manner, the scheduling result of task G dependence task D, thus can be defined as the rearmounted dependence task of task D by task G.According to the mode of the preposition dependence task and rearmounted dependence task of determining task D, the dependence between each task shown in Fig. 3 can represent by such as following table 1.
Table 1
| Task | pre | next |
| Begin | A、B | |
| A | Begin | C、D |
| B | Begin | E、F |
| C | A | End |
| D | A | G |
| E | B | G、F |
| F | B、E | End |
| G | D、E | End |
Above in table 1, pre represents preposition dependence task, and next represents rearmounted dependence task.It should be noted that, Begin can see first being scheduled of task as, and End can see last being scheduled of task as.For first being scheduled of task, do not dispatch any task due to before it, this first being scheduled of task does not have dependable scheduling result, and thus this first being scheduled of task does not have preposition dependence task.For needing the scheduling result relying on other tasks, but not by the task of any Task Dependent scheduling result, there is no rearmounted dependence task.In addition, for any one task, it both can as the preposition dependence task of other tasks, also can as the rearmounted dependence task of other tasks.And a task can have one or more preposition dependence task, also can have one or more rearmounted dependence task.
203: carry out task of selecting the task of meeting schedulable condition to dispatch as current needs in the task of dispatching at all needs, obtain selected task;
The method that the present embodiment provides is in actual application, scheduling due to a task needs the scheduling result relying on its preposition dependence task, thus the current task of needing scheduling is being selected, obtain selected task, and before dispatching selected task, the preposition dependence task of this selected task needs all scheduling complete.Particularly more for the task quantity needing to carry out dispatching, the comparatively complicated situation of the dependence between task.In this case, the task that the quantity that easily can there is preposition dependence task is greater than, dispatches this task with the preposition dependence task of two or more, needs the scheduling result relying on its all preposition dependence task.For this reason, the method that the present embodiment provides, when selecting the task of current needs scheduling, includes but not limited to that the schedulable condition of selected task demand fulfillment is defined as all preposition dependence task all to be dispatched complete.
For the task C in the dependence shown in above-mentioned table 1, preposition dependence task due to task C is task A, namely need first scheduler task A before scheduler task C, if thus using task C as current need the task of scheduling time, the schedulable condition of demand fulfillment is that task A scheduling is complete.For the task G in the dependence shown in above-mentioned table 1, preposition dependence task due to task G is task D and task E, namely first scheduler task D and task E is needed before scheduler task G, if during the task of thus being dispatched as current needs by task G, the schedulable condition of demand fulfillment is that task D and task E scheduling is complete.
For there being multiple situation meeting the task of schedulable condition, when selecting the task of current needs scheduling, the method that the present embodiment provides includes but not limited to adopt any one mode in the following two kinds processing mode:
Mode one: using the multiple task of meeting schedulable condition all as the task of current needs scheduling;
For this kind of mode one, because multiple preposition dependence task meeting the task of schedulable condition is all dispatched complete, that is, this the multiple task of meeting schedulable condition all can be dispatched, thus by using the multiple task of meeting schedulable condition all as the task that current needs are dispatched, and this multiple task of meeting schedulable condition to be dispatched simultaneously, thus the degree of parallelism of task scheduling can be improved, save the time that whole task is scheduled, and then improve the dispatching efficiency of task.For the ease of understanding this kind of mode one, still for the dependence shown in Fig. 3, if task A and task B all dispatches complete, then now mean that all preposition dependence task of task C, D and E is all dispatched complete, namely task C, D and E all meet schedulable condition, next simultaneously thus can using task C, D and E all as the current task of needing scheduling, thus scheduler task C, D and E.
Mode two: meet in the task of schedulable condition task of selecting a task to dispatch as current needs multiple according to priority.
Wherein, the present embodiment does not limit the determination mode of priority, includes but not limited to, when determining the task of needing to carry out dispatching, also comprise the step of the priority determining each task.Determine that the tandem that the mode of the priority of each task includes but not limited to be transfused to according to task is determined, also can be determined by the priority parameters manually arranged.No matter adopt the priority of which kind of mode determination task, occur multiple meet the task of schedulable condition time, task an of task can be selected to dispatch as current needs according to the priority of the task of meeting schedulable condition.Such as, still for the dependence shown in Fig. 3, if task A and task B all dispatches complete, then now all preposition dependence task of task C, D and E is all dispatched complete, namely task C, D and E all meet schedulable condition, if the priority of task C is the highest in task C, D and E, thus task C can being dispatched as current needs of task, thus following scheduler task C.
Certainly, except adopt above-mentioned according to priority except multiple mode meeting in the task of schedulable condition task of selecting task to dispatch as current needs, the task that in the task of schedulable condition, Stochastic choice task is dispatched as current needs can also be met multiple, or meeting in the task of schedulable condition task of selecting a task to dispatch as current needs according to other modes multiple, the present embodiment does not do concrete restriction to this.
It should be noted that, above-mentioned schedulable condition is applicable to the task with preposition dependence task, for the task that first needs is scheduled, because it does not have preposition dependence task, it is complete that the method that the present embodiment provides will give tacit consent to its preposition dependence task scheduling, thus these first needs task of being scheduled still can first be scheduled.In addition, during the task that triggering scheduling first needs are scheduled, the present embodiment does not limit concrete triggering mode, include but not limited to be the task creation timer that first needs is scheduled, when the time that time is up set by timer, then the task that can trigger these first needs is scheduled is dispatched.
204: the scheduling result according to the preposition dependence task of selected task is dispatched selected task;
About the process that the scheduling result of the preposition dependence task according to selected task is dispatched selected task, the present embodiment does not do concrete restriction, includes but not limited to carry out according to actual demand.In addition, if a task is scheduled once, then the scheduling result of this task is one.A task is needed by the situation of periodic scheduling, each needs carry out a dispatching cycle of task equal correspondence of dispatching, within dispatching cycle, often dispatch this task once all can obtain a scheduling result, to this, when the method that the present embodiment provides is dispatched selected task according to the scheduling result of the preposition dependence task of selected task, include but not limited to:
The preposition dependence task obtaining selected task, according to one or more scheduling result before obtaining after being scheduled the dispatching cycle of its correspondence, is dispatched selected task according to the scheduling result got.
Such as, with dispatching cycle corresponding to the preposition dependence task of selected task for sky, namely this preposition dependence task need every day to be dispatched is once example, the scheduling result of this preposition dependence task according to one to multiple sky before obtaining after being scheduled the dispatching cycle of its correspondence can be obtained, according to the scheduling result in one to multiple sky before this preposition dependence task, selected task is dispatched.Such as, with selected task for task M, its preposition dependence task is task N, and the dispatching cycle of task N is sky is example, can obtain the scheduling result of this task N according to one to multiple sky before obtaining after being scheduled the dispatching cycle of its correspondence, the scheduling result according to sky one to multiple before this task N is dispatched task M.
It should be noted that, because the length of dispatching cycle is except sky, can also be hour, week and the moon etc.Therefore, when obtaining the scheduling result of task corresponding to the dispatching cycle of different length, can be dispatching cycle the process of the scheduling result of the task in sky with reference to above-mentioned acquisition, repeating no more herein.
In addition, because each task can have one or more preposition dependence task, for having plural preposition dependence task, and the situation that length dispatching cycle corresponding to each preposition dependence task is different, the method that the present embodiment provides stands good.Such as, with selected task, there are two preposition dependence task, to be respectively the dispatching cycle that these two preposition dependence task are corresponding hour and day, namely one of them preposition dependence task needs scheduling per hour once, another preposition dependence task needs dispatch once every day, then can obtain the scheduling result of one of them preposition dependence task according to one or more hour before obtaining after being scheduled the dispatching cycle of its correspondence, and obtain the scheduling result of another preposition dependence task according to one to multiple sky before obtaining after being scheduled the dispatching cycle of its correspondence, all scheduling result according to getting are dispatched selected task.Such as, with selected task for task H, it comprises two preposition dependence task, be respectively task X and task Y, and the dispatching cycle of task X is hour, the dispatching cycle of task Y is sky is example, the scheduling result of this task X according to one or more hour before obtaining after being scheduled the dispatching cycle of its correspondence can be obtained, and obtain the scheduling result of this task Y according to one to multiple sky before obtaining after being scheduled the dispatching cycle of its correspondence, according to the scheduling result in one to multiple sky before the scheduling result of one or more hour before this task X and task Y, task H is dispatched.
Further, no matter the preposition dependence task of selected task has several, and whether length dispatching cycle of each preposition dependence task is identical, before obtaining after obtaining the preposition dependence task of selected task and being scheduled according to the dispatching cycle of its correspondence during one or more scheduling result, the present embodiment does not limit the quantity of one or more scheduling result before acquisition, specifically can set according to actual demand.Such as, alternatively, slip dependent mechanism can be adopted, namely set up between for two tasks successively dependence time, one can being arranged and rely on number parameter, coming to be the number that two task creations rely on scheduling result according to relying on number parameter.Wherein, the dependence number parameter between task can be arranged according to demand, and the present embodiment does not do concrete restriction to this.Such as, with Existence dependency relationship between task P and task Q, the scheduling result of task P dependence task Q, and to rely on number parameter be 3 is example.If the dispatching cycle of task P is sky, the dispatching cycle of task Q is also sky, and the dependence number parameter between task P and task Q is 2, then represent that scheduler task P needs to depend on the scheduling result a few days ago when task Q dispatched according to the dispatching cycle of its correspondence.That is, if task Q is scheduled every day from September 2, then when needs scheduler task P on the 4th in September, need to depend on the scheduling result of task Q in September 2 and September 3.
It should be noted that, in the method that the present embodiment provides, the length of the dispatching cycle that selected task is corresponding should be more than or equal to the length of dispatching cycle corresponding to the preposition dependence task of selected task.In above-mentioned example, dispatching cycle corresponding to the preposition dependence task of the dispatching cycle that selected task is corresponding and this selected task, namely the length of the dispatching cycle that selected task is corresponding equaled the length of dispatching cycle corresponding to the preposition dependence task of selected task all for sky.In practical application, the length of the dispatching cycle that selected task is corresponding can also be greater than the length of dispatching cycle corresponding to the preposition dependence task of selected task.Such as, the length of the dispatching cycle that selected task is corresponding is year, the length of the dispatching cycle that the preposition dependence task of selected task is corresponding is the moon, the preposition dependence task then obtaining selected task is according to before obtaining after being scheduled the dispatching cycle of its correspondence during one or more scheduling result, its mode is identical with the principle of above-mentioned example, and the preposition dependence task that can obtain selected task to be scheduled the scheduling result obtained in one or more month front.
In addition, length for dispatching cycle corresponding to selected task is greater than the situation of the length of dispatching cycle corresponding to the preposition dependence task of selected task, the preposition dependence task obtaining selected task is according to before obtaining after being scheduled the dispatching cycle of its correspondence during one or more scheduling result, one or more scheduling result front got is except the preposition dependence task for selected task is from except one or more scheduling result before being scheduled in its dispatching cycle, one or more scheduling result front got can also be one or more scheduling result before the preposition dependence task of selected task is within the dispatching cycle that selected task is scheduled.Such as, the length of the dispatching cycle that selected task is corresponding is week, the length of the dispatching cycle that the preposition dependence task of selected task is corresponding is sky, the preposition dependence task then obtaining selected task according to before obtaining after being scheduled the dispatching cycle of its correspondence during one or more scheduling result, the scheduling result in one to multiple sky before within the preposition dependence task that can obtain selected task be scheduled in selected task a week.More specifically, if selected task is task R, its dispatching cycle is week, the preposition dependence task of selected task is task T, if at the second week scheduler task R in March, and the dependence number parameter between task R and task T is 3, then the preposition dependence task obtaining selected task, according to before obtaining after being scheduled the dispatching cycle of its correspondence during one or more scheduling result, can obtain the scheduling result that task T is scheduled the Monday in the second week in March to Wednesday.
It should be noted that, for the situation that selected task is the task that first is scheduled, due to non-scheduler task before first being scheduled of task, namely this first being scheduled of task is without preposition dependence task, then when this step 204 is dispatched selected task in the scheduling result of the preposition dependence task according to selected task, the scheduling result can giving tacit consent to the preposition dependence task of this selected task is 0, thus can directly dispatch selected task.Certainly, for the situation of first being scheduled of task as selected task, the processing mode of this step 204 can also have other modes, and the present embodiment does not do concrete restriction to this.
205: according to selecting the mode of selected task and dispatching the rearmounted dependence task of the selected task of mode process of selected task and all tasks of not dispatching, until the task of having dispatched that all needs carry out dispatching.
Achieve the scheduling to selected task by each step above-mentioned, carry out dispatching the task that in multiple task, other are not scheduled for needs, realize principle by above-mentioned steps and be scheduled successively, the present embodiment repeats no longer one by one to this.During concrete enforcement, for the ease of recording the task and convenient scheduling residue task that were scheduled, the method that the present embodiment provides can adopt the mode shown in above-mentioned table 1 to be that each task arranges a preposition dependence task mark, and whether the preposition dependence task that this preposition dependence task mark can be used for each task be recorded in current task scheduling flow is finished; After the complete task of each scheduling, perform mark according to the preposition dependence task that the task of having dispatched revises unenforced task corresponding.After concrete modification mode includes but not limited to have dispatched a task at every turn, delete using this task as the preposition dependence task mark that the task of preposition dependence task is corresponding.After amendment completes, determine the next task of needing scheduling according to the amended preposition dependence task mark of each task, and need the task of scheduling to dispatch, until the task that all needs carry out dispatching all is dispatched complete to the next one.Wherein, while preposition dependence task mark is set for each task, can also arrange rearmounted dependence task mark for each task, can be determined the rearmounted dependence task of each task for convenience by this rearmounted dependence task mark, the present embodiment does not do concrete restriction to this.After setting up preposition dependence task mark and rearmounted dependence task mark, the information of each task can as shown in table 2 below:
Table 2
| Task | Pre | Next |
| A | B | |
| B | A | C |
| C | B | D |
| D | C |
Wherein, in list items, Pre is the mark of the preposition dependence task of each task.As shown in Table 2, it is task A that preposition dependence task all dispatches complete task, then can determine that the current task of needing scheduling is task A.After task A completes scheduling, the mark that in table 2, each task is corresponding can be revised, thus the content of table 3 can be obtained:
Table 3
| Task | Pre | Next |
| B | C | |
| C | B | D |
| D | C |
In table 3, because task A scheduling is complete.Therefore, the relevant entries of task A is deleted.In addition, because task A scheduling is complete, in table 2, the Pre mark of each task is modified.The all preposition dependence task scheduling of current task B is complete as can be known from Table 3, thus can determine that task B is the current task of needing scheduling.In like manner, by said process by that analogy, the scheduling to all tasks is completed.
It should be noted that, due to when dispatching the task of current needs scheduling, the situation of task scheduling failure may be there is.Therefore, can arrange a task run pond and mission failure wait retry pond, the present embodiment does not do concrete restriction to this.Wherein, task run pond is for placing all tasks, and mission failure waits for that retry pond is for placing the task of task scheduling failure.From task run pond, take out the current task of needing scheduling during each scheduler task, the task scheduling needing current needs to dispatch failure, then put it into mission failure and wait in retry pond.During the task that scheduling is failed before needing to reschedule in mission failure wait retry pond, before scheduler task unsuccessfully can being waited in retry pond, the failed task of scheduling is again inserted in task run pond and is dispatched.
In addition, when determining the task of needing to carry out dispatching, the task of can carry out for each needs determined dispatching arranges this parameter of retry interval time of makeing mistakes, and the present embodiment does not do concrete restriction to this.When the task scheduling in task run pond is failed, failed tasks can be placed in mission failure and waits for retry pond.Now, task is placed in task run pond by the retry interval time of makeing mistakes again that can be corresponding according to the task that scheduling is failed.Such as, to dispatch failed task for task A, and retry interval time of makeing mistakes corresponding to task A was 5 seconds is example.When task A dispatches unsuccessfully, task A can be inserted mission failure and wait for retry pond, after 5 seconds, task A can be inserted in task run pond again.In addition, can also arrange this parameter of maximum reattempt times for setting up of task, the present embodiment does not do concrete restriction to this.When the number of times of task scheduling failure reaches maximum reattempt times, then can stop again dispatching this task.Such as, to dispatch failed task for task A, and maximum reattempt times corresponding to task A are 3 times is example.When the failed number of times of task A scheduling reaches 3 times, then can stop dispatching task A.
The method that the embodiment of the present invention provides, by determining the dependence between each task of needing to carry out dispatching, and after the preposition dependence task determining each task according to dependence and rearmounted dependence task, the scheduling result of preposition dependence task according to the selected task meeting schedulable condition carries out task scheduling to selected task, and dispatch other tasks of not dispatching according to this, thus task scheduling process can be simplified, and can ensure that task performs in order.Therefore, the efficiency of scheduler task is higher.
Another embodiment of the present invention provides a kind of task scheduling apparatus based on HIVE task, and this device is for performing the method that in each embodiment above-mentioned, any embodiment provides.See Fig. 4, this task scheduling apparatus comprises:
First determination module 401, for determining the multiple tasks needing to carry out dispatching, and determines the dependence between each task;
Second determination module 402, for determining the preposition dependence task of each task and rearmounted dependence task according to described dependence;
Selecting module 403, for selecting the current task of needing scheduling in the task of carrying out dispatching at all needs, obtaining selected task;
Scheduler module 404, the scheduling result according to the preposition dependence task of described selected task is dispatched described selected task;
Processing module 405, for according to selecting the rearmounted dependence task of selected task described in the mode of described selected task and the mode process of the selected task of execution cost and all tasks of not dispatching, until dispatched the task that all needs carry out dispatching.
As a kind of embodiment, described selection module 403, the task of dispatching as current needs for selecting the task of meeting schedulable condition in the task of carrying out dispatching at all needs; Wherein, described schedulable condition is that all preposition dependence task are all dispatched complete.
As a kind of embodiment, described selection module 403, for when have multiple meet the task of schedulable condition time, using the described multiple task of meeting schedulable condition all as the task of current needs scheduling, or, according to priority described multiple meet in the task of schedulable condition select the task task of dispatching as current needs.
As a kind of embodiment, each needs carry out a dispatching cycle of task equal correspondence of dispatching; See Fig. 5, described scheduler module 404, comprising:
Acquiring unit 4041, for the preposition dependence task that obtains described selected task according to one or more scheduling result before obtaining after being scheduled the dispatching cycle of its correspondence;
Scheduling unit 4042, for dispatching described selected task according to the scheduling result got.
Alternatively, described second determination module 402, for arbitrary task of dispatching needs, by with the task of described arbitrary task Existence dependency relationship in be defined as the preposition dependence task of described arbitrary task by the task of described arbitrary Task Dependent scheduling result, and the task of the scheduling result relying on described arbitrary task is defined as the rearmounted dependence task of described arbitrary task.
The device that the embodiment of the present invention provides, by determining the dependence between each task of needing to carry out dispatching, and after the preposition dependence task determining each task according to dependence and rearmounted dependence task, the scheduling result of preposition dependence task according to the selected task meeting schedulable condition carries out task scheduling to selected task, and dispatch other tasks of not dispatching according to this, thus task scheduling process can be simplified, and can ensure that task performs in order.Therefore, the efficiency of scheduler task is higher.
Fig. 5 is the block diagram of a kind of server 1900 according to an exemplary embodiment.Such as, server 1900 may be provided in a server.With reference to Fig. 5, server 1900 comprises processing components 1922, and it comprises one or more processor further, and the memory resource representated by storer 1932, can such as, by the instruction of the execution of processing element 1922, application program for storing.The application program stored in storer 1932 can comprise each module corresponding to one group of instruction one or more.In addition, processing components 1922 is configured to perform instruction, to perform based on the method performed by the task scheduling apparatus of HIVE task in the embodiment corresponding to above-mentioned Fig. 1 or Fig. 2, the process of manner of execution specifically with reference to above-mentioned Fig. 1 or method content embodiment illustrated in fig. 2, can repeat no more herein.
Server 1900 can also comprise the power management that a power supply module 1926 is configured to perform server 1900, a wired or wireless network interface 1950 is configured to server 1900 to be connected to network, and input and output (I/O) interface 1958.Server 1900 can operate the operating system based on being stored in storer 1932, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or similar.
The server that the embodiment of the present invention provides, by determining the dependence between each task of needing to carry out dispatching, and after the preposition dependence task determining each task according to dependence and rearmounted dependence task, the scheduling result of preposition dependence task according to the selected task meeting schedulable condition carries out task scheduling to selected task, and dispatch other tasks of not dispatching according to this, thus task scheduling process can be simplified, and can ensure that task performs in order.Therefore, the efficiency of scheduler task is higher.
It should be noted that: the task scheduling apparatus based on HIVE task that above-described embodiment provides is when scheduler task, only be illustrated with the division of above-mentioned each functional module, in practical application, can distribute as required and by above-mentioned functions and be completed by different functional modules, inner structure by device is divided into different functional modules, to complete all or part of function described above.In addition, the task scheduling apparatus based on HIVE task that above-described embodiment provides belongs to same design with the method for scheduling task embodiment based on HIVE task, and its specific implementation process refers to embodiment of the method, repeats no more here.
The invention described above embodiment sequence number, just to describing, does not represent the quality of embodiment.
One of ordinary skill in the art will appreciate that all or part of step realizing above-described embodiment can have been come by hardware, the hardware that also can carry out instruction relevant by program completes, program can be stored in a kind of computer-readable recording medium, the above-mentioned storage medium mentioned can be ROM (read-only memory), disk or CD etc.
These are only preferred embodiment of the present invention, not in order to limit the present invention, within the spirit and principles in the present invention all, any amendment done, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (10)
1. based on a method for scheduling task for HIVE task, it is characterized in that, described method comprises:
Determine the multiple tasks needing to carry out dispatching, and determine the dependence between each task;
The preposition dependence task of each task and rearmounted dependence task is determined according to described dependence;
Carry out at all needs in the task of dispatching, selecting the current task of needing scheduling, obtain selected task;
Scheduling result according to the preposition dependence task of described selected task is dispatched described selected task;
According to selecting the rearmounted dependence task of selected task described in the mode of described selected task and the mode process of the selected task of execution cost and all tasks of not dispatching, until dispatched the task that all needs carry out dispatching.
2. method according to claim 1, is characterized in that, described carrying out at all needs selects the current task of needing scheduling in the task of dispatching, comprising:
Task of selecting the task of meeting schedulable condition to dispatch as current needs in the task of dispatching is carried out at all needs; Wherein, described schedulable condition is that all preposition dependence task are all dispatched complete.
3. method according to claim 2, is characterized in that, described task of carrying out selecting the task of meeting schedulable condition to dispatch as current needs in the task of dispatching at all needs, comprising:
If there is the multiple task of meeting schedulable condition, using the described multiple task of meeting schedulable condition all as the task of current needs scheduling, or, according to priority described multiple meet in the task of schedulable condition select the task task of dispatching as current needs.
4. method according to claim 1, is characterized in that, each needs carry out a dispatching cycle of task equal correspondence of dispatching; The scheduling result of the described preposition dependence task according to described selected task is dispatched described selected task, comprising:
The preposition dependence task obtaining described selected task, according to one or more scheduling result before obtaining after being scheduled the dispatching cycle of its correspondence, is dispatched described selected task according to the scheduling result got.
5. the method according to claim arbitrary in Claims 1-4, is characterized in that, describedly determines the preposition dependence task of each task and rearmounted dependence task according to described dependence, comprising:
To the arbitrary task needing to dispatch, by with the task of described arbitrary task Existence dependency relationship in be defined as the preposition dependence task of described arbitrary task by the task of described arbitrary Task Dependent scheduling result, and the task of the scheduling result relying on described arbitrary task is defined as the rearmounted dependence task of described arbitrary task.
6. based on a task scheduling apparatus for HIVE task, it is characterized in that, described device comprises:
First determination module, for determining the multiple tasks needing to carry out dispatching, and determines the dependence between each task;
Second determination module, for determining the preposition dependence task of each task and rearmounted dependence task according to described dependence;
Selecting module, for selecting the current task of needing scheduling in the task of carrying out dispatching at all needs, obtaining selected task;
Scheduler module, the scheduling result according to the preposition dependence task of described selected task is dispatched described selected task;
Processing module, for according to selecting the rearmounted dependence task of selected task described in the mode of described selected task and the mode process of the selected task of execution cost and all tasks of not dispatching, until dispatched the task that all needs carry out dispatching.
7. device according to claim 6, is characterized in that, described selection module, the task of dispatching as current needs for selecting the task of meeting schedulable condition in the task of carrying out dispatching at all needs; Wherein, described schedulable condition is that all preposition dependence task are all dispatched complete.
8. device according to claim 7, it is characterized in that, described selection module, for when have multiple meet the task of schedulable condition time, using the described multiple task of meeting schedulable condition all as the task of current needs scheduling, or, according to priority described multiple meet in the task of schedulable condition select the task task of dispatching as current needs.
9. device according to claim 6, is characterized in that, each needs carry out a dispatching cycle of task equal correspondence of dispatching; Described scheduler module, comprising:
Acquiring unit, for the preposition dependence task that obtains described selected task according to one or more scheduling result before obtaining after being scheduled the dispatching cycle of its correspondence;
Scheduling unit, for dispatching described selected task according to the scheduling result got.
10. the device according to claim arbitrary in claim 6 to 9, it is characterized in that, described second determination module, for arbitrary task of dispatching needs, by with the task of described arbitrary task Existence dependency relationship in be defined as the preposition dependence task of described arbitrary task by the task of described arbitrary Task Dependent scheduling result, and the task of the scheduling result relying on described arbitrary task is defined as the rearmounted dependence task of described arbitrary task.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410831524.5A CN104536811B (en) | 2014-12-26 | 2014-12-26 | Method for scheduling task based on HIVE tasks and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410831524.5A CN104536811B (en) | 2014-12-26 | 2014-12-26 | Method for scheduling task based on HIVE tasks and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104536811A true CN104536811A (en) | 2015-04-22 |
| CN104536811B CN104536811B (en) | 2018-10-16 |
Family
ID=52852343
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410831524.5A Active CN104536811B (en) | 2014-12-26 | 2014-12-26 | Method for scheduling task based on HIVE tasks and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104536811B (en) |
Cited By (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105321045A (en) * | 2015-11-04 | 2016-02-10 | 北京知聚科技有限公司 | Service process formal model construction method and system |
| CN105389207A (en) * | 2015-11-06 | 2016-03-09 | 成都视达科信息技术有限公司 | Adaptive data request processing method and apparatus, task scheduler and adaptive data request processing system |
| CN105701624A (en) * | 2016-02-26 | 2016-06-22 | 广州品唯软件有限公司 | Task processing method and device |
| CN105912387A (en) * | 2015-08-25 | 2016-08-31 | 乐视网信息技术（北京）股份有限公司 | Method and device for dispatching data processing operation |
| CN106528275A (en) * | 2015-09-10 | 2017-03-22 | 网易（杭州）网络有限公司 | Processing method of data tasks and task scheduler |
| WO2017101468A1 (en) * | 2015-12-14 | 2017-06-22 | 中兴通讯股份有限公司 | Version task scheduling management method and system |
| CN107085536A (en) * | 2016-11-14 | 2017-08-22 | 阿里巴巴集团控股有限公司 | A kind of task management method and device |
| CN107168781A (en) * | 2017-04-07 | 2017-09-15 | 广东银禧科技股份有限公司 | 3D printing subtask scheduling method and device |
| CN107179896A (en) * | 2016-03-09 | 2017-09-19 | 阿里巴巴集团控股有限公司 | Task processing method and device |
| CN107818112A (en) * | 2016-09-13 | 2018-03-20 | 腾讯科技（深圳）有限公司 | A kind of big data analysis operating system and task submit method |
| CN108197778A (en) * | 2017-12-15 | 2018-06-22 | 远光软件股份有限公司 | A kind of method for allocating tasks and device |
| CN108763482A (en) * | 2018-05-25 | 2018-11-06 | 联想(北京)有限公司 | Method, apparatus and server cluster for data analysis flow scheduling |
| CN109062680A (en) * | 2018-08-15 | 2018-12-21 | 腾讯科技（深圳）有限公司 | A kind of data load method, device and storage medium |
| CN109783208A (en) * | 2017-11-13 | 2019-05-21 | 阿里巴巴集团控股有限公司 | Big data job scheduling method, platform, equipment and medium |
| CN109995824A (en) * | 2017-12-29 | 2019-07-09 | 阿里巴巴集团控股有限公司 | Method for scheduling task and device in a kind of peer-to-peer network |
| CN110096345A (en) * | 2019-03-16 | 2019-08-06 | 平安科技（深圳）有限公司 | Intelligent task dispatching method, device, equipment and storage medium |
| CN110633914A (en) * | 2019-09-20 | 2019-12-31 | 中国银行股份有限公司 | Batch job scheduling processing method and device |
| CN110737516A (en) * | 2018-07-20 | 2020-01-31 | 网宿科技股份有限公司 | A method and apparatus for performing data processing tasks |
| CN110764910A (en) * | 2019-10-23 | 2020-02-07 | 中国银行股份有限公司 | Batch job scheduling processing method and device |
| CN110837509A (en) * | 2019-11-08 | 2020-02-25 | 深圳市彬讯科技有限公司 | Method, device, equipment and storage medium for scheduling dependence |
| CN111190725A (en) * | 2019-08-09 | 2020-05-22 | 腾讯科技（深圳）有限公司 | Task processing method and device, storage medium and server |
| WO2020186809A1 (en) * | 2019-03-19 | 2020-09-24 | 深圳壹账通智能科技有限公司 | Big data platform-based hive task scheduling method and apparatus, device, and storage medium |
| CN113010289A (en) * | 2021-03-17 | 2021-06-22 | 杭州遥望网络科技有限公司 | Task scheduling method, device and system |
| CN114240214A (en) * | 2021-12-22 | 2022-03-25 | 上海中通吉网络技术有限公司 | Method for demand management |
| CN117032999A (en) * | 2023-10-09 | 2023-11-10 | 之江实验室 | CPU-GPU cooperative scheduling method and device based on asynchronous running |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102279888A (en) * | 2011-08-24 | 2011-12-14 | 北京新媒传信科技有限公司 | Method and system for scheduling tasks |
| CN102364447A (en) * | 2011-10-28 | 2012-02-29 | 北京航空航天大学 | A Job Scheduling Method for Optimizing Communication Energy Consumption Between Multitasks |
| CN102681894A (en) * | 2011-03-09 | 2012-09-19 | 中兴通讯股份有限公司 | Method and system for multitask concurrent scheduling |
| CN102880503A (en) * | 2012-08-24 | 2013-01-16 | 新浪网技术(中国)有限公司 | Data analysis system and data analysis method |
| US20130247058A1 (en) * | 2010-10-07 | 2013-09-19 | Commissariat a' l'Energie Atomique et aux Energies Alternative | System for scheduling the execution of tasks based on logical time vectors |
-
2014
- 2014-12-26 CN CN201410831524.5A patent/CN104536811B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130247058A1 (en) * | 2010-10-07 | 2013-09-19 | Commissariat a' l'Energie Atomique et aux Energies Alternative | System for scheduling the execution of tasks based on logical time vectors |
| CN102681894A (en) * | 2011-03-09 | 2012-09-19 | 中兴通讯股份有限公司 | Method and system for multitask concurrent scheduling |
| CN102279888A (en) * | 2011-08-24 | 2011-12-14 | 北京新媒传信科技有限公司 | Method and system for scheduling tasks |
| CN102364447A (en) * | 2011-10-28 | 2012-02-29 | 北京航空航天大学 | A Job Scheduling Method for Optimizing Communication Energy Consumption Between Multitasks |
| CN102880503A (en) * | 2012-08-24 | 2013-01-16 | 新浪网技术(中国)有限公司 | Data analysis system and data analysis method |
Cited By (31)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105912387A (en) * | 2015-08-25 | 2016-08-31 | 乐视网信息技术（北京）股份有限公司 | Method and device for dispatching data processing operation |
| CN106528275A (en) * | 2015-09-10 | 2017-03-22 | 网易（杭州）网络有限公司 | Processing method of data tasks and task scheduler |
| CN105321045A (en) * | 2015-11-04 | 2016-02-10 | 北京知聚科技有限公司 | Service process formal model construction method and system |
| CN105389207A (en) * | 2015-11-06 | 2016-03-09 | 成都视达科信息技术有限公司 | Adaptive data request processing method and apparatus, task scheduler and adaptive data request processing system |
| WO2017101468A1 (en) * | 2015-12-14 | 2017-06-22 | 中兴通讯股份有限公司 | Version task scheduling management method and system |
| CN105701624A (en) * | 2016-02-26 | 2016-06-22 | 广州品唯软件有限公司 | Task processing method and device |
| CN107179896A (en) * | 2016-03-09 | 2017-09-19 | 阿里巴巴集团控股有限公司 | Task processing method and device |
| CN107818112A (en) * | 2016-09-13 | 2018-03-20 | 腾讯科技（深圳）有限公司 | A kind of big data analysis operating system and task submit method |
| CN107818112B (en) * | 2016-09-13 | 2021-12-14 | 腾讯科技（深圳）有限公司 | Big data analysis operating system and task submitting method |
| CN107085536A (en) * | 2016-11-14 | 2017-08-22 | 阿里巴巴集团控股有限公司 | A kind of task management method and device |
| CN107168781A (en) * | 2017-04-07 | 2017-09-15 | 广东银禧科技股份有限公司 | 3D printing subtask scheduling method and device |
| CN109783208A (en) * | 2017-11-13 | 2019-05-21 | 阿里巴巴集团控股有限公司 | Big data job scheduling method, platform, equipment and medium |
| CN108197778A (en) * | 2017-12-15 | 2018-06-22 | 远光软件股份有限公司 | A kind of method for allocating tasks and device |
| CN109995824A (en) * | 2017-12-29 | 2019-07-09 | 阿里巴巴集团控股有限公司 | Method for scheduling task and device in a kind of peer-to-peer network |
| CN109995824B (en) * | 2017-12-29 | 2022-10-04 | 阿里巴巴集团控股有限公司 | Task scheduling method and device in peer-to-peer network |
| CN108763482B (en) * | 2018-05-25 | 2022-05-31 | 联想(北京)有限公司 | Method and device for scheduling data analysis process and server cluster |
| CN108763482A (en) * | 2018-05-25 | 2018-11-06 | 联想(北京)有限公司 | Method, apparatus and server cluster for data analysis flow scheduling |
| CN110737516A (en) * | 2018-07-20 | 2020-01-31 | 网宿科技股份有限公司 | A method and apparatus for performing data processing tasks |
| CN109062680A (en) * | 2018-08-15 | 2018-12-21 | 腾讯科技（深圳）有限公司 | A kind of data load method, device and storage medium |
| WO2020186787A1 (en) * | 2019-03-16 | 2020-09-24 | 平安科技（深圳）有限公司 | Intelligent task scheduling method, device and apparatus and storage medium |
| CN110096345A (en) * | 2019-03-16 | 2019-08-06 | 平安科技（深圳）有限公司 | Intelligent task dispatching method, device, equipment and storage medium |
| CN110096345B (en) * | 2019-03-16 | 2024-04-12 | 平安科技（深圳）有限公司 | Intelligent task scheduling method, device, equipment and storage medium |
| WO2020186809A1 (en) * | 2019-03-19 | 2020-09-24 | 深圳壹账通智能科技有限公司 | Big data platform-based hive task scheduling method and apparatus, device, and storage medium |
| CN111190725A (en) * | 2019-08-09 | 2020-05-22 | 腾讯科技（深圳）有限公司 | Task processing method and device, storage medium and server |
| CN110633914A (en) * | 2019-09-20 | 2019-12-31 | 中国银行股份有限公司 | Batch job scheduling processing method and device |
| CN110764910A (en) * | 2019-10-23 | 2020-02-07 | 中国银行股份有限公司 | Batch job scheduling processing method and device |
| CN110837509A (en) * | 2019-11-08 | 2020-02-25 | 深圳市彬讯科技有限公司 | Method, device, equipment and storage medium for scheduling dependence |
| CN113010289A (en) * | 2021-03-17 | 2021-06-22 | 杭州遥望网络科技有限公司 | Task scheduling method, device and system |
| CN114240214A (en) * | 2021-12-22 | 2022-03-25 | 上海中通吉网络技术有限公司 | Method for demand management |
| CN117032999A (en) * | 2023-10-09 | 2023-11-10 | 之江实验室 | CPU-GPU cooperative scheduling method and device based on asynchronous running |
| CN117032999B (en) * | 2023-10-09 | 2024-01-30 | 之江实验室 | CPU-GPU cooperative scheduling method and device based on asynchronous running |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104536811B (en) | 2018-10-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104536811A (en) | HIVE task based task scheduling method and device | |
| CN106802826B (en) | A thread pool-based business processing method and device | |
| US8595735B2 (en) | Holistic task scheduling for distributed computing | |
| US8589929B2 (en) | System to provide regular and green computing services | |
| CN107944773A (en) | Business process control method, device and storage medium | |
| CN109298929B (en) | Timed task execution time recommending method, device, equipment and storage medium | |
| CN107015849B (en) | Timing task reminder method and device | |
| US8458136B2 (en) | Scheduling highly parallel jobs having global interdependencies | |
| CN103744730B (en) | Task scheduling method and device | |
| WO2022253165A1 (en) | Scheduling method, system, server and computer readable storage medium | |
| CN110968371A (en) | A method and system for configuring workflow in a data warehouse | |
| CN110069481B (en) | Unified cleaning method and related equipment for production library data based on big data | |
| CN112748993A (en) | Task execution method and device, storage medium and electronic equipment | |
| CN109284324A (en) | The dispatching device of flow tasks based on Apache Oozie frame processing big data | |
| CN116089040A (en) | Service flow scheduling method and device, electronic equipment and storage medium | |
| CN113010276A (en) | A task scheduling method, device, terminal device and storage medium | |
| CN103177334A (en) | Method and system for scheduling robot resources | |
| CN109408204A (en) | A kind of method for scheduling task and device of distributed task scheduling system | |
| CN110362387B (en) | Distributed task processing method, device, system and storage medium | |
| CN102096857A (en) | Collaboration method and device for data processing process | |
| CN102508720A (en) | Method for improving efficiency of preprocessing module and efficiency of post-processing module and system | |
| CN116841758A (en) | Workflow task processing method, device, computer equipment and storage medium | |
| Chen et al. | Optimal algorithms and a PTAS for cost-aware scheduling | |
| CN110968401A (en) | Task scheduling method and device based on Quartz | |
| US20180341521A1 (en) | Managing job schedules |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 511446 Guangzhou City, Guangdong Province, Panyu District, South Village, Huambo Business District Wanda Plaza, block B1, floor 28 Applicant after: Guangzhou Huaduo Network Technology Co., Ltd. Address before: 510655, Guangzhou, Whampoa Avenue, No. 2, creative industrial park, building 3-08, Applicant before: Guangzhou Huaduo Network Technology Co., Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |