CN114371808B - Scheduling system-based flow node parameter transmission method and system - Google Patents
Scheduling system-based flow node parameter transmission method and system Download PDFInfo
- Publication number
- CN114371808B CN114371808B CN202210022798.4A CN202210022798A CN114371808B CN 114371808 B CN114371808 B CN 114371808B CN 202210022798 A CN202210022798 A CN 202210022798A CN 114371808 B CN114371808 B CN 114371808B
- Authority
- CN
- China
- Prior art keywords
- parameter
- configuration
- script
- parameter information
- configuration management
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 70
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000013515 script Methods 0.000 claims abstract description 101
- 238000004458 analytical method Methods 0.000 claims abstract description 41
- 230000014509 gene expression Effects 0.000 claims abstract description 11
- 238000012546 transfer Methods 0.000 claims abstract description 10
- 238000007726 management method Methods 0.000 claims description 60
- 230000015654 memory Effects 0.000 claims description 46
- 230000001360 synchronised effect Effects 0.000 claims description 17
- 238000012545 processing Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 14
- 238000010276 construction Methods 0.000 claims description 6
- 238000012384 transportation and delivery Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 description 11
- 230000008569 process Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000013523 data management Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000002093 peripheral effect Effects 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 230000001413 cellular effect Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 238000010295 mobile communication Methods 0.000 description 2
- 101001006776 Homo sapiens Kinesin-like protein KIFC1 Proteins 0.000 description 1
- 102100027942 Kinesin-like protein KIFC1 Human genes 0.000 description 1
- 239000008186 active pharmaceutical agent Substances 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000013499 data model Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04847—Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
- G06Q10/103—Workflow collaboration or project management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Physics & Mathematics (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Entrepreneurship & Innovation (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- Operations Research (AREA)
- General Business, Economics & Management (AREA)
- Marketing (AREA)
- Economics (AREA)
- Data Mining & Analysis (AREA)
- Human Computer Interaction (AREA)
- Computer And Data Communications (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a method and a system for transmitting flow node parameters based on a dispatching system, wherein the method comprises the following steps: adding task flow nodes in a configuration management flow chart editing interface; constructing a first parameter transfer rule according to the first configuration instruction; obtaining a first script execution instruction; analyzing the task flow node through a regular expression according to the first script execution instruction and the first parameter transmission rule to obtain first analysis parameter information; inputting the first analysis parameter information into a configuration management module, and obtaining first real-time parameter information of a first real-time flow node according to the configuration management module; and transmitting the first real-time parameter information to the downstream script of the task flow node as an execution parameter. The system solves the technical problems that the prior art has no unified configuration management interface, has poor flexibility, can not quickly locate and discover the problems on the system, and lacks necessary log information.
Description
Technical Field
The invention relates to the field of data processing, in particular to a scheduling system-based flow node parameter transmission method and system.
Background
Enterprises usually adopt a mode of task scheduling and script development to process data, but with the development of the enterprises and the increase of data volume, the original method of using linux crontab to schedule tasks is adopted, multiple scripts are used as a method of data transmission among scripts through a hive intermediate data table, complex dependency configuration among multiple tasks cannot be met, an existing scheduling system can manage complex dependency among the multiple scripts through configuration task flows, data parameter transmission among the multiple scripts in the flows is not supported, only all parameters and custom parameters are supported, and the function of dynamically transmitting parameters according to script execution results cannot be realized.
The global parameters and the custom parameters supported by the existing scheduling system can be set before the task flow is operated, for example, the task execution parameters are set to be on a certain day, the task with a specified date is executed after the set date variable is acquired from the script, but the parameters cannot be dynamically transmitted to the downstream nodes of the flow according to the script execution result.
However, in the process of realizing the technical scheme of the application, the technology at least has the following technical problems:
The prior art has the problems that a unified configuration management interface is not available, the flexibility is poor, the problems can not be rapidly positioned and found on a system, and necessary log information is lacking.
Disclosure of Invention
The application solves the technical problems that the prior art has no unified configuration management interface, has poor flexibility, can not quickly locate and find problems on the system and lacks necessary log information by providing the scheduling system flow node parameter transmission method and system, achieves the purpose of providing the configuration interface, realizes node dynamic parameter transmission by configuring parameter transmission rules through the interface, can inquire flow dynamic parameter information in the system and solves the problem of script parameter dynamic transmission; the operation of a developer is simplified, and the task is easier to maintain in a configuration mode; the script parameter transfer relation can be checked through the system, the data management work is facilitated, and the technical effect of script code management cost is reduced.
The present invention has been made in view of the above problems, and it is an object of the present invention to provide a method of overcoming or at least partially solving the above problems.
In a first aspect, the present application provides a scheduling system based flow node parameter transmission method, the method comprising: adding task flow nodes in a configuration management flow chart editing interface; constructing a first parameter transmission rule according to a first configuration instruction, wherein the first parameter transmission rule is an acquisition rule of a script downstream transmission parameter; obtaining a first script execution instruction; analyzing the task flow node through a regular expression according to the first script execution instruction and the first parameter transmission rule to obtain first analysis parameter information; inputting the first analysis parameter information into a configuration management module, and obtaining first real-time parameter information of a first real-time flow node according to the configuration management module; and transmitting the first real-time parameter information to the downstream script of the task flow node as an execution parameter.
On the other hand, the application also provides a system for transmitting the flow node parameters based on the dispatching system, which comprises: the first adding unit is used for adding task flow nodes on a configuration management flow chart editing interface; the first construction unit is used for constructing a first parameter transmission rule according to a first configuration instruction, wherein the first parameter transmission rule is an acquisition rule of a script downstream transmission parameter; the first obtaining unit is used for obtaining a first script execution instruction; the second obtaining unit is used for carrying out instructions according to the first script, analyzing the task flow nodes through regular expressions according to the first parameter transmission rule, and obtaining first analysis parameter information; the third obtaining unit is used for inputting the first analysis parameter information into a configuration management module and obtaining first real-time parameter information of a first real-time flow node according to the configuration management module; and the first transmission unit is used for transmitting the first real-time parameter information to the downstream script of the task flow node as an execution parameter.
In a third aspect, the present application provides an electronic device comprising a bus, a transceiver, a memory, a processor and a computer program stored on the memory and executable on the processor, the transceiver, the memory and the processor being connected by the bus, the computer program when executed by the processor implementing the steps in the method of controlling output data as described in any one of the above.
In a fourth aspect, the present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method of controlling output data of any of the above.
One or more technical schemes provided by the application have at least the following technical effects or advantages:
Because the configuration management flow chart editing interface is adopted, task flow nodes are added; constructing a first parameter transmission rule according to a first configuration instruction, wherein the first parameter transmission rule is an acquisition rule of a script downstream transmission parameter; obtaining a first script execution instruction; analyzing the task flow node through a regular expression according to the first script execution instruction and the first parameter transmission rule to obtain first analysis parameter information; inputting the first analysis parameter information into a configuration management module, and obtaining first real-time parameter information of a first real-time flow node according to the configuration management module; and transmitting the first real-time parameter information to a script at the downstream of the task flow node as an execution parameter. The configuration interface is further provided, the user configures parameter transmission rules through the interface, the node dynamic parameter transmission is realized, the flow dynamic parameter information can be queried in the system, and the problem of script parameter dynamic transmission is solved; the operation of a developer is simplified, and the task is easier to maintain in a configuration mode; the script parameter transfer relation can be checked through the system, the data management work is facilitated, and the technical effect of script code management cost is reduced.
The foregoing description is only an overview of the present application, and is intended to be implemented in accordance with the teachings of the present application in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present application more readily apparent.
Drawings
FIG. 1 is a flow chart of a method for transmitting parameters of a flow node based on a scheduling system according to the present application;
FIG. 2 is a schematic flow chart of constructing a first parameter transmission rule in a scheduling system-based flow node parameter transmission method according to the present application;
FIG. 3 is a schematic flow chart of storing analysis parameter information to redis in a method for transferring parameters of a flow node based on a scheduling system according to the present application;
FIG. 4 is a schematic diagram of a system for transmitting parameters based on a process node of a scheduling system according to the present application;
Fig. 5 is a schematic structural view of an exemplary electronic device of the present application.
Reference numerals illustrate: a first adding unit 11, a first building unit 12, a first obtaining unit 13, a second obtaining unit 14, a third obtaining unit 15, a first transferring unit 16, a bus 1110, a processor 1120, a transceiver 1130, a bus interface 1140, a memory 1150, an operating system 1151, application programs 1152 and a user interface 1160.
Detailed Description
In the description of the present application, those skilled in the art will appreciate that the present application may be embodied as methods, apparatus, electronic devices, and computer-readable storage media. Accordingly, the present application may be embodied in the following forms: complete hardware, complete software (including firmware, resident software, micro-code, etc.), a combination of hardware and software. Furthermore, in some embodiments, the application may also be embodied in the form of a computer program product in one or more computer-readable storage media, which contain computer program code.
Any combination of one or more computer-readable storage media may be employed by the computer-readable storage media described above. The computer-readable storage medium includes: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of the computer readable storage medium include the following: portable computer magnetic disks, hard disks, random access memories, read-only memories, erasable programmable read-only memories, flash memories, optical fibers, optical disk read-only memories, optical storage devices, magnetic storage devices, or any combination thereof. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, device.
Summary of the application
The application provides a method, a device and electronic equipment through flow charts and/or block diagrams.
It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions. These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
These computer readable program instructions may also be stored in a computer readable storage medium that can cause a computer or other programmable data processing apparatus to function in a particular manner. Thus, instructions stored in a computer-readable storage medium produce an instruction means which implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
The present application will be described below with reference to the drawings in the present application.
Example 1
As shown in fig. 1, the present application provides a method for transmitting parameters based on a flow node of a scheduling system, wherein the method includes:
step S100: adding task flow nodes in a configuration management flow chart editing interface;
specifically, the system is divided into a configuration management module and a script execution module, wherein the configuration management module provides scripts, a dispatching configuration page and dispatching flow management, and the script execution module is deployed at each server node and is responsible for executing instructions issued by the configuration management module. The configuration management flow chart editing interface provides a convenient and easy-to-use working interface for developers, configures task scheduling flows and ensures development working efficiency. And dragging and newly adding task flow nodes in the configuration management flow chart editing interface, wherein the flow nodes are a series of link steps in the flow, one flow task consists of a plurality of nodes, the node types comprise task nodes, active nodes, branch nodes, aggregation nodes, decision nodes and edge nodes, and the flow logic is realized through the combination of the nodes of different types.
Step S200: constructing a first parameter transmission rule according to a first configuration instruction, wherein the first parameter transmission rule is an acquisition rule of a script downstream transmission parameter;
as shown in fig. 2, further, the step S200 of constructing a first parameter delivery rule according to the first configuration instruction further includes:
Step S210: according to the first configuration instruction, configuring an acquisition mode and a configuration parameter prefix, wherein the configuration acquisition mode is realized by analyzing script output content, and the configuration parameter prefix is realized by configuring global parameters and local parameters;
step S220: and constructing the first parameter transfer rule according to the configuration acquisition mode and the configuration parameter prefix.
Specifically, when a task flow node is newly added, a first parameter transmission rule is set, wherein the first parameter transmission rule is an acquisition rule for transmitting parameters downstream by a script, and the acquisition rule comprises a mode for acquiring the parameters and a parameter prefix. The configuration acquisition mode is realized by analyzing script output content and is set as follows: analyzing script output content; the configuration parameter prefix is realized by configuring global parameters and local parameters, and is set as global parameters { flow name }: { parameter name }: "and local parameters } { flow name/node name }: { parameter name }:" or { flow name/sub-flow name/node name }: { parameter name }: ". And a parameter transmission rule is configured through an interface, so that a foundation is provided for the subsequent realization of the dynamic parameter transmission.
Step S300: obtaining a first script execution instruction;
step S400: analyzing the task flow node through a regular expression according to the first script execution instruction and the first parameter transmission rule to obtain first analysis parameter information;
Specifically, the script execution module is deployed at each server node, and is responsible for executing the instruction issued by the configuration management module, analyzing the script execution result log, and uploading the script execution result log to the configuration management module. When the script execution module executes the flow node configured with parameter transfer, a standard output stream and an analysis script output log are obtained, parameter information output by the analysis node, namely the first analysis parameter information, is matched through a regular expression according to a configuration rule, wherein the regular expression is a logic formula for operating a character string, namely a rule character string is formed by a plurality of specific characters defined in advance and the combination of the specific characters, and the rule character string is used for expressing a filtering logic for the character string.
Step S500: inputting the first analysis parameter information into a configuration management module, and obtaining first real-time parameter information of a first real-time flow node according to the configuration management module;
Specifically, the script execution module reports the parsed first parsing parameter information to the configuration management module, and the first parsing parameter information is stored in redis, redis (Remote Dictionary Server), namely a remote dictionary service, which is an open-source log-type, key-Value database written in ANSI C language, supported by a network and capable of being based on memory and persistent, and provides APIs of multiple languages. The system adopts high available redis as an intermediate medium for script parameter transmission, and the execution module acquires and analyzes script output content according to configured parameter transmission rules and uploads the script output content to the configuration management module. The first real-time flow node is a flow node in execution of the system, first real-time parameter information is obtained according to a current flow execution ID and a node name, the first real-time parameter information is a script parameter matched with redis, the script parameter comprises a global parameter and a current node parameter, the flow execution ID global obtains the global parameter, and the flow execution ID node name obtains the local parameter.
Step S600: and transmitting the first real-time parameter information to the downstream script of the task flow node as an execution parameter.
Specifically, before the downstream script is executed, the configuration management module searches the matched script parameters in the redis and transmits the matched script parameters to the downstream script of the task flow as execution parameters, and after the configuration management module acquires the script parameters, the configuration management module submits the script and parameter variables to the script execution module, and the script execution module transmits the script parameters and executes the script. The configuration interface is provided, the user configures the parameter transmission rule through the interface, the node dynamic parameter transmission is realized, the flow dynamic parameter information can be queried in the system, and the technical effect of the script parameter dynamic transmission is solved.
As shown in fig. 3, further, the present application further includes:
Step S710: the configuration management module and the script execution module have a first connection relationship;
Step S720: uploading the first analysis parameter information obtained by the script execution module to the configuration management module based on the first connection relation;
step S730: the configuration management module stores the first analysis parameter information to redis.
Specifically, the system is divided into a configuration management module and a script execution module, the first connection relation is a connection interaction relation of the configuration management module and the script execution module, the configuration management module provides script and scheduling configuration page and scheduling flow management and is responsible for collecting information reported by the execution module, the script execution module is deployed at each server node and is responsible for executing an instruction issued by the configuration management module, analyzing a script execution result log, and based on the first connection relation, uploading an analysis parameter information result obtained by the script execution module to the configuration management module. The configuration management module stores the first analysis parameter information to the redis, the system adopts the high-availability redis as an intermediate medium for script parameter transmission, and node parameter transmission is realized based on the redis, so that the technical effects of high performance and high processing speed are achieved.
Further, step S730 of the present application further includes:
step S731: the redis uses a preset hash structure to store the first analysis parameter information;
Step S732: storing global parameters of the first analysis parameter information according to a first storage mode, wherein the first storage mode is a global field;
step S733: and storing the local parameters of the first analysis parameter information according to a second storage mode, wherein the second storage mode is a json mode.
Further, the local variable of the flow takes the node name as a field, and the value of the field is a parameter json; the sub-flow local variable takes the name of the sub-flow and the name of the node as field, and the value of the field is taken as parameter json.
Specifically, the redis uses a hash (hash) structure to store the first parsing parameter information, i.e. the flow parameter, where the hash structure is a data structure that is directly accessed according to the key value, and the record is accessed by mapping the key value to a position in the table, so as to speed up the searching. The hash key is a unique ID automatically generated by the process execution, the global variable parameter of the first analysis parameter information is stored in the first storage mode, namely stored in a global field, the global function is to acquire the global configuration variable content of the system, and the global flag is used for indicating that the search operation will search all the conforming items in the searched character string, and the search operation is not just the first one.
Further, the local variable parameters of the first analysis parameter information are stored in the second storage mode, that is, a plurality of parameters are stored in a json mode, json is a grammar for storing and exchanging text information, and is similar to XML, smaller and faster than XML, and is easier to analyze. The local variable of the flow uses the node name as field, the field represents the characteristic of the field type, the value of the field is parameter json, the local variable of the sub-flow uses the sub-flow name + the node name as field, the value of the field is parameter json. For example, HSET "flow execution ID", "global", "node name" { } ", { \" dt\ ":20211018, \" finish\ ": true }. And the high-availability redis is used as an intermediate medium for script parameter transmission, and the analysis parameters are stored in the redis, so that the processing speed is high, and the technical effect of node parameter transmission is realized based on the redis.
Further, the application also comprises:
Step S810: according to the synchronous access service line data of the data interface layer, first service data are obtained, wherein the data interface layer comprises a plurality of synchronous nodes;
Step S820: obtaining a plurality of synchronization completion parameters according to the plurality of synchronization nodes;
Step S830: summarizing the plurality of synchronous completion parameter input data summarization layers to obtain first summarized data;
Step S840: and executing data processing of the corresponding service line according to the first summarized data.
Specifically, when processing call volume data, a task flow is configured according to the idea of hierarchical processing, the data interface layer is a temporary data area and is used for operating data storage, namely, the ODS layer is totally called Operational Data Store, storing service data extracted from a source system such as a log, a member, an order and the like, and synchronizing all service line data at the ODS layer. The data summarization layer, DW (Data Warehouse) layer, is the main body of the data warehouse, and the data obtained from the ODS layer establishes various data models according to the subject.
And when the DW layer collects data and the task flow is configured, the ODS layer comprises a plurality of synchronous nodes, the ODS layer synchronous nodes can be set as parameter producers, the service line synchronous completion condition is set as parameter values, after the DW layer acquires the parameters of the service line synchronous completion, the plurality of synchronous completion parameters are input into the DW layer for collection, and the data processing work of the corresponding service line is executed according to the first collected data. By solving the problem of dynamic transmission of script parameters, the operation of developers is simplified, tasks are easier to maintain in a configuration mode, the script parameter transmission relation can be checked through a system, the data management work is facilitated, and the technical effect of script code management cost is further reduced.
In summary, the method and the system for transmitting the flow node parameters based on the scheduling system provided by the application have the following technical effects:
Because the configuration management flow chart editing interface is adopted, task flow nodes are added; constructing a first parameter transmission rule according to a first configuration instruction, wherein the first parameter transmission rule is an acquisition rule of a script downstream transmission parameter; obtaining a first script execution instruction; analyzing the task flow node through a regular expression according to the first script execution instruction and the first parameter transmission rule to obtain first analysis parameter information; inputting the first analysis parameter information into a configuration management module, and obtaining first real-time parameter information of a first real-time flow node according to the configuration management module; and transmitting the first real-time parameter information to a script at the downstream of the task flow node as an execution parameter. The configuration interface is further provided, the user configures parameter transmission rules through the interface, the node dynamic parameter transmission is realized, the flow dynamic parameter information can be queried in the system, and the problem of script parameter dynamic transmission is solved; the operation of a developer is simplified, and the task is easier to maintain in a configuration mode; the script parameter transfer relation can be checked through the system, the data management work is facilitated, and the technical effect of script code management cost is reduced.
Example two
Based on the same inventive concept as the method for transmitting the flow node parameters based on the scheduling system in the foregoing embodiment, the present invention further provides a system for transmitting the flow node parameters based on the scheduling system, as shown in fig. 4, where the system includes:
A first adding unit 11, where the first adding unit 11 is configured to add task flow nodes in a configuration management flow chart editing interface;
A first construction unit 12, where the first construction unit 12 is configured to construct a first parameter transmission rule according to a first configuration instruction, where the first parameter transmission rule is an acquisition rule of a script downstream transmission parameter;
A first obtaining unit 13, where the first obtaining unit 13 is configured to obtain a first script execution instruction;
the second obtaining unit 14 is configured to parse the task flow node according to the first parameter transfer rule and a regular expression according to the first script execution instruction by the second obtaining unit 14, so as to obtain first parsed parameter information;
The third obtaining unit 15 is configured to input the first analysis parameter information into a configuration management module, and obtain first real-time parameter information of a first real-time flow node according to the configuration management module;
the first transmitting unit 16 is configured to transmit the first real-time parameter information to the script downstream of the task flow node as an execution parameter by using the first transmitting unit 16.
Further, the system further comprises:
The first configuration unit is used for configuring an acquisition mode and a configuration parameter prefix according to the first configuration instruction, wherein the configuration acquisition mode is realized by analyzing script output content, and the configuration parameter prefix is realized by configuring global parameters and local parameters;
the second construction unit is used for constructing the first parameter transmission rule according to the configuration acquisition mode and the configuration parameter prefix.
Further, the system further comprises:
The first connection unit is used for enabling the configuration management module and the script execution module to have a first connection relationship;
The first uploading unit is used for uploading the first analysis parameter information obtained by the script execution module to the configuration management module based on the first connection relation;
the first storage unit is used for storing the first analysis parameter information to redis by the configuration management module.
Further, the system further comprises:
The second storage unit is used for storing the first analysis parameter information by using a preset hash structure;
The third storage unit is used for storing global parameters of the first analysis parameter information according to a first storage mode, wherein the first storage mode is a global field;
and the fourth storage unit is used for storing the local parameters of the first analysis parameter information according to a second storage mode, wherein the second storage mode is a json mode.
Further, the system further comprises:
A fourth obtaining unit, configured to obtain first service data according to synchronous access service line data of a data interface layer, where the data interface layer includes a plurality of synchronization nodes;
a fifth obtaining unit, configured to obtain a plurality of synchronization completion parameters according to the plurality of synchronization nodes;
The sixth obtaining unit is used for summarizing the input data summarizing layers of the synchronous completion parameters to obtain first summarized data;
and the first execution unit is used for executing data processing of the corresponding service line according to the first summarized data.
The foregoing various modifications and specific examples of the method for transmitting a flow node parameter based on a scheduling system in the first embodiment of fig. 1 are equally applicable to a system for transmitting a flow node parameter based on a scheduling system in this embodiment, and those skilled in the art will clearly know the implementation method of a system for transmitting a flow node parameter based on a scheduling system in this embodiment through the foregoing detailed description of the method for transmitting a flow node parameter based on a scheduling system, so that they will not be described in detail herein for brevity of description.
In addition, the application also provides an electronic device, which comprises a bus, a transceiver, a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the transceiver, the memory and the processor are respectively connected through the bus, and when the computer program is executed by the processor, the processes of the method embodiment for controlling output data are realized, and the same technical effects can be achieved, so that repetition is avoided and redundant description is omitted.
Exemplary electronic device
In particular, referring to FIG. 5, the present application also provides an electronic device comprising a bus 1110, a processor 1120, a transceiver 1130, a bus interface 1140, a memory 1150, and a user interface 1160.
In the present application, the electronic device further includes: computer programs stored on the memory 1150 and executable on the processor 1120, which when executed by the processor 1120, implement the various processes of the method embodiments described above for controlling output data.
A transceiver 1130 for receiving and transmitting data under the control of the processor 1120.
In the present application, bus architecture (represented by bus 1110), bus 1110 may include any number of interconnected buses and bridges, with bus 1110 connecting various circuits, including one or more processors, represented by processor 1120, and memory, represented by memory 1150.
Bus 1110 represents one or more of any of several types of bus structures, including a memory bus and memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. By way of example, and not limitation, such an architecture includes: industry standard architecture buses, micro-channel architecture buses, expansion buses, video electronics standards association, and peripheral component interconnect buses.
Processor 1120 may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments may be implemented by instructions in the form of integrated logic circuits in hardware or software in a processor. The processor includes: general purpose processors, central processing units, network processors, digital signal processors, application specific integrated circuits, field programmable gate arrays, complex programmable logic devices, programmable logic arrays, micro control units or other programmable logic devices, discrete gates, transistor logic devices, discrete hardware components. The methods, steps and logic blocks disclosed in the present application may be implemented or performed. For example, the processor may be a single-core processor or a multi-core processor, and the processor may be integrated on a single chip or located on multiple different chips.
The processor 1120 may be a microprocessor or any conventional processor. The method steps disclosed in connection with the present application may be performed directly by a hardware decoding processor or by a combination of hardware and software modules in a decoding processor. The software modules may be located in random access memory, flash memory, read only memory, programmable read only memory, erasable programmable read only memory, registers, and the like, as known in the art. The readable storage medium is located in a memory, and the processor reads the information in the memory and, in combination with its hardware, performs the steps of the above method.
Bus 1110 may also connect together various other circuits such as peripheral devices, voltage regulators, or power management circuits, bus interface 1140 providing an interface between bus 1110 and transceiver 1130, all of which are well known in the art. Therefore, the present application will not be further described.
The transceiver 1130 may be one element or a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. For example: the transceiver 1130 receives external data from other devices, and the transceiver 1130 is configured to transmit the data processed by the processor 1120 to the other devices. Depending on the nature of the computer device, a user interface 1160 may also be provided, for example: touch screen, physical keyboard, display, mouse, speaker, microphone, trackball, joystick, stylus.
It should be appreciated that in the present application, the memory 1150 may further include memory located remotely from the processor 1120, which may be connected to a server through a network. One or more portions of the above-described networks may be an ad hoc network, an intranet, an extranet, a virtual private network, a local area network, a wireless local area network, a wide area network, a wireless wide area network, a metropolitan area network, an internet, a public switched telephone network, a plain old telephone service network, a cellular telephone network, a wireless fidelity network, and combinations of two or more of the foregoing. For example, the cellular telephone network and wireless network may be global system for mobile communications devices, code division multiple access devices, worldwide interoperability for microwave access devices, general packet radio service devices, wideband code division multiple access devices, long term evolution devices, LTE frequency division duplex devices, LTE time division duplex devices, advanced long term evolution devices, general mobile communications devices, enhanced mobile broadband devices, mass machine class communications devices, ultra-reliable low-latency communications devices, and the like.
It should be appreciated that the memory 1150 in the present application can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. Wherein the nonvolatile memory includes: read-only memory, programmable read-only memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, or flash memory.
The volatile memory includes: random access memory, which serves as an external cache. By way of example, and not limitation, many forms of RAM are available, such as: static random access memory, dynamic random access memory, synchronous dynamic random access memory, double data rate synchronous dynamic random access memory, enhanced synchronous dynamic random access memory, synchronous link dynamic random access memory, and direct memory bus random access memory. The memory 1150 of the electronic device described herein includes, but is not limited to, the memory described above and any other suitable type of memory.
In the present application, memory 1150 stores the following elements of operating system 1151 and application programs 1152: an executable module, a data structure, or a subset thereof, or an extended set thereof.
Specifically, the operating system 1151 includes various device programs, such as: a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The applications 1152 include various applications such as: and the media player and the browser are used for realizing various application services. A program for implementing the method of the present application may be included in the application 1152. The application 1152 includes: applets, objects, components, logic, data structures, and other computer apparatus-executable instructions that perform particular tasks or implement particular abstract data types.
In addition, the application also provides a computer readable storage medium, on which a computer program is stored, where the computer program when executed by a processor implements each process of the above-mentioned method embodiment for controlling output data, and the same technical effects can be achieved, and for avoiding repetition, a detailed description is omitted herein.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the application is subject to the protection scope of the claims.
Claims (8)
1. A scheduling system based flow node parameter transfer method, the method comprising:
adding task flow nodes in a configuration management flow chart editing interface;
Constructing a first parameter transmission rule according to a first configuration instruction, wherein the first parameter transmission rule is an acquisition rule of a script downstream transmission parameter;
The constructing a first parameter transfer rule according to the first configuration instruction includes: according to the first configuration instruction, configuring an acquisition mode and a configuration parameter prefix, wherein the configuration acquisition mode is realized by analyzing script output content, and the configuration parameter prefix is realized by configuring global parameters and local parameters; constructing the first parameter transfer rule according to the configuration acquisition mode and the configuration parameter prefix;
obtaining a first script execution instruction;
analyzing the task flow node through a regular expression according to the first script execution instruction and the first parameter transmission rule to obtain first analysis parameter information;
Inputting the first analysis parameter information into a configuration management module, and obtaining first real-time parameter information of a first real-time flow node according to the configuration management module;
and transmitting the first real-time parameter information to the downstream script of the task flow node as an execution parameter.
2. The method of claim 1, wherein the method comprises:
the configuration management module and the script execution module have a first connection relationship;
uploading the first analysis parameter information obtained by the script execution module to the configuration management module based on the first connection relation;
The configuration management module stores the first analysis parameter information to redis.
3. The method according to claim 2, wherein the method comprises:
The redis uses a preset hash structure to store the first analysis parameter information;
Storing global parameters of the first analysis parameter information according to a first storage mode, wherein the first storage mode is a global field;
and storing the local parameters of the first analysis parameter information according to a second storage mode, wherein the second storage mode is a json mode.
4. The method of claim 3 wherein the flow local variable has a node name as field and a value of field as parameter json; the sub-flow local variable takes the name of the sub-flow and the name of the node as field, and the value of the field is taken as parameter json.
5. The method of claim 1, wherein the method further comprises:
According to the synchronous access service line data of the data interface layer, first service data are obtained, wherein the data interface layer comprises a plurality of synchronous nodes;
obtaining a plurality of synchronization completion parameters according to the plurality of synchronization nodes;
summarizing the plurality of synchronous completion parameter input data summarization layers to obtain first summarized data;
and executing data processing of the corresponding service line according to the first summarized data.
6. A scheduling system based flow node parameter delivery system, the system comprising:
The first adding unit is used for adding task flow nodes on a configuration management flow chart editing interface;
The first construction unit is used for constructing a first parameter transmission rule according to a first configuration instruction, wherein the first parameter transmission rule is an acquisition rule of a script downstream transmission parameter;
The first configuration unit is used for configuring an acquisition mode and a configuration parameter prefix according to the first configuration instruction, wherein the configuration acquisition mode is realized by analyzing script output content, and the configuration parameter prefix is realized by configuring global parameters and local parameters;
the second construction unit is used for constructing the first parameter transmission rule according to the configuration acquisition mode and the configuration parameter prefix;
the first obtaining unit is used for obtaining a first script execution instruction;
the second obtaining unit is used for carrying out instructions according to the first script, analyzing the task flow nodes through regular expressions according to the first parameter transmission rule, and obtaining first analysis parameter information;
The third obtaining unit is used for inputting the first analysis parameter information into a configuration management module and obtaining first real-time parameter information of a first real-time flow node according to the configuration management module;
And the first transmission unit is used for transmitting the first real-time parameter information to the downstream script of the task flow node as an execution parameter.
7. A scheduling system flow node parameter based delivery electronic device comprising a bus, a transceiver, a memory, a processor and a computer program stored on the memory and executable on the processor, the transceiver, the memory and the processor being connected by the bus, characterized in that the computer program when executed by the processor implements the steps of the method according to any of claims 1-5.
8. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210022798.4A CN114371808B (en) | 2022-01-10 | 2022-01-10 | Scheduling system-based flow node parameter transmission method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210022798.4A CN114371808B (en) | 2022-01-10 | 2022-01-10 | Scheduling system-based flow node parameter transmission method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114371808A CN114371808A (en) | 2022-04-19 |
| CN114371808B true CN114371808B (en) | 2024-09-10 |
Family
ID=81144719
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210022798.4A Active CN114371808B (en) | 2022-01-10 | 2022-01-10 | Scheduling system-based flow node parameter transmission method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114371808B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106600238A (en) * | 2016-12-26 | 2017-04-26 | 上海轻维软件有限公司 | Network service fulfillment realization method based on Activiti flow engine and apparatus thereof |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7539974B2 (en) * | 2003-10-24 | 2009-05-26 | Microsoft Corporation | Scalable synchronous and asynchronous processing of monitoring rules |
| CN107315576A (en) * | 2016-04-26 | 2017-11-03 | 中兴通讯股份有限公司 | A kind of method and system of dynamic expansion software flow |
| CN107766132B (en) * | 2017-06-25 | 2019-03-15 | 平安科技（深圳）有限公司 | Multi-task scheduling method, application server and computer readable storage medium |
| CN107332713A (en) * | 2017-08-10 | 2017-11-07 | 上海新炬网络技术有限公司 | A kind of traffic failure engine of positioning implementation method based on script |
| CN109542556B (en) * | 2018-10-30 | 2022-04-15 | 珠海伟诚科技股份有限公司 | Method and system for interaction between process and form based on Activiti |
| CN109710413B (en) * | 2018-12-29 | 2020-09-08 | 重庆誉存大数据科技有限公司 | Integral calculation method of rule engine system of semi-structured text data |
| CN110727670B (en) * | 2019-10-11 | 2022-08-09 | 北京小向创新人工智能科技有限公司 | Data structure prediction transfer and automatic data processing method based on flow chart |
| CN113760343A (en) * | 2020-06-05 | 2021-12-07 | 北京沃东天骏信息技术有限公司 | Method and device for processing service request |
| CN112330180A (en) * | 2020-11-17 | 2021-02-05 | 深圳市智象科技有限公司 | Automatic operating system with flexible configuration |
| CN112732680A (en) * | 2021-01-22 | 2021-04-30 | 浪潮卓数大数据产业发展有限公司 | Data warehouse design method |
| CN113434216A (en) * | 2021-06-29 | 2021-09-24 | 烽火通信科技股份有限公司 | Scene-based command line construction method and system |
| CN113590297A (en) * | 2021-08-04 | 2021-11-02 | 亚联（天津）信息技术有限责任公司 | Method and equipment for transmitting process parameters back and forth |
-
2022
- 2022-01-10 CN CN202210022798.4A patent/CN114371808B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106600238A (en) * | 2016-12-26 | 2017-04-26 | 上海轻维软件有限公司 | Network service fulfillment realization method based on Activiti flow engine and apparatus thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114371808A (en) | 2022-04-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7170779B2 (en) | Methods and systems for automatic intent mining, classification, and placement | |
| CN108304201B (en) | Object updating method, device and equipment | |
| CN112148356B (en) | Document generation method, interface development method, device, server and storage medium | |
| CN110351325B (en) | Data processing method and related equipment | |
| EP2564318A1 (en) | Data center operation | |
| CN111241182A (en) | Data processing method and apparatus, storage medium, and electronic apparatus | |
| CN110209700B (en) | Data stream association method and device, electronic equipment and storage medium | |
| CN108694221B (en) | Data real-time analysis method, module, equipment and device | |
| CN110580189A (en) | method and device for generating front-end page, computer equipment and storage medium | |
| CN114116065A (en) | Method and device for acquiring topological graph data object and electronic equipment | |
| CN112154420A (en) | Automatic intelligent cloud service testing tool | |
| CN111949850A (en) | Multi-source data acquisition method, device, equipment and storage medium | |
| JP2023036681A (en) | Task processing method, processing device, electronic equipment, storage medium, and computer program | |
| CN111782730A (en) | File uploading method and device and storage medium | |
| CN111369237A (en) | Data processing method and device and computer storage medium | |
| CN114371808B (en) | Scheduling system-based flow node parameter transmission method and system | |
| WO2021129812A1 (en) | Method and system for running artificial intelligence application, and device | |
| KR20210000041A (en) | Method and apparatus for analyzing log data in real time | |
| CN113138767B (en) | Code language conversion method, device, electronic equipment and storage medium | |
| CN114996557B (en) | Service stability determination method, device, equipment and storage medium | |
| CN117008920A (en) | Engine system, request processing method and device, computer equipment and storage medium | |
| CN109542986B (en) | Element normalization method, device, equipment and storage medium of network data | |
| CN115378824B (en) | Model similarity determination method, device, equipment and storage medium | |
| US12132804B2 (en) | Runtime module conversion | |
| CN115563183B (en) | Query method, query device and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 100000 floors 1-3, block a, global creative Plaza, No. 10, Furong street, Chaoyang District, Beijing Applicant after: Bairong Zhixin (Beijing) Technology Co.,Ltd. Address before: 100000 floors 1-3, block a, global creative Plaza, No. 10, Furong street, Chaoyang District, Beijing Applicant before: Bairong Zhixin (Beijing) credit investigation Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |