CN111914010B - Method, device, equipment and storage medium for processing business - Google Patents
Method, device, equipment and storage medium for processing business Download PDFInfo
- Publication number
- CN111914010B CN111914010B CN202010771528.4A CN202010771528A CN111914010B CN 111914010 B CN111914010 B CN 111914010B CN 202010771528 A CN202010771528 A CN 202010771528A CN 111914010 B CN111914010 B CN 111914010B
- Authority
- CN
- China
- Prior art keywords
- execution
- business process
- business
- execution sequence
- level
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 377
- 238000012545 processing Methods 0.000 title abstract description 39
- 230000008569 process Effects 0.000 claims abstract description 333
- 239000011159 matrix material Substances 0.000 claims description 57
- 230000015654 memory Effects 0.000 claims description 19
- 238000007639 printing Methods 0.000 claims description 10
- 238000004458 analytical method Methods 0.000 claims description 8
- 238000012795 verification Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 9
- 239000008280 blood Substances 0.000 description 7
- 210000004369 blood Anatomy 0.000 description 7
- 238000004590 computer program Methods 0.000 description 7
- 238000013500 data storage Methods 0.000 description 6
- 230000003993 interaction Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000003139 buffering effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/1734—Details of monitoring file system events, e.g. by the use of hooks, filter drivers, logs
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a method, a device, equipment and a storage medium for processing business, and relates to the fields of big data, information flow and the like. The specific implementation scheme is as follows: determining a first execution sequence of each business process contained in the target business and a dependency relationship among each business process; the first execution sequence comprises serial execution of each business process; according to the dependency relationship, the first execution sequence is adjusted to obtain a second execution sequence; the second execution sequence comprises serial and parallel execution of each business process; and executing each business process according to the second execution sequence. Through the scheme, the dependency relationship among the business processes in the target business is determined, the execution efficiency is improved by changing the execution sequence of the business processes, and the method and the system can be used for the cloud computing server.
Description
Technical Field
The present application relates to the field of data processing, and in particular, to the fields of big data, information flow, and the like.
Background
With the continuous development and application of internet technology, massive data and various data computing engines are generated. The data computation engine is a component used to compute data. In order to Extract valuable data from the mass data, a large number of data ETL (Extract-Transform-Load) services need to be developed, and these ETL services are run on line according to a certain period. Among them, ETL is used to describe the process of extracting (extracting), converting (transforming), and loading (load) data from a data source to a destination data source.
Currently, main-stream ETL services are developed based on SQL, the internal logic of the ETL services is complex, the related data volume is extremely large, and one-time execution can take hours or even days, so that the service processing efficiency is low.
Disclosure of Invention
The application provides a method, a device, equipment and a storage medium for processing business.
According to an aspect of the present application, there is provided a method for service processing, including the steps of:
determining a first execution sequence of each business process contained in the target business and a dependency relationship among each business process; the first execution sequence comprises serial execution of each business process;
according to the dependency relationship, the first execution sequence is adjusted to obtain a second execution sequence; the second execution sequence comprises serial and parallel execution of each business process;
and executing each business process according to the second execution sequence.
According to another aspect of the present application, there is provided an apparatus for service processing, including:
the business process attribute analysis module is used for determining a first execution sequence of each business process contained in the target business and a dependency relationship among each business process; the first execution sequence comprises serial execution of each business process;
The execution sequence adjusting module is used for adjusting the first execution sequence according to the dependency relationship to obtain a second execution sequence; the second execution sequence comprises serial and parallel execution of each business process;
and the business process execution module is used for executing each business process according to the second execution sequence.
According to a third aspect of the present application, an embodiment of the present application provides an electronic device, including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the methods provided by any one of the embodiments of the present application.
According to a fourth aspect of the present application, embodiments of the present application provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method provided by any of the embodiments of the present application.
According to a fifth aspect of the present application, embodiments of the present application provide a computer program product comprising a computer program which, when executed by a processor, implements a method as described above.
According to the method and the device for determining the dependency relationship of the business processes, the problem of low execution efficiency of the business processes is solved, the dependency relationship among the business processes in the target business is determined, and the execution efficiency is improved by changing the execution sequence of the business processes.
It should be understood that the description of this section is not intended to identify key or critical features of the embodiments of the application or to delineate the scope of the application. Other features of the present application will become apparent from the description that follows.
Drawings
The drawings are for better understanding of the present solution and do not constitute a limitation of the present application. Wherein:
FIG. 1 is a flow chart of a method of business processing according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a business process according to an embodiment of the present application;
FIG. 3 is a flow chart of a second execution order according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a business process execution matrix according to an embodiment of the present application;
FIG. 5 is a flow chart of a second execution order according to an embodiment of the present application;
FIG. 6 is a schematic diagram of a business process execution matrix according to an embodiment of the present application;
FIG. 7 is a schematic diagram of a business process execution matrix according to an embodiment of the present application;
FIG. 8 is a flow chart of a process of performing business according to an embodiment of the present application;
FIG. 9 is a schematic diagram of an apparatus for traffic handling according to an embodiment of the present application;
FIG. 10 is a schematic diagram of a business processing system architecture according to an embodiment of the present application;
FIG. 11 is a flow chart of business processing according to a business processing system architecture according to an embodiment of the present application;
fig. 12 is a block diagram of an electronic device for implementing a method of business processing of an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
As shown in fig. 1, in an embodiment of the present application, a method for service processing is provided, which may include the following steps:
s101: determining a first execution sequence of each business process contained in the target business and a dependency relationship among each business process; the first execution sequence comprises serial execution of each business process;
S102: according to the dependency relationship, the first execution sequence is adjusted to obtain a second execution sequence; the second execution sequence comprises serial and parallel execution of each business process;
s103: and executing each business process according to the second execution sequence.
In one example, the target business may be a static code consisting of one or more business processes with explicit execution tasks. For example, the target service may be an xml (extended markup language, extensible markup language) file, and the service process may be an ETL (Extract-Transform-Load) process.
In the case where there is a dependency relationship between business processes, the business process in which there is a dependency relationship may be referred to as a business process having a blood-related relationship.
As shown in connection with fig. 2, for example, the target service is a service for acquiring winning team information. According to the analysis of the target service, it can be determined that the target service includes four service processes, which are respectively: a first business process (corresponding to jobId:1 in fig. 2) for obtaining information of a team; a second process (corresponding to jobId:2 in fig. 2) of acquiring game information; a third business process (corresponding to jobId:3 in fig. 2) of matching the team information with the game information; fourth business process (corresponding jobId:4 in fig. 2) determines winner information.
The execution sequence of the four business processes is serial execution, and the execution sequences of the four business processes can be marked by the execution sequences of the jobId 1 to the jobId 4 respectively. The above described execution order may be referred to as a first execution order.
Further, the dependency analysis of the business process can determine: the third business process needs to rely on the data processing results of the first business process and the second process, and the fourth business process needs to rely on the data processing results of the third business process. Based on this, a data dependency tree can be constructed. The data dependency tree may be constructed from column-level data blood-edges or from table-level data blood-edges.
For example, a third business process may be referred to as a parent node of the first business process and the second business process, and a fourth business process may be referred to as a parent node of the third business process, as determined from the data dependency tree.
After the dependency relationship is determined according to the data dependency tree, the first execution sequence can be adjusted based on the dependency relationship. For example, the first and second business processes may be adapted to be performed in parallel, followed by the third and fourth business processes being performed in series. The adjusted execution order may be referred to as a second execution order.
And executing the four business processes according to the second execution sequence.
Through the adjustment, each business process executed by default in series can be adjusted by utilizing the dependency relationship, so as to obtain a second business process. The second business process includes serial execution and parallel execution.
Each business process may be parsed by the data calculation engine into a plurality of jobs, which are executed serially. The data computation engine may be Spark, hadoop MR, flink, storm, etc.
Each job, when executed, splits into multiple jobs (jobs). The actual execution process of each job in the executor (executor) is called task (task), and the multiple tasks can be executed in parallel or executed in series. The executor performs various tasks by means of its logical core (core).
The data compute engine will send the task to the executor where the idle logical core exists. Because the total logic core time slices consumed by executing each business process are fixed, the utilization rate of the logic core can be improved by reducing the idle rate of the logic core in the executor in the process of the task alternate operation corresponding to each parallel business process, thereby reducing the total operation time consumption.
That is, through the above scheme, the total time consumption of executing each business process by using the second execution sequence is greatly reduced compared with the total time consumption of the first execution sequence, so that the operation performance of the business is improved. The conventional manner generally splits one target service into multiple target sub-services, but the conventional manner needs to manually comb the target sub-service dependency relationship and the running sequence of each target sub-service. Once business logic is revised, it is difficult to operate and maintain. The above embodiment of the present application determines the dependency relationship between each business process in the target business, and improves the execution efficiency by changing the execution sequence of each business process.
As shown in connection with fig. 3, in one embodiment, step S102 may include:
s1021: determining a dependency level between business processes according to the dependency relationship;
s1022: establishing a business process execution matrix, wherein an execution level in the business process execution matrix corresponds to a dependency level;
s1023: according to the dependency level and the first execution sequence, placing the business process in an execution level corresponding to the dependency level to which the business process belongs;
s1024: a second execution order is determined according to the execution hierarchy.
As shown in connection with fig. 2, the first business process is to obtain team information from the external data storage component instead of information generated in the target business, so the external data storage component can act as a leaf node. The nodes, e.g., jobId:0 in fig. 2, are all leaf nodes.
The first business process may be a parent node of the leaf node and the first business process may be a first dependency level.
The second business process is also race information obtained from the external data storage component, and thus the external data storage component may act as a leaf node. The second process may act as a parent node for the leaf node. Similarly, the second business process may act as a first dependency level.
The third business process is used as a father node of the corresponding nodes of the first business process and the second business process. The third business process may act as a second dependency level.
The fourth business process is used as a father node of the node corresponding to the third business process. The fourth business process may act as a third dependency level.
As shown in connection with fig. 2 and 4, a business process execution matrix is established, in which execution levels correspond to dependency levels.
And placing the business process in an execution hierarchy corresponding to the dependence hierarchy to which the business process belongs according to the dependence hierarchy and the first execution sequence. That is, the first business process and the second business process are sequentially placed at a first execution level (depth: 1 in fig. 4) of the business process execution matrix, the third business process is placed at a second execution level (depth: 2 in fig. 4) of the business process execution matrix, and the fourth business process is placed at a third execution level (depth: 3 in fig. 4) of the business process execution matrix.
The first business process and the second business process may be sequenced in the first execution hierarchy according to the first execution order. In addition, when repetition occurs, the business process needs to be subjected to de-duplication processing. For example, the target service includes first to fifth total five service processes. Based on the dependency level, it may be determined that the third business process is a parent node of the first and second business processes and the fifth business process is a parent node of the second and fourth business processes. Then the first execution hierarchy includes a first business process, a second business process, and a second business process, a fourth business process when the business process execution matrix is established. Since the second business process occurs twice, it needs to be deduplicated. After the deduplication process, the first execution hierarchy includes first, second, and fourth business processes. The second tier of execution hierarchy includes third and fifth business processes.
In case all business processes have been placed on the execution hierarchy, the second execution order may be determined according to the execution hierarchy, e.g. each execution hierarchy is executed serially, the business processes in each execution hierarchy being executed in parallel. Wherein the number of business processes executed in parallel can be determined according to the number of threads. For example, the number of threads may be determined according to the total number of the actuator logic cores held during the execution of the target service, where the number of threads and the total number of the actuator logic cores are in a positive correlation.
By the scheme, the second execution sequence can be determined by using the execution hierarchy in the business process execution matrix, so that each business process is executed based on the second execution sequence, and the execution efficiency can be improved.
As shown in fig. 5, in an embodiment, in a case that a predetermined type of service procedure is included in the target service, step S102 may further include:
s1025: acquiring an execution sequence number m of a business process of a preset type in a first execution sequence, and a sequence number of an execution level corresponding to each business process with the execution sequence number smaller than m;
s1026: determining a maximum value n in sequence numbers of each execution level;
s1027: moving a business process of a predetermined type from a current execution level to an n+1th execution level;
S1028: acquiring arrangement conditions of execution levels of business processes with execution sequence numbers larger than m;
s1029: according to the arrangement condition of the execution levels, each business process with the execution sequence number larger than m is moved to the execution level after the n+1th execution level of the business process execution matrix according to the original execution level structure;
the execution sequence number m and the maximum value n are positive integers.
The predetermined type of business process may be a business process including a plurality of sub-business processes, a business process for changing the execution order of the subsequent business processes, or the like.
Taking a predetermined type of service as an example of a service procedure for changing the execution order of the subsequent service procedures. As shown in fig. 6, according to the dependency levels determined in steps S1021 to S1024, 16 business processes included in the target business may be respectively placed in four execution levels in the business process execution matrix.
And acquiring an execution sequence number m of the business process of the preset type in the first execution sequence, and a sequence number of an execution level corresponding to each business process with the execution sequence number smaller than m. For example, a dashed box composed of line segments and points shown in fig. 6 may represent a predetermined type of business process having a sequence number m in the first execution order.
The solid line box may represent a business process with a sequence number less than m in the first execution order, collectively denoted as m-.
The dashed box formed by the dots may represent a business process with a sequence number greater than m in the first execution order, collectively denoted as m+.
In fig. 6, the sequence numbers of the execution levels corresponding to the business processes with the execution sequence numbers smaller than m are 1 to 3. It can thus be determined that the maximum value n=3 in the sequence numbers of the execution hierarchy corresponding to each business process whose execution sequence number is smaller than m.
As shown in connection with fig. 7, a predetermined type of business process is moved from the current execution level to the n+1th execution level of the business process execution matrix. That is, the business process with the sequence number m in the first execution order is moved from the first execution level depth:1 to the fourth execution level depth:4.
Next, each business process with the first execution sequence number greater than m is moved to the execution level after the n+1th execution level as a whole according to its respective arrangement of levels in fig. 6.
Namely, each business process with the execution sequence number larger than m in the first execution level depth 1 in fig. 6 is kept to be moved to the n+2th execution level in the original sequence, namely, the fifth execution level depth 5;
moving each business process with the execution sequence number greater than m in the second execution level depth 2 in fig. 6 to the n+3 execution level, namely, a sixth execution level depth 6;
Moving each business process with the execution sequence number greater than m in the third execution level depth:3 in FIG. 6 to the n+4 execution level, namely a seventh execution level depth:7;
and (3) moving each business process with the execution sequence number larger than m in the fourth execution level depth:4 in fig. 6 to the n+5 execution level, namely, the eighth execution level depth:8, in the original sequence.
The adjusted business process execution matrix shown in fig. 7 can be obtained by adjusting the business process execution matrix in fig. 6.
The above scheme can be equivalent to the arrangement of the execution hierarchy of each business process with the execution sequence number larger than m. And according to the arrangement condition, moving the whole business process to an execution level after the n+1th execution level of the business process execution matrix.
Through the scheme, each business process can be ensured to be executed correctly. This is because, after the business process execution matrix is established according to the dependency hierarchy, erroneous execution of the target business may be caused in the case where a predetermined type of business process exists. For example, the purpose of a business process of a predetermined type is to determine whether the current date is a weekend. Ending the execution of the target service under the condition that the current date is the weekend, namely not executing each service process of m+; otherwise, under the condition that the current date is not the weekend, continuing to execute each business process of m+ until the business process of m+ is executed. If the business process execution matrix in fig. 6 is used, after the first execution level is executed, the business process of m-in the second execution level is not executed, and each business process of m+ in the first execution level is already executed, which is obviously wrong.
By adjusting the business process execution matrix in fig. 6 by the method, the adjusted business process execution matrix in fig. 7 is obtained, so that each business process can be executed correctly. The method can ensure that each business process of m-is executed first, and judges through the business process of a preset type with the sequence number of m, and only when the judging result is that the subsequent business process needs to be executed continuously, each business process of the subsequent m+ is executed continuously.
As shown in fig. 8, in one embodiment, step S103 may include:
s1031: each execution level of the serial execution business process execution matrix;
s1032: business processes in each execution hierarchy are executed in parallel.
For each execution level of the business process execution matrix, each execution level of the business process execution matrix may be executed in series. That is, when all the business processes in the first execution hierarchy are executed, the business processes in the second execution hierarchy are executed. Similarly, when all the business processes in the second execution hierarchy are executed, executing the business processes in the third execution hierarchy. And so on until all the execution levels of the business process execution matrix are completely executed.
The business processes in the various execution levels may be performed in parallel. Further, the number of parallel execution of each business process may be determined according to the number of threads.
By the scheme, the parallel execution and serial execution can be adopted to accelerate the execution of each business process.
In one embodiment, the method for service processing may further include the steps of:
according to the first execution sequence, the log information of each business process is printed.
Log (log) information may facilitate a user's query for the execution status or execution status of each business process, etc. For printing of log information, the first execution order may be followed. That is, log information of each business process is sequentially printed in accordance with the serial number of the first execution order.
For example, the target service includes three service processes with the first execution sequence of 1, 2, and 3. Wherein the business processes with sequence numbers 1 and 3 are in a first execution level and the business process with sequence number 2 is in a second execution level.
In the case that the service process with the sequence number 3 is completed before the service process with the sequence number 1 is executed, log information of the service process with the sequence number 3 may be cached. And printing log information of the business process with the serial number of 3 until the business process with the serial number of 2 in the second execution level is completed.
With the above-described arrangement, although each business process is executed in the second execution order, log information of each business process can be printed in the first execution order. And the user can check conveniently.
As shown in fig. 9, an apparatus for service processing according to an embodiment of the present application may include the following components:
a business process attribute analysis module 901, configured to determine a first execution sequence of each business process included in the target business, and a dependency relationship between each business process; the first execution sequence comprises serial execution of each business process;
an execution sequence adjustment module 902, configured to adjust the first execution sequence according to the dependency relationship to obtain a second execution sequence; the second execution sequence comprises serial and parallel execution of each business process;
the business process execution module 903 is configured to execute each business process according to the second execution order.
In one embodiment, the execution sequence adjustment module 902 may include:
a dependency level analysis submodule 9021 for determining a dependency level between business processes according to the dependency relationship;
a business process execution matrix building sub-module 9022, configured to build a business process execution matrix, where an execution level in the business process execution matrix corresponds to a dependency level;
A business process placement sub-module 9023, configured to place a business process in an execution hierarchy corresponding to a dependency hierarchy to which the business process belongs according to the dependency hierarchy and the first execution order;
a second execution order determination submodule 9024 for determining a second execution order according to the execution hierarchy.
In an embodiment, in a case that the target service includes a predetermined type of service procedure, the execution sequence adjustment module 902 may further include:
the business process information obtaining sub-module 9025 is configured to obtain an execution sequence number m of a predetermined type of business process in the first execution sequence, and a sequence number of an execution hierarchy corresponding to each business process with an execution sequence number smaller than m;
a serial number maximum value determining submodule 9026, configured to determine a maximum value n in serial numbers of each execution hierarchy;
a first movement submodule 9027 for moving a predetermined type of business process from a current execution level to an n+1th execution level;
an execution hierarchy arrangement condition obtaining sub-module 9028, configured to obtain an arrangement condition of execution hierarchies of each business process with an execution sequence number greater than m;
the second moving submodule 9029 is used for moving each business process with the execution sequence number larger than m into an execution level after the n+1th execution level of the business process execution matrix according to the original execution level structure according to the arrangement condition of the execution levels;
The execution sequence number m and the maximum value n are positive integers.
In one embodiment, business process execution module 903 may comprise:
a first execution submodule 9031, configured to serially execute each execution level of the business process execution matrix;
a second execution submodule 9032 is configured to execute the business processes in each execution hierarchy in parallel.
In one embodiment, the apparatus for service processing further includes:
and the log information printing sub-module is used for printing log information of each business process according to the first execution sequence.
Fig. 10 is a schematic diagram of a service processing system according to an embodiment of the present application, and this embodiment provides a preferred example based on the foregoing embodiment. As shown in fig. 10, a computing device carrying business processing functions may include an interaction layer, an engine layer, and a data source. Various ways in which a user may interact with a computing device are provided in the interaction layer, which may include, for example, but not limited to, webSocket, http/HTTPS, TCP, CLI, and the like. The data source comprises various data storage components which can be accessed through SQL, such as MySQL, hive, elastic search and the like, and can also comprise components which support SQL access in a native way, components which support SQL access indirectly by means of a third party component and the like.
The engine layer comprises a service container, and further comprises a service configuration file, a resource configuration file and an executor; the actuators may include, but are not limited to: the system comprises a service operation module, a data blood-source module, a view module, a log module, a monitoring module and other functional modules.
In this embodiment, the user may submit the target service processing request to the computing device (further to the service container) through an interaction means provided in the interaction layer, such as a Command-Line Interface (CLI) means.
The service container obtains a target service processing request submitted by a user, and verifies the validity of information included in the target service processing request, for example, whether the target service exists or not is identified according to the target service identifier. In the case that the validity verification of the information included in the target service abnormal location request is passed, the executor performs the target service configuration information verification, such as verifying whether the database of the target service exists, whether the connection is possible, and the like. And under the condition that the target service configuration information passes the verification, executing the loading service configuration file, the resource configuration file and the like, starting the data blood-edge module to analyze the logic code of the target service, and determining the blood-edge relation of the target service.
The view module determines a data dependency tree and a business process execution matrix according to the blood relationship.
And the business operation module performs the execution of the business process according to the execution hierarchy in the business process execution matrix.
The log module is used for printing execution conditions of each business process.
As shown in fig. 11, the method for implementing service processing by the above system may include the following steps:
(1) And the service container performs parameter verification. And carrying out configuration verification by an actuator under the condition that the parameter verification passes.
The service container receives a data processing service operation instruction of a user. And performing parameter verification on the received data processing service operation instruction to determine the executable performance of the instruction. The data processing service operation instructions may include user information and/or service configuration files, etc.
The parameter verification may include verification of user rights, verification of integrity of configuration information in a service profile, or the like.
In the case where a service profile is not included in the data processing service execution instruction, there may be a case where the profile has been stored in advance in the service container. The configuration check may check whether the service profile has been stored in the service container or whether the source or destination database declared in the service profile is included in the resource profile, etc.
(2) The executor loads the configuration file.
And under the condition that the inspection is passed, the executor loads the ETL service configuration file and the resource configuration file.
The loading of the ETL service profile by the executor may include: and acquiring an ETL service configuration file corresponding to the data processing service operation instruction from the service container, and reading the ETL service configuration file.
The executor loads the resource configuration file.
(3) The service operation module performs preparation work.
(4) The data blood-edge module analyzes the SQL-level data blood-edges of all ETL processes in the service.
Because the data sources supported by embodiments of the present application must be components that can directly or indirectly support access through SQL. Thus, all business logic is expressed in SQL. The logic execution plan corresponding to the ETL service can be resolved by an SQL resolver built in the Spark data calculation engine, and the logic execution plan comprises the following steps: fields, input table names, output table names, filter conditions, etc.
(5) And concatenating the SQL-level data blood edges to form the intra-service table-level blood edges.
Through proper data organization, the dependency relationship of SQL level fields and tables can be obtained. That is, the data of the corresponding field or table depends on which tables and fields are generated, i.e., SQL level column level data blood edges, table level data blood edges. And one ETL service is formed by connecting a plurality of SQL (structured query language) in series, so that the service level column-level data blood-edge relationship, the table-level data blood-edge relationship and the like of the ETL service can be obtained by connecting the data blood-edge of each SQL in series. The data blood-edge relationships may constitute a table-level dependency tree, i.e., table-level blood-edges within a business.
(6) And according to the table-level blood edges in the service, all ETL processes in the service are arranged into an execution matrix.
And analyzing a plurality of ETL processes in one ETL service by utilizing a scheduling algorithm of the ETL processes, and converting each ETL process into an execution matrix according to the table-level data blood-edge relationship.
The data of the leaf nodes of the table level dependency tree are all stored in the data storage component and are not the data inserted by the current service. For a leaf node, the value of its execution hierarchy (depth) is artificially marked as 0 (not as part of the execution matrix).
Setting the execution hierarchy value of the parent node of each leaf node to 1; setting the execution level value of the parent node of the node with the execution level value of 1 to 2; setting the execution level value of the parent node of the node with the execution level value of 2 to 3; similarly, the execution level value of the parent node of each level node is 1 greater than the execution level value of its child node.
And performing deduplication on the ETL processes corresponding to the nodes with the same execution level values according to the first execution sequence to form an ETL process execution matrix. The ETL processes within each execution hierarchy are ordered sequentially according to a first execution order.
For special ETL processes (such as ETL processes including multiple ETL sub-processes, ETL processes capable of changing the execution effect of subsequent ETL processes), matrix splitting needs to be performed. Let the first execution order of a particular ETL process be m. m-represents the ETL process with the first execution sequence smaller than m, m+ represents the ETL process with the first execution sequence larger than m, and the matrix splitting process is as follows:
and dividing the ETL process in the execution matrix into three parts of which the first execution sequence is smaller than m, m and larger than m based on m. And (3) removing the ETL process parts with m and more than m from the ETL process execution matrix. The special ETL procedure (m) is inserted at the end of the execution matrix. And (3) completely inserting the matrix of the ETL process larger than m to the tail of the execution matrix to form a new execution matrix.
(7) Executing according to the execution matrix.
Each execution level of the execution matrix is run serially in turn.
And submitting each ETL process in the current execution hierarchy to Spark one by one through the thread pool. Spark performs the ETL processes on the submission in parallel. In the execution process, log printing requirements of each ETL process are continuously submitted.
(8) The log module judges whether log of the last ETL process is printed, and in the printed condition, the log of the current ETL process is printed in sequence, and log of the subsequent ETL process is continuously printed. In the event that the log of the last ETL process was not printed, the log of the current ETL process is cached in order. The ordered cache may be cached in a first order of execution.
(9) And the service operation module judges whether each ETL process of the current execution level is executed, and if not, the operation is continued.
(10) Under the condition that the execution is finished, the log module marks the log cache completion of the ETL process after the execution is finished.
The log module judges whether log of the last ETL process is printed or not, and prints the current ETL process according to log sequence and marks the current ETL process under the condition that the log is printed. In the case that log of the last ETL process has not been printed, it is necessary to continue waiting for the operation result of the service operation module.
In the case that log buffering of the next ETL process is completed, the log of the next ETL process is printed, and the completion of printing is marked.
(11) And the service operation module judges whether the ETL service is executed completely under the condition that all ETL processes in the current execution level are executed completely, namely judges whether all ETL processes in the execution level are executed completely, and executes the next level under the condition that the ETL processes in the current execution level are not executed completely. And under the condition that the execution is finished, ending the execution process of the ETL service.
According to embodiments of the present application, there is also provided an electronic device, a readable storage medium and a computer program product.
As shown in fig. 12, a block diagram of an electronic device according to a method of service processing according to an embodiment of the present application is shown. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the application described and/or claimed herein.
As shown in fig. 12, the electronic device includes: one or more processors 1210, memory 1220, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the electronic device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In other embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple electronic devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 1210 is illustrated in fig. 12.
Memory 1220 is a non-transitory computer-readable storage medium provided herein. Wherein the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the methods of business processes provided herein. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to perform the method of business processing provided by the present application.
The memory 1220 is used as a non-transitory computer readable storage medium, and may be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as program instructions/modules (e.g., the business process attribute analysis module 901, the execution sequence adjustment module 902, and the business process execution module 903 shown in fig. 9) corresponding to the business process method in the embodiments of the present application. The processor 1210 performs various functional applications of the server and data processing, i.e., a method of implementing service processing in the above-described method embodiment, by running non-transitory software programs, instructions, and modules stored in the memory 1220.
Memory 1220 may include a storage program area that may store an operating system, at least one application program required for functions, and a storage data area; the storage data area may store data created according to the use of the electronic device of the method of business processing, and the like. In addition, memory 1220 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 1220 may optionally include memory remotely located from processor 1210, which may be connected to the electronic device of the method of business processing through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device of the method for processing a service may further include: an input device 1230 and an output device 1240. Processor 1210, memory 1220, input device 1230, and output device 1240 may be connected by a bus or other means, for example in fig. 12.
The input device 1230 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device for the method of business process, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointer stick, one or more mouse buttons, a track ball, a joystick, etc. The output means 1240 may include a display device, auxiliary lighting means (e.g., LEDs), tactile feedback means (e.g., vibration motors), and the like. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device may be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASIC (application specific integrated circuit), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
These computing programs (also referred to as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, provided that the desired results of the technical solutions disclosed in the present application can be achieved, and are not limited herein.
The above embodiments do not limit the scope of the application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application are intended to be included within the scope of the present application.
Claims (12)
1. A method of traffic handling, comprising:
determining a first execution sequence of each business process contained in a target business and a dependency relationship among the business processes; the first execution sequence comprises serial execution of each business process;
according to the dependency relationship, the first execution sequence is adjusted to obtain a second execution sequence; the second execution sequence comprises serial and parallel execution of each business process;
Executing the business processes according to the second execution sequence;
the step of adjusting the first execution sequence according to the dependency relationship to obtain a second execution sequence includes:
determining an execution hierarchy corresponding to each business process according to the dependency relationship and the first execution sequence, determining the second execution sequence according to the execution hierarchy, and in the case that the target business contains a business process of a preset type, moving the business process of the preset type from a current execution hierarchy to a next execution hierarchy n+1 of each business process of which the execution sequence is smaller than m and corresponds to the highest execution hierarchy according to an execution sequence number m of the business process of the preset type in the first execution sequence, acquiring arrangement conditions of the execution hierarchies of each business process of which the execution sequence is larger than m, and integrally moving each business process of which the execution sequence is larger than m to the execution hierarchy after the n+1 execution hierarchy based on the arrangement conditions, wherein m and n are positive integers.
2. The method of claim 1, wherein the adjusting the first execution order according to the dependency relationship to obtain a second execution order comprises:
Determining a dependency level between business processes according to the dependency relationship;
establishing a business process execution matrix, wherein an execution level in the business process execution matrix corresponds to the dependency level;
according to the dependency level and the first execution sequence, placing the business process in an execution level corresponding to the dependency level to which the business process belongs;
and determining the second execution sequence according to the execution hierarchy.
3. The method according to claim 2, wherein in the case that the target service includes a service process of a predetermined type, the adjusting the first execution order according to the dependency relationship, to obtain a second execution order, further includes:
acquiring an execution sequence number m of the business process of the preset type in a first execution sequence and a sequence number of an execution level corresponding to each business process with the execution sequence number smaller than m;
determining a maximum value n in the sequence numbers of the execution levels;
moving the business process of the preset type from the current execution level to an n+1th execution level;
acquiring arrangement conditions of execution levels of business processes with execution sequence numbers larger than m;
according to the arrangement condition of the execution levels, each business process with the execution sequence number larger than m is moved to the execution level after the n+1th execution level of the business process execution matrix according to the original matrix structure;
The execution sequence number m and the maximum value n are positive integers.
4. A method according to claim 2 or 3, said executing said business processes according to said second execution order comprising:
serially executing each of the execution levels of the business process execution matrix;
the business processes in each of the execution hierarchies are executed in parallel.
5. A method according to any one of claims 1 to 3, further comprising:
and printing log information of each business process according to the first execution sequence.
6. An apparatus for traffic handling, comprising:
the business process attribute analysis module is used for determining a first execution sequence of each business process contained in the target business and a dependency relationship among the business processes; the first execution sequence comprises serial execution of each business process;
the execution sequence adjusting module is used for adjusting the first execution sequence according to the dependency relationship to obtain a second execution sequence; the second execution sequence comprises serial and parallel execution of each business process;
a business process execution module for executing each business process according to the second execution sequence;
the execution sequence adjustment module is specifically configured to determine an execution hierarchy corresponding to each business process according to the dependency relationship and the first execution sequence, determine the second execution sequence according to the execution hierarchy, and, when the target business includes a business process of a predetermined type, move the business process of the predetermined type from a current execution hierarchy to an execution hierarchy n+1 of a next execution hierarchy corresponding to each business process of which the execution sequence is less than m, where the next execution hierarchy corresponds to a highest execution hierarchy, and obtain an arrangement condition of execution hierarchies of each business process of which the execution sequence is greater than m, and integrally move each business process of which the execution sequence is greater than m to an execution hierarchy after the n+1 based on the arrangement condition, where m and n are positive integers.
7. The apparatus of claim 6, wherein the execution order adjustment module comprises:
a dependency level analysis sub-module for determining a dependency level between each business process according to the dependency relationship;
a business process execution matrix building sub-module for building a business process execution matrix, wherein an execution level in the business process execution matrix corresponds to the dependency level;
a business process placement sub-module, configured to place the business process in an execution hierarchy corresponding to the dependency hierarchy to which the business process belongs according to the dependency hierarchy and the first execution order;
and the second execution sequence determining submodule is used for determining the second execution sequence according to the execution hierarchy.
8. The apparatus of claim 7, wherein the execution order adjustment module further comprises, in a case where a predetermined type of business process is included in the target business:
the business process information acquisition sub-module is used for acquiring an execution sequence number m of the business process of the preset type in the first execution sequence and a sequence number of an execution level corresponding to each business process with the execution sequence number smaller than m;
a serial number maximum value determining sub-module, configured to determine a maximum value n in serial numbers of the execution levels;
A first moving submodule, configured to move the predetermined type of business process from a current execution level to an n+1th execution level;
the execution hierarchy arrangement condition acquisition sub-module is used for acquiring the arrangement condition of the execution hierarchy of each business process with the execution sequence number larger than m;
the second moving submodule is used for moving each business process with the execution sequence number larger than m into an execution level after the n+1th execution level of the business process execution matrix according to the original matrix structure according to the arrangement condition of the execution levels;
the execution sequence number m and the maximum value n are positive integers.
9. The apparatus of claim 7 or 8, the business process execution module comprising:
the first execution sub-module is used for serially executing each execution level of the business process execution matrix;
and the second execution sub-module is used for executing the business processes in each execution hierarchy in parallel.
10. The apparatus of any of claims 6 to 8, further comprising:
and the log information printing sub-module is used for printing log information of each business process according to the first execution sequence.
11. An electronic device, comprising:
at least one processor; and
A memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 5.
12. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010771528.4A CN111914010B (en) | 2020-08-04 | 2020-08-04 | Method, device, equipment and storage medium for processing business |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010771528.4A CN111914010B (en) | 2020-08-04 | 2020-08-04 | Method, device, equipment and storage medium for processing business |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111914010A CN111914010A (en) | 2020-11-10 |
| CN111914010B true CN111914010B (en) | 2024-02-20 |
Family
ID=73286504
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010771528.4A Active CN111914010B (en) | 2020-08-04 | 2020-08-04 | Method, device, equipment and storage medium for processing business |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111914010B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112766907B (en) * | 2021-01-20 | 2024-08-09 | 中国工商银行股份有限公司 | Service data processing method, device and server |
| CN113744037B (en) * | 2021-08-16 | 2023-09-29 | 同盾科技有限公司 | Method and device for communication of participants in knowledge federation, electronic equipment and storage medium |
| CN114647464B (en) * | 2022-05-19 | 2022-09-06 | 恒生电子股份有限公司 | Application parallel starting processing method and device and electronic equipment |
| CN115604101B (en) * | 2022-09-29 | 2024-02-02 | 北京亚控科技发展有限公司 | System management method and related equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101567013A (en) * | 2009-06-02 | 2009-10-28 | 阿里巴巴集团控股有限公司 | Method and apparatus for implementing ETL scheduling |
| CN103793349A (en) * | 2013-12-27 | 2014-05-14 | 远光软件股份有限公司 | Data processing method and data processing device |
| CN109661674A (en) * | 2016-12-22 | 2019-04-19 | 华为技术有限公司 | The system and method for Visual calculation workflow |
| CN110427411A (en) * | 2019-08-02 | 2019-11-08 | 河南开合软件技术有限公司 | Associated data is carried out visualization method by figure layer by one kind |
| CN110471656A (en) * | 2018-05-10 | 2019-11-19 | 北京京东尚科信息技术有限公司 | The method of adjustment and device of component level |
| CN110704023A (en) * | 2019-09-26 | 2020-01-17 | 北京华大九天软件有限公司 | Matrix block division method and device based on topological sorting |
| CN111427943A (en) * | 2020-03-27 | 2020-07-17 | 北京明略软件系统有限公司 | Task management method and device in ET L system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8027981B2 (en) * | 2008-12-10 | 2011-09-27 | International Business Machines Corporation | System, method and program product for classifying data elements into different levels of a business hierarchy |
| US9460171B2 (en) * | 2013-11-08 | 2016-10-04 | International Business Machines Corporation | Processing data in data migration |
| US10489225B2 (en) * | 2017-08-10 | 2019-11-26 | Bank Of America Corporation | Automatic resource dependency tracking and structure for maintenance of resource fault propagation |
-
2020
- 2020-08-04 CN CN202010771528.4A patent/CN111914010B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101567013A (en) * | 2009-06-02 | 2009-10-28 | 阿里巴巴集团控股有限公司 | Method and apparatus for implementing ETL scheduling |
| CN103793349A (en) * | 2013-12-27 | 2014-05-14 | 远光软件股份有限公司 | Data processing method and data processing device |
| CN109661674A (en) * | 2016-12-22 | 2019-04-19 | 华为技术有限公司 | The system and method for Visual calculation workflow |
| CN110471656A (en) * | 2018-05-10 | 2019-11-19 | 北京京东尚科信息技术有限公司 | The method of adjustment and device of component level |
| CN110427411A (en) * | 2019-08-02 | 2019-11-08 | 河南开合软件技术有限公司 | Associated data is carried out visualization method by figure layer by one kind |
| CN110704023A (en) * | 2019-09-26 | 2020-01-17 | 北京华大九天软件有限公司 | Matrix block division method and device based on topological sorting |
| CN111427943A (en) * | 2020-03-27 | 2020-07-17 | 北京明略软件系统有限公司 | Task management method and device in ET L system |
Non-Patent Citations (1)
| Title |
|---|
| 基于数据生成―消耗依赖的语义工作流并行化重构方法;孙晋永;闻立杰;匡增雄;李涛;张展;;计算机集成制造系统;20200615(06);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111914010A (en) | 2020-11-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111914010B (en) | Method, device, equipment and storage medium for processing business | |
| US11455217B2 (en) | Transaction consistency query support for replicated data from recovery log to external data stores | |
| US11269834B2 (en) | Detecting quasi-identifiers in datasets | |
| US9710455B2 (en) | Feature text string-based sensitive text detecting method and apparatus | |
| CN111639078A (en) | Data query method and device, electronic equipment and readable storage medium | |
| US10579619B2 (en) | Validation of query plan | |
| KR102361153B1 (en) | Managing data profiling operations related to data type | |
| JP7222040B2 (en) | Model training, image processing method and device, storage medium, program product | |
| EP3671468B1 (en) | Cloud-based platform instrumentation and monitoring system for maintenance of user-configured programs | |
| CN111259107B (en) | Determinant text storage method and device and electronic equipment | |
| JP6570156B2 (en) | Database system optimization method, system, electronic apparatus, and storage medium | |
| EP3172682B1 (en) | Distributing and processing streams over one or more networks for on-the-fly schema evolution | |
| CN111461343A (en) | Model parameter updating method and related equipment thereof | |
| US11442930B2 (en) | Method, apparatus, device and storage medium for data aggregation | |
| US11645234B2 (en) | Rule-based collections of subset(s) of metadata in response to a trigger event occurring | |
| US9805091B2 (en) | Processing a database table | |
| US20160378798A1 (en) | Aggregating and summarizing sequences of hierarchical records | |
| US9588998B2 (en) | Protecting storage data during system migration | |
| CN110727666A (en) | Cache assembly, method, equipment and storage medium for industrial internet platform | |
| CN111290714A (en) | Data reading method and device | |
| CN113778973B (en) | Data storage method and device | |
| CN113032209B (en) | Operation monitoring method, device, server and medium | |
| CN112328807A (en) | Anti-cheating method, device, equipment and storage medium | |
| KR101638048B1 (en) | Sql query processing method using mapreduce | |
| CN113760489B (en) | Resource allocation method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |