CN117076542A - Data processing method and related device - Google Patents
Data processing method and related device Download PDFInfo
- Publication number
- CN117076542A CN117076542A CN202311114620.3A CN202311114620A CN117076542A CN 117076542 A CN117076542 A CN 117076542A CN 202311114620 A CN202311114620 A CN 202311114620A CN 117076542 A CN117076542 A CN 117076542A
- Authority
- CN
- China
- Prior art keywords
- data
- target
- information
- task
- rule
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 14
- 238000004364 calculation method Methods 0.000 claims abstract description 66
- 238000000034 method Methods 0.000 claims abstract description 46
- 238000006243 chemical reaction Methods 0.000 claims abstract description 15
- 238000007781 pre-processing Methods 0.000 claims abstract description 12
- 238000012545 processing Methods 0.000 claims description 54
- 238000013507 mapping Methods 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 13
- 238000004891 communication Methods 0.000 claims description 10
- 238000012163 sequencing technique Methods 0.000 claims description 9
- 230000004044 response Effects 0.000 claims description 6
- 230000000007 visual effect Effects 0.000 description 16
- 238000010586 diagram Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 11
- 238000004458 analytical method Methods 0.000 description 7
- 238000007405 data analysis Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000007726 management method Methods 0.000 description 4
- DDCPKNYKNWXULB-RXMQYKEDSA-N (2r)-2-azaniumyl-3-[(2-methylpropan-2-yl)oxy]propanoate Chemical compound CC(C)(C)OC[C@@H]([NH3+])C([O-])=O DDCPKNYKNWXULB-RXMQYKEDSA-N 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000007792 addition Methods 0.000 description 2
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000013481 data capture Methods 0.000 description 1
- 238000013499 data model Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/04—Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Accounting & Taxation (AREA)
- Finance (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- Technology Law (AREA)
- General Business, Economics & Management (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application discloses a data processing method and a related device, wherein the method comprises the following steps: creating a scheduling task based on the target scheduling frame; rule configuration is carried out according to the scheduling task, and a target rule is obtained; loading cue data from a target data source according to a target rule, and preprocessing and converting the cue data to obtain a target data table; dynamically submitting a target data table to a data cluster component to execute a data calculation task to obtain a result message; performing format conversion on the result message to obtain target result data; and storing the target result data into a target theme database, and distributing the target result data to a target terminal based on the data distribution rule information. Therefore, the method can be realized, after analyzing and calculating the massive data streams of different types, customer cue data are formed, and then the customer cue data are matched and distributed to corresponding terminal equipment in real time according to distribution rules for users to use and perform customer service, so that the utilization rate of the data is improved, and the individuation and the efficiency of the customer service are improved.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a data processing method and a related device.
Background
With the traditional financial arts, for example: the trades of security dealer, bank, insurance, etc. begin to develop and change to the internet continuously, and more security dealer begins to make digital transformation. And utilizing industry data accumulated for many years to build a big data platform which accords with the industry data, and simultaneously starting the exploration and the application of the data on the basis. In the field of security traders, each trader may need to serve hundreds or thousands of clients at the same time, and under limited effort, how to improve the service efficiency of the trader, and adapting to diversified business requirements becomes a core requirement of the scheme. The current common modes are mainly: potential clients are mined based on the big data model to assist in exhibition, and the difficulty of model training and the diversity of combined service scenes can influence the efficiency of final client data analysis, so that the client service quality is influenced. How to effectively provide a complete data processing and analysis flow, realize rapid customer analysis and provide high-quality customer service is a current urgent problem to be solved.
Therefore, there is a need for a data processing method that solves the above problems.
Disclosure of Invention
The embodiment of the application provides a data processing method and a related device, which can form customer clue data after analyzing and calculating mass data streams of different types, and then match and distribute the customer clue data to corresponding terminal equipment in real time according to distribution rules for users to use and perform customer service, thereby finally ensuring the high efficiency of customer data processing and providing lower-delay, higher-efficiency and better customer service.
In a first aspect, an embodiment of the present application provides a data processing method, applied to a server, including:
creating a scheduling task based on the target scheduling frame;
rule configuration is carried out according to the scheduling task, and a target rule is obtained;
loading the clue data from a target data source according to the target rule, and preprocessing and converting the clue data to obtain a target data table;
dynamically submitting the target data table to a data cluster component to execute a data calculation task to obtain a result message;
performing format conversion on the result message to obtain target result data;
and storing the target result data into a target theme database, and distributing the target result data to a target terminal based on data distribution rule information.
In a second aspect, an embodiment of the present application provides a data processing apparatus, applied to a server, where the apparatus includes a creation unit, a configuration unit, and a processing unit; wherein,
the creating unit is used for creating a scheduling task based on the target scheduling frame;
the configuration unit is used for carrying out rule configuration according to the scheduling task to obtain a target rule;
the processing unit is used for loading the clue data from the target data source according to the target rule, and preprocessing and converting the clue data to obtain a target data table;
The processing unit is further used for dynamically submitting the target data table to the data cluster component to execute a data calculation task, so as to obtain a result message;
the processing unit is further used for performing format conversion on the result message to obtain target result data;
the processing unit is further configured to store the target result data into a target topic database, and distribute the target result data to a target terminal based on data distribution rule information.
In a third aspect, an embodiment of the present application provides a server, including a processor, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the processor, the programs including instructions for performing steps in any of the methods of the first aspect of the embodiments of the present application.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program causes a computer to perform part or all of the steps as described in any of the methods of the first aspect of the embodiments of the present application.
In a fifth aspect, embodiments of the present application provide a computer program product, wherein the computer program product comprises a non-transitory computer readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in any of the methods of the first aspect of the embodiments of the present application. The computer program product may be a software installation package.
It can be seen that in the embodiment of the present application, a scheduling task is created based on a target scheduling frame; rule configuration is carried out according to the scheduling task, and a target rule is obtained; loading cue data from a target data source according to a target rule, and preprocessing and converting the cue data to obtain a target data table; dynamically submitting a target data table to a data cluster component to execute a data calculation task to obtain a result message; performing format conversion on the result message to obtain target result data; and storing the target result data into a target theme database, and distributing the target result data to a target terminal based on the data distribution rule information. Therefore, the method can be realized, after analysis and calculation are carried out on mass data streams of different types, customer cue data are formed, and then the customer cue data are matched and distributed to corresponding terminal equipment in real time according to distribution rules for users to utilize and carry out customer service, so that the high efficiency of customer data processing is finally ensured, and lower-delay, higher-efficiency and better customer service is provided.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments or the description of the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a system architecture for data processing according to an embodiment of the present application;
FIG. 2 is a schematic flow chart of a data processing method according to an embodiment of the present application;
FIG. 3A is a schematic diagram of a visual interface of a thread data creation configuration file according to an embodiment of the present application;
FIG. 3B is a schematic diagram of a visual interface of a target topic database information configuration provided by an embodiment of the present application;
FIG. 3C is a schematic diagram of a visual interface for setting configuration information of a data source by using cue data according to an embodiment of the present application
FIG. 3D is a schematic diagram of a visual interface for setting variable information on cue data according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a server according to an embodiment of the present application;
Fig. 5 is a schematic diagram of a functional unit structure of a data processing apparatus according to an embodiment of the present application.
Detailed Description
In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The terms first, second and the like in the description and in the claims and in the above-described figures are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
In order to better understand the aspects of the embodiments of the present application, the following description will first describe electronic devices, related terms, concepts and related contexts to which the embodiments of the present application may relate.
The electronic device may be a portable electronic device that also contains other functions such as personal digital assistant and/or music player functions, such as a cell phone, tablet computer, wearable electronic device with wireless communication capabilities (e.g., a smart watch), etc. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices that are equipped with IOS systems, android systems, microsoft systems, or other operating systems. The portable electronic device may also be other portable electronic devices such as a Laptop computer (Laptop) or the like. It should also be understood that in other embodiments, the electronic device may be a desktop computer instead of a portable electronic device, or may be a server in its own right.
The following describes in detail a data processing system architecture adapted to the scheme of the present application, in conjunction with a schematic diagram of a system architecture of data processing provided in an embodiment of the present application in fig. 1.
In particular, as shown in fig. 1, the data processing system of the present application includes: the system comprises a task scheduling module, an execution service module, a theme database module, a data source module, a configuration module, a distribution module, a storage module and a task notification module.
Specifically, the task scheduling module performs task scheduling for realizing the whole thread task service (mot-task-service) based on a target scheduling framework (xxl-job is taken as an example in the scheme of the application); the configuration module is used for carrying out data rule configuration according to the data type of the task scheduling requirement to obtain a target rule; the execution service module acquires data from the data source module according to the task request and the target rule of the task scheduling module, wherein the data source module comprises a real-time data source and a timing data source which are respectively used for providing a data stream of a real-time data type and a data stream of a timing data type; the execution service module submits the acquired data stream to the data cluster assembly module to form an executable computing task, the data cluster assembly module finishes data processing to obtain a result message, the result message is transmitted to the topic database cluster, and the target topic database in the topic database cluster performs data format conversion and other processes to obtain result data; further, the distribution module uploads the result data to a storage module, namely, a Database (DB) for storage; finally, the task notification module distributes the result data in the database to one or more terminal devices according to the result data, and the result data are used in customer service by each terminal device.
The present application will be described in detail with reference to specific examples.
Referring to fig. 2, fig. 2 is a flow chart of a data processing method applied to a server, where the server includes at least one service processing unit, as shown in fig. 2, and the data processing method specifically includes the following steps:
s201, creating a scheduling task based on the target scheduling frame.
Illustratively, as shown in FIG. 1, in the present application, the target schedule framework implements the timing schedule of the entire thread task service (mot-task-service) in an open source schedule framework xxl-job. Based on the xxl-job scheduling framework, scheduling tasks are created.
The xxl-job is a Java-based distributed task scheduling framework and has the characteristics of easiness in use and high reliability.
Specifically, the creation of a scheduled task using xxl-job specifically includes the steps of:
illustratively, the relevant dependencies of xxl-job are introduced in the project's build file (e.g., maven's pon.xml); further, adding the address of the xxl-job dispatch center and other relevant configuration information in the configuration file of the project; further, writing a task Handler class for processing specific task logic; further, add @ JobHandler notes on the task Handler method: the @ JobHandler annotation provided by xxl-job is used to identify the method as a task that can be scheduled; further, registering the task Handler with a xxl-job dispatch center so that the task Handler can be dispatched; finally, a scheduling task is created in a management interface of the xxl-job scheduling center, and relevant information such as execution time, execution rules and the like of the task is specified.
S202, rule configuration is carried out according to the scheduling task, and a target rule is obtained.
The target rule refers to a rule based on which cue data is extracted from a target data source and preprocessed and converted in a specific manner. The types of cue data include real-time data types and timing data types.
Specifically, the server determines the data type of the cue data according to the task request corresponding to the scheduling task.
Illustratively, different data types correspond to different types of target data sources. The server registers the target data source as SPARK VIEW according to the different data types. Among the target data source types include, but are not limited to: ORACLE, mySQL, HDFS, kafka, files, etc. It should be noted that the real-time data type also needs to rely on connect or source framework to collect DB data into Kafka.
Further, the server writes corresponding target data source connection code according to the engine service. And according to the specific target data source type, performing connection and reading operations by using a proper database connection library or a file reading library.
S203, loading the clue data from a target data source according to the target rule, and preprocessing and converting the clue data to obtain a target data table.
Specifically, the server acquires cue data from a target data source according to a target rule, and converts the acquired cue data into a Spark data structure.
Further, it is registered as Spark View. Thus, subsequent Spark tasks may go directly through the target data table, for example: the Spark SQL or Data Frame API operates on and analyzes the Data.
S204, dynamically submitting the target data table to a data cluster component to execute a data calculation task, and obtaining a result message.
Illustratively, the server, after completing the loading, preprocessing and format conversion of the data according to step S203, dynamically submits the resulting data table (Spark SQL) to the dataset component. In the scheme of the application, the CDH/TBDS is taken as an example of the data cluster component, wherein the CDH/TBDS is a large data cluster component.
Further, the data table submitted by the server forms an executable calculation task, and the CDH/TBDS completes data calculation processing to obtain a result message.
S205, performing format conversion on the result message to obtain target result data.
Further, before the result message obtained in step S204 flows into the kafka theme, the result message needs to be uniformly converted into target result data in the same format through a uniform result processing rule, for example: json format.
S206, storing the target result data into a target theme database, and distributing the target result data to a target terminal based on data distribution rule information.
Further, the server stores the target result data in a target subject database through the distribution service.
Further, according to the data distribution rule information in the target rule, the target result data is distributed to the target terminal, and the service personnel using the target terminal perform customer service according to the received target result data.
The timing task returns a result through SPARK calculation, and the real-time task outputs the result to the target subject database through registration SPARK STREAMING.
It can be seen that, in the data processing method described in the embodiment of the present application, a scheduling task is created based on a target scheduling frame; rule configuration is carried out according to the scheduling task, and a target rule is obtained; loading cue data from a target data source according to a target rule, and preprocessing and converting the cue data to obtain a target data table; dynamically submitting a target data table to a data cluster component to execute a data calculation task to obtain a result message; performing format conversion on the result message to obtain target result data; and storing the target result data into a target theme database, and distributing the target result data to a target terminal based on the data distribution rule information. Therefore, after analysis and calculation are carried out on mass data streams of different types, customer cue data are formed, and then the customer cue data are matched and distributed to corresponding terminal equipment in real time according to distribution rules for users to utilize and carry out customer service, and finally feedback is formed to complete the whole closed loop.
In one possible example, the scheduling task includes information for specifying a data type of the thread data, where the data type includes at least one of: a real-time data type and a timing data type; the rule configuration according to the scheduling task may include the following steps: determining the data type of the cue data according to the scheduling task; if the data type is the real-time data type, a configuration file is created for the cue data; and configuring a calculation factor, wherein the calculation factor is used for defining a logic rule of the scheduling task so as to realize calculation and reasoning of the cue data; and configuring service attribute information; and combining the configuration file of the target theme database, the calculation factors and the service attribute information to obtain the target rule.
In practical application, two different types of thread data respectively correspond to different requirements and different processing modes. The real-time data type often changes rapidly in a short period, and data monitoring and result feedback are needed in time so as to better monitor data change. The timing data type often changes less or slowly in one period, so that data can be acquired periodically in one period by setting one period of data acquisition, and then data analysis and processing are performed according to the acquired data.
Illustratively, for real-time data sources used to collect real-time data types, such as: the data in the database can be updated in real time by using a connector framework such as Debezium or CDC (Change Data Capture), and the data in the database is changed and collected to the Kafka theme, so that the real-time pushing and the synchronization of the data can be realized. For non-real time data sources that are used to collect timing data types, such as data in a file system (e.g., HDFS), the file content may be read directly and parsed. Therefore, the processing of which the data type is the real-time data type needs to be set in advance so as to better meet the requirement of data analysis processing.
For example, if it is determined that the type of thread data required in the task is a real-time data type, the server provides a visual setting interface for a developer or a user to set a configuration file for the thread data of the real-time data type according to actual requirements, where the configuration file is used to define a mapping relationship between the target subject database and the target data source, and the target data source includes one or more mapping relationships including, but not limited to, a name of each target data source, a name of the target subject database, and so on.
Further, a calculation factor is configured, wherein the calculation factor is a core configuration of the whole scheduling task and is responsible for a logic rule of the whole scheduling task.
Further, the service attribute information is configured.
In this example, the server determines the type of the cue data according to the scheduling task, and if the cue data is determined to be of a real-time data type, relevant configuration is performed through the visual interface to obtain the target rule for subsequent data acquisition, data analysis and calculation, so that the target result data is obtained for service personnel to provide services for the clients more efficiently and accurately.
In one possible example, the creating a configuration file for the cue data may include the steps of: creating the target topic database in a topic database cluster; configuring the topic information of the target topic database, wherein the topic information comprises a table name, name information of the target topic database and data query frequency information; defining data field information of the target theme database, wherein the data field information comprises field names and field types of one or more data fields; and combining the subject information and the data field information to form a configuration file of the cue data.
Illustratively, a detailed description will be provided below in connection with FIG. 3A. Fig. 3A is a schematic view of a visual interface of a cue data creation configuration file according to an embodiment of the present application, specifically, as shown in fig. 3A:
the server creates a configuration file for the cue data by providing a visual interface for a developer or a system user, and performs configuration in response to the related operation of the visual interface. Specifically, the target topic database is created in the topic database cluster, and topic information of the target topic database is set.
Illustratively, the subject information includes as shown in FIG. 3A: table names, table types, table topic information (e.g., kafka topic), table description information, data query frequency information, and the like of the target topic database.
Further, as shown in fig. 3B, fig. 3B is a schematic view of a visual interface of information configuration of a target topic database according to an embodiment of the present application. Specifically, in response to an operation of the developer or system user at the interface to define data field information of the target subject database, wherein the data field information includes field names, field descriptions, field types, and the like of one or more data fields as shown in fig. 3B.
Further, the topic information and the data field information are combined to form a configuration file of the target topic database.
It can be seen that in this example, the server creates a configuration file for the cue data by providing a visual interface, and the configuration file may be set according to the requirement of the scheduled task or the actual data requirement, so that the setting of the configuration file may be more personalized and flexible.
In one possible example, the configuration calculation factor may include the steps of: setting data source configuration information for the cue data, wherein the data source configuration information is used for defining a mapping relation between the target data source and the target subject database, the target data source comprises one or more mapping relations, and the mapping relation is used for indicating the name of each target data source and the name of the target subject database; setting variable information for the cue data, wherein the variable information is used for dynamic parameter management when the scheduling task is executed; setting calculation rule information for the cue data, wherein the calculation rule information is used for representing calculation logic of each data field contained in the cue data; and combining the data source configuration information, the variable information and the calculation rule information to obtain the calculation factor.
Illustratively, a detailed description will be provided below in connection with FIG. 3C. FIG. 3C is a schematic diagram of a visual interface for setting data source configuration information by using clue data, specifically, before setting the data source configuration information, as shown in FIG. 3C, a server may set related information in response to an operation of a developer or a system user in the visual interface, where the method includes: factor name, used to describe the goal currently achieved by the data analysis process, such as: large funds transfer out (real time); the calculation type: timing or real time; factor descriptions, rules for detailing the triggering of message alerts by the current factor calculation, such as: the real-time accumulated transfer amount of more than 10 ten thousand per day accounts for more than 30% of the total assets of the previous transaction day, and reminds service personnel of timely paying attention.
Further, one or more data sources are selected as target data sources, namely, the data source names are added in the lower data source box shown in fig. 3C; setting a data source type, for example: if a real-time data source is used in actual data calculation, the registration of the data source can be completed by configuring KAFKA theme mapping; setting a registry name; set table reference fields, etc. The specific operation can realize the addition, deletion and modification of the information through editing or deleting below the right operation, so that the variable information related to the cue data is obtained.
Further, as shown in fig. 3D, fig. 3D is a schematic view of a visual interface of thread data set variable information according to an embodiment of the present application. As shown in fig. 3D, variable information configuration is performed for the thread data, where the variable information is used for dynamic parameter management when executing the scheduling task. Including but not limited to the following information: variable code, data type, custom tags, etc. The specific operation can realize the addition, deletion and modification of the information through editing or deleting below the right operation, so that the variable information related to the cue data is obtained.
Further, calculation rule information is set for the cue data. As shown in fig. 3C, the data fields are selected by customization, for example: transaction day, natural day, asset duty, transaction amount, etc. Then, by inputting a corresponding database statement in the next Fang Shanyuan grid, a manner of setting calculation for data corresponding to one or more fields is set as calculation rule information.
In this example, the server may set the configuration information of the data source for the cue data by providing the visual configuration interface and responding to the operation of the developer or the system user on the visual interface, and then the server may directly perform data analysis processing on the obtained cue data according to the configuration information, so as to obtain the corresponding target result data for the terminal device to perform the customer analysis and the customer service by using the user, thereby improving the quality and efficiency of the customer service.
In one possible example, the configuring service attribute information may include the steps of: configuring basic information for the scheduling task, wherein the basic information comprises at least one of the following: task name information, task description information and task priority information of the scheduling task; setting the associated calculation factors for the scheduling tasks; setting content template information for the scheduling task, wherein the content template information is used for creating and managing a message template for distributing the target result data; setting data distribution rule information for the scheduling task so as to realize distribution and pushing of the target result data; setting message channel information, wherein the message channel information is used for representing a channel for transmitting the target result data; and combining the basic information, the calculation factor, the content template information, the data distribution rule information and the message channel information of the scheduling task to obtain the service attribute information.
Illustratively, the server needs to configure basic information for the scheduled task while setting target rules for the scheduled task, where the basic information includes, but is not limited to: task name, task description, and task priority, etc.
Further, an associated calculation factor is set for each scheduling task, that is, different scheduling tasks expect different targets to be achieved, so that in the process of using the data fields, the influence duty ratio of the numerical values corresponding to different data fields on the result is also different, and therefore different calculation can be achieved through associating different calculation factors, and different calculation results are obtained.
Further, content template information is set for the scheduling task, and is used for sending target result data to the terminal equipment for use through a unified template.
Further, the server obtains corresponding cue data based on the scheduling task and the target rule, and obtains corresponding target result data after data analysis and calculation through the data cluster component. The target result data needs to be used in the actual customer service to play its role, so that information of a message channel for transmitting the target result data needs to be preset, and when the scheduling task is completed, the information can be timely transmitted to one or more terminal devices through the preset message channel. In actual use, message channels include, but are not limited to: enterprise WeChat, mailbox, etc.
In this example, the server may obtain the content to be pushed directly according to the content template after obtaining the target result data by presetting the basic information of the scheduling task, and further push the content to one or more terminal devices through a preset message channel, so as to improve the efficiency, flexibility and individuation of content pushing.
In one possible example, the scheduling task includes at least one task, each of the tasks corresponding to one of the target result data; the method for setting data distribution rule information for the scheduling task may include the steps of: determining the priority of each task according to the task priority information; sequencing the at least one task according to the priority of each task to obtain a sequencing result; setting the distribution sequence of the target result data corresponding to each task according to the sequencing result; or determining a to-be-selected relation corresponding to each task, wherein the to-be-selected relation comprises a main consultation relation and a service relation; setting a target data field according to the relation to be selected, wherein the target data field is one or more of the data fields; and establishing a mapping relation between the target data field and the candidate relation, and storing the mapping relation into the distribution rule information.
For example, one or more tasks may be created simultaneously by the server as scheduled tasks, and each scheduled task corresponds to one target result data. And the users corresponding to the one or more target result data may be the same or respectively correspond to one or more terminal devices. Therefore, a different priority order needs to be set for each task, so that the server can respectively send corresponding target result data to one or more terminal devices according to the priority order.
For example, different tasks in the actual application scenario may correspond to different candidate relationships, for example: a business relationship, a service relationship, etc. And the requirements of different candidate relations on the target result data may be different, different data fields may be preset according to the emphasis of the actual candidate relations, and corresponding result data are screened from the target result data according to the data fields and sent to different terminal devices respectively. For example: service relationships focus on services, then more on the user's personalized needs, the user's revenue information, etc.; the business relationship is a method of developing market from acquaintances by using the original relationship such as relatives, working relationships, business relationships, etc. More emphasis is placed on the social relationship information of the user, etc.
In this example, the server can realize orderly sending of the target result data by presetting task priority, or set the data field according to the relation to be selected, so that the target result data can meet the requirement of the user more accurately, and the user analysis and the image are more accurate, thereby providing better user service.
In one possible example, the method may include the steps of: according to the task priority information, sequentially sending the target result data to the target terminal, wherein the target result data is used for providing customer service response and operation for a user using the target terminal; or extracting data corresponding to the target data field from the target result data according to the mapping relation; and respectively sending the data corresponding to the target data field to the target terminal according to the data distribution rule.
Illustratively, the server is configured to distribute the data according to preset data distribution rules, for example: according to the task priority information, target result data is sequentially pushed to one or more target terminals after content to be pushed in a standard pushing format is generated according to a content template; or pushing the content corresponding to all or part of the data fields in the target result data to the corresponding target terminals according to different candidate relations.
In this example, the server may directly implement ordered and accurate pushing of the target result data according to the preset data distribution rule, so that the user of the target terminal may perform the customer analysis and customer service planning according to the received data, thereby providing efficient and high-quality customer service and improving the customer experience.
Referring to fig. 4, fig. 4 is a schematic structural diagram of a server, as shown in fig. 4, the server includes a processor, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the processor, and the server is applied to the solution of the present application, and the programs include instructions for performing the following steps:
creating a scheduling task based on the target scheduling frame;
rule configuration is carried out according to the scheduling task, and a target rule is obtained;
loading the clue data from a target data source according to the target rule, and preprocessing and converting the clue data to obtain a target data table;
dynamically submitting the target data table to a data cluster component to execute a data calculation task to obtain a result message;
Performing format conversion on the result message to obtain target result data;
and storing the target result data into a target theme database, and distributing the target result data to a target terminal based on data distribution rule information.
It can be seen that, in the data processing method described in the embodiment of the present application, a preset bifurcation tree is obtained, where each node of the preset bifurcation tree corresponds to a service processing unit; searching a plurality of nodes and determining a plurality of target nodes; in the searching process, determining the conflict quantity corresponding to each target node; judging whether the target node corresponding to the minimum conflict quantity is a leaf node or not; if the target node is the leaf node, determining a neighborhood node of the target node corresponding to the minimum conflict quantity; determining a target child node corresponding to the neighborhood node, and calculating the conflict quantity of the target child node; searching a preset bifurcation tree step by step upwards by taking a target child node as a reference node to obtain an associated node of the target child node; distributing a plurality of transaction request data to the neighborhood node and the associated node, and updating the conflict quantity of the associated node according to the conflict quantity of the target child node to obtain a first bifurcation tree so as to complete the distribution of the plurality of transaction request data; the target path is determined based on the amount of collision for each node of the first bifurcation tree. By means of the method, iteration selection judgment is carried out on the nodes in the searching process of the preset bifurcation tree, the searching range is gradually converged from the complex tree structure space, the high efficiency of client transaction data processing is finally guaranteed, and lower-delay, higher-efficiency and better user experience is provided.
In one possible example, the scheduling task includes information for specifying a data type of the thread data, where the data type includes at least one of: a real-time data type and a timing data type;
the above program includes instructions for performing the following steps:
determining the data type of the cue data according to the scheduling task;
if the data type is the real-time data type, a configuration file is created for the cue data; the method comprises the steps of,
configuring a calculation factor, wherein the calculation factor is used for defining a logic rule of the scheduling task so as to realize calculation and reasoning of the cue data; the method comprises the steps of,
configuring service attribute information;
and combining the configuration file of the target theme database, the calculation factors and the service attribute information to obtain the target rule.
In one possible example, the creating a configuration file for the cue data, the program includes instructions for:
creating the target topic database in a topic database cluster;
configuring the topic information of the target topic database, wherein the topic information comprises a table name, name information of the target topic database and data query frequency information;
Defining data field information of the target theme database, wherein the data field information comprises field names and field types of one or more data fields;
and combining the subject information and the data field information to form a configuration file of the cue data.
In one possible example, the configuration calculation factor, the above program includes instructions for performing the steps of:
setting data source configuration information for the cue data, wherein the data source configuration information is used for defining a mapping relation between the target data source and the target subject database, the target data source comprises one or more mapping relations, and the mapping relation is used for indicating the name of each target data source and the name of the target subject database;
setting variable information for the cue data, wherein the variable information is used for dynamic parameter management when the scheduling task is executed;
setting calculation rule information for the cue data, wherein the calculation rule information is used for representing calculation logic of each data field contained in the cue data;
and combining the data source configuration information, the variable information and the calculation rule information to obtain the calculation factor.
In one possible example, the configuration service attribute information, the program includes instructions for:
configuring basic information for the scheduling task, wherein the basic information comprises at least one of the following: task name information, task description information and task priority information of the scheduling task;
setting the associated calculation factors for the scheduling tasks;
setting content template information for the scheduling task, wherein the content template information is used for creating and managing a message template for distributing the target result data;
setting data distribution rule information for the scheduling task so as to realize distribution and pushing of the target result data;
setting message channel information, wherein the message channel information is used for representing a channel for transmitting the target result data;
and combining the basic information, the calculation factor, the content template information, the data distribution rule information and the message channel information of the scheduling task to obtain the service attribute information.
In one possible example, the scheduling task includes at least one task, each of the tasks corresponding to one of the target result data;
The program sets data distribution rule information for the scheduling task, and includes instructions for performing the steps of:
determining the priority of each task according to the task priority information;
sequencing the at least one task according to the priority of each task to obtain a sequencing result;
setting the distribution sequence of the target result data corresponding to each task according to the sequencing result; or,
determining a to-be-selected relation corresponding to each task, wherein the to-be-selected relation comprises a main consultation relation and a service relation;
setting a target data field according to the relation to be selected, wherein the target data field is one or more of the data fields;
and establishing a mapping relation between the target data field and the candidate relation, and storing the mapping relation into the distribution rule information.
In one possible example, the target result data is distributed to a target terminal based on a data distribution rule, and the program includes instructions for performing the steps of:
according to the task priority information, sequentially sending the target result data to the target terminal, wherein the target result data is used for providing customer service response and operation for a user using the target terminal; or,
Extracting data corresponding to the target data field from the target result data according to the mapping relation;
and respectively sending the data corresponding to the target data field to the target terminal according to the data distribution rule.
The foregoing description of the embodiments of the present application has been presented primarily in terms of a method-side implementation. It will be appreciated that the electronic device, in order to achieve the above-described functions, includes corresponding hardware structures and/or software modules that perform the respective functions. Those of skill in the art will readily appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as hardware or combinations of hardware and computer software. Whether a function is implemented as hardware or computer software driven hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The embodiment of the application can divide the functional units of the server according to the method example, for example, each functional unit can be divided corresponding to each function, or two or more functions can be integrated in one processing unit. The integrated units may be implemented in hardware or in software functional units. It should be noted that, in the embodiment of the present application, the division of the units is schematic, which is merely a logic function division, and other division manners may be implemented in actual practice.
In the case of dividing the respective functional modules with the respective functions, fig. 5 shows a schematic diagram of the functional unit structure of a data processing apparatus, as shown in fig. 5, the data processing apparatus 500 is applied to a server, and the data processing apparatus 500 may include a creation unit 501, a configuration unit 502 and a processing unit 503, wherein,
a creating unit 501 for creating a scheduling task based on a target scheduling frame;
a configuration unit 502, configured to perform rule configuration according to the scheduling task, so as to obtain a target rule;
a processing unit 503, configured to load the cue data from the target data source according to the target rule, and perform preprocessing and conversion on the cue data to obtain a target data table;
the processing unit 503 is further configured to dynamically submit the target data table to the data cluster component to perform a data calculation task, so as to obtain a result packet;
the processing unit 503 is further configured to perform format conversion on the result packet to obtain target result data;
the processing unit 503 is further configured to store the target result data in a target topic database, and distribute the target result data to a target terminal based on data distribution rule information.
It can be seen that, in the data processing device provided by the embodiment of the present application, the creating unit creates a scheduling task based on the target scheduling frame; the configuration unit carries out rule configuration according to the scheduling task to obtain a target rule; the processing unit loads the clue data from the target data source according to the target rule, and performs preprocessing and conversion on the clue data to obtain a target data table; the processing unit dynamically submits the target data table to the data cluster component to execute the data calculation task, and a result message is obtained; the processing unit performs format conversion on the result message to obtain target result data; the processing unit stores the target result data into a target subject database, and distributes the target result data to the target terminal based on the data distribution rule information. Therefore, the method can be realized, after analyzing and calculating the massive data streams of different types, customer cue data are formed, and then the customer cue data are matched and distributed to corresponding terminal equipment in real time according to distribution rules for users to use and perform customer service, so that the utilization rate of the data is improved, and the individuation and the efficiency of the customer service are improved.
It should be noted that, all relevant contents of each step related to the above method embodiment may be cited to the functional description of the corresponding functional module, which is not described herein.
The server provided in this embodiment is configured to execute the data processing method, so that the same effects as those of the implementation method can be achieved.
In case of an integrated unit, the server may comprise a processing module, a storage module and a communication module. The processing module may be configured to control and manage actions of the server, for example, may be configured to support the server to perform the steps performed by the creating unit 501, the configuring unit 502, and the processing unit 503. The storage module may be used to support a server for storing program code, data, etc. And the communication module can be used for supporting the communication between the server and other devices.
Wherein the processing module may be a processor or a controller. Which may implement or perform the various exemplary logic blocks, modules and circuits described in connection with this disclosure. A processor may also be a combination that performs computing functions, e.g., including one or more microprocessors, digital signal processing (digital signal processing, DSP) and microprocessor combinations, and the like. The memory module may be a memory. The communication module can be a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip and other equipment which interact with other electronic equipment.
The embodiment of the application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program makes a computer execute part or all of the steps of any one of the above method embodiments, and the computer includes an electronic device.
Embodiments of the present application also provide a computer program product comprising a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform part or all of the steps of any one of the methods described in the method embodiments above. The computer program product may be a software installation package, said computer comprising an electronic device.
It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present application is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present application.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.
In the several embodiments provided by the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, such as the above-described division of units, merely a division of logic functions, and there may be additional manners of dividing in actual implementation, such as multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, or may be in electrical or other forms.
The units described above as separate components may or may not be physically separate, and components shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units described above, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable memory. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a memory, comprising several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the above-mentioned method of the various embodiments of the present application. And the aforementioned memory includes: a U-disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.
Those of ordinary skill in the art will appreciate that all or a portion of the steps in the various methods of the above embodiments may be implemented by a program that instructs associated hardware, and the program may be stored in a computer readable memory, which may include: flash disk, read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.
The foregoing has outlined rather broadly the more detailed description of embodiments of the application, wherein the principles and embodiments of the application are explained in detail using specific examples, the above examples being provided solely to facilitate the understanding of the method and core concepts of the application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present application, the present description should not be construed as limiting the present application in view of the above.
Claims (10)
1. A data processing method, applied to a server, the method comprising:
creating a scheduling task based on the target scheduling frame;
rule configuration is carried out according to the scheduling task, and a target rule is obtained;
Loading the clue data from a target data source according to the target rule, and preprocessing and converting the clue data to obtain a target data table;
dynamically submitting the target data table to a data cluster component to execute a data calculation task to obtain a result message;
performing format conversion on the result message to obtain target result data;
and storing the target result data into a target theme database, and distributing the target result data to a target terminal based on data distribution rule information.
2. The method of claim 1, wherein the scheduling task includes information for specifying a data type of the thread data, the data type including at least one of: a real-time data type and a timing data type;
the rule configuration according to the scheduling task comprises the following steps:
determining the data type of the cue data according to the scheduling task;
if the data type is the real-time data type, a configuration file is created for the cue data; the method comprises the steps of,
configuring a calculation factor, wherein the calculation factor is used for defining a logic rule of the scheduling task so as to realize calculation and reasoning of the cue data; the method comprises the steps of,
Configuring service attribute information;
and combining the configuration file of the target theme database, the calculation factors and the service attribute information to obtain the target rule.
3. The method of claim 2, wherein the creating a configuration file for the cue data comprises:
creating the target topic database in a topic database cluster;
configuring the topic information of the target topic database, wherein the topic information comprises a table name, name information of the target topic database and data query frequency information;
defining data field information of the target theme database, wherein the data field information comprises field names and field types of one or more data fields;
and combining the subject information and the data field information to form a configuration file of the cue data.
4. The method of claim 2, wherein configuring the calculation factor comprises:
setting data source configuration information for the cue data, wherein the data source configuration information is used for defining a mapping relation between the target data source and the target subject database, the target data source comprises one or more mapping relations, and the mapping relation is used for indicating the name of each target data source and the name of the target subject database;
Setting variable information for the cue data, wherein the variable information is used for dynamic parameter management when the scheduling task is executed;
setting calculation rule information for the cue data, wherein the calculation rule information is used for representing calculation logic of each data field contained in the cue data;
and combining the data source configuration information, the variable information and the calculation rule information to obtain the calculation factor.
5. The method of claim 2, wherein configuring the service attribute information comprises:
configuring basic information for the scheduling task, wherein the basic information comprises at least one of the following: task name information, task description information and task priority information of the scheduling task;
setting the associated calculation factors for the scheduling tasks;
setting content template information for the scheduling task, wherein the content template information is used for creating and managing a message template for distributing the target result data;
setting data distribution rule information for the scheduling task so as to realize distribution and pushing of the target result data;
setting message channel information, wherein the message channel information is used for representing a channel for transmitting the target result data;
And combining the basic information, the calculation factor, the content template information, the data distribution rule information and the message channel information of the scheduling task to obtain the service attribute information.
6. The method of claim 5, wherein the scheduling tasks include at least one task, each of the tasks corresponding to one of the target result data;
the setting of data distribution rule information for the scheduling task includes:
determining the priority of each task according to the task priority information;
sequencing the at least one task according to the priority of each task to obtain a sequencing result;
setting the distribution sequence of the target result data corresponding to each task according to the sequencing result; or,
determining a to-be-selected relation corresponding to each task, wherein the to-be-selected relation comprises a main consultation relation and a service relation;
setting a target data field according to the relation to be selected, wherein the target data field is one or more of the data fields;
and establishing a mapping relation between the target data field and the candidate relation, and storing the mapping relation into the distribution rule information.
7. The method of claim 6, wherein the distributing the target result data to a target terminal based on data distribution rule information comprises:
according to the task priority information, sequentially sending the target result data to the target terminal, wherein the target result data is used for providing customer service response and operation for a user using the target terminal; or,
extracting data corresponding to the target data field from the target result data according to the mapping relation;
and respectively sending the data corresponding to the target data field to the target terminal according to the data distribution rule.
8. A data processing apparatus, characterized by being applied to a server, the apparatus comprising a creation unit, a configuration unit and a processing unit; wherein,
the creating unit is used for creating a scheduling task based on the target scheduling frame;
the configuration unit is used for carrying out rule configuration according to the scheduling task to obtain a target rule;
the processing unit is used for loading the clue data from the target data source according to the target rule, and preprocessing and converting the clue data to obtain a target data table;
The processing unit is further used for dynamically submitting the target data table to the data cluster component to execute a data calculation task, so as to obtain a result message;
the processing unit is further used for performing format conversion on the result message to obtain target result data;
the processing unit is further configured to store the target result data into a target topic database, and distribute the target result data to a target terminal based on data distribution rule information.
9. A server comprising a processor, a memory, a communication interface, and one or more programs stored in the memory and configured to be executed by the processor, the programs comprising instructions for performing the steps in the method of any of claims 1-7.
10. A computer-readable storage medium, characterized in that a computer program for electronic data exchange is stored, wherein the computer program causes a computer to perform the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311114620.3A CN117076542B (en) | 2023-08-29 | 2023-08-29 | Data processing method and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311114620.3A CN117076542B (en) | 2023-08-29 | 2023-08-29 | Data processing method and related device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117076542A true CN117076542A (en) | 2023-11-17 |
| CN117076542B CN117076542B (en) | 2024-06-07 |
Family
ID=88705898
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311114620.3A Active CN117076542B (en) | 2023-08-29 | 2023-08-29 | Data processing method and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117076542B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106557471A (en) * | 2015-09-24 | 2017-04-05 | 上海汽车集团股份有限公司 | Method for scheduling task and device |
| CN106951315A (en) * | 2017-03-17 | 2017-07-14 | 北京搜狐新媒体信息技术有限公司 | A kind of data task dispatching method and system based on ETL |
| CN109948046A (en) * | 2018-01-05 | 2019-06-28 | 深圳壹账通智能科技有限公司 | Information push method, device, mobile terminal and storage medium |
| CN111325626A (en) * | 2020-02-21 | 2020-06-23 | 中国工商银行股份有限公司 | Account checking data pushing method and device |
| WO2021012553A1 (en) * | 2019-07-25 | 2021-01-28 | 深圳壹账通智能科技有限公司 | Data processing method and related device |
| CN113377823A (en) * | 2021-06-28 | 2021-09-10 | 中国农业银行股份有限公司 | Value data processing method, device, equipment and storage medium |
| CN113836131A (en) * | 2021-09-29 | 2021-12-24 | 平安科技（深圳）有限公司 | Big data cleaning method and device, computer equipment and storage medium |
| CN115169316A (en) * | 2022-07-27 | 2022-10-11 | 中国农业银行股份有限公司 | Data processing template generation method and device, electronic equipment and storage medium |
| CN115174582A (en) * | 2022-09-06 | 2022-10-11 | 中国中金财富证券有限公司 | Data scheduling method and related device |
| CN115186650A (en) * | 2022-09-07 | 2022-10-14 | 中国中金财富证券有限公司 | Data detection method and related device |
| WO2023046141A1 (en) * | 2021-09-27 | 2023-03-30 | 华为技术有限公司 | Acceleration framework and acceleration method for database network load performance, and device |
| CN116560626A (en) * | 2023-04-14 | 2023-08-08 | 北京滴普科技有限公司 | Data processing method, system, equipment and storage medium based on custom rules |
-
2023
- 2023-08-29 CN CN202311114620.3A patent/CN117076542B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106557471A (en) * | 2015-09-24 | 2017-04-05 | 上海汽车集团股份有限公司 | Method for scheduling task and device |
| CN106951315A (en) * | 2017-03-17 | 2017-07-14 | 北京搜狐新媒体信息技术有限公司 | A kind of data task dispatching method and system based on ETL |
| CN109948046A (en) * | 2018-01-05 | 2019-06-28 | 深圳壹账通智能科技有限公司 | Information push method, device, mobile terminal and storage medium |
| WO2021012553A1 (en) * | 2019-07-25 | 2021-01-28 | 深圳壹账通智能科技有限公司 | Data processing method and related device |
| CN111325626A (en) * | 2020-02-21 | 2020-06-23 | 中国工商银行股份有限公司 | Account checking data pushing method and device |
| CN113377823A (en) * | 2021-06-28 | 2021-09-10 | 中国农业银行股份有限公司 | Value data processing method, device, equipment and storage medium |
| WO2023046141A1 (en) * | 2021-09-27 | 2023-03-30 | 华为技术有限公司 | Acceleration framework and acceleration method for database network load performance, and device |
| CN113836131A (en) * | 2021-09-29 | 2021-12-24 | 平安科技（深圳）有限公司 | Big data cleaning method and device, computer equipment and storage medium |
| CN115169316A (en) * | 2022-07-27 | 2022-10-11 | 中国农业银行股份有限公司 | Data processing template generation method and device, electronic equipment and storage medium |
| CN115174582A (en) * | 2022-09-06 | 2022-10-11 | 中国中金财富证券有限公司 | Data scheduling method and related device |
| CN115186650A (en) * | 2022-09-07 | 2022-10-14 | 中国中金财富证券有限公司 | Data detection method and related device |
| CN116560626A (en) * | 2023-04-14 | 2023-08-08 | 北京滴普科技有限公司 | Data processing method, system, equipment and storage medium based on custom rules |
Non-Patent Citations (2)
| Title |
|---|
| 潘玉峰;: "面向金融模型海量估值的复杂计算框架的建设和实践", 中国金融电脑, no. 07, 7 July 2018 (2018-07-07), pages 157 - 158 * |
| 蒋杰，等: "《腾讯大数据构建之道》", 机械工业出版社, pages: 157 - 158 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117076542B (en) | 2024-06-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9336184B2 (en) | Representation of an interactive document as a graph of entities | |
| US9304672B2 (en) | Representation of an interactive document as a graph of entities | |
| US8555248B2 (en) | Business object change management using release status codes | |
| US11755461B2 (en) | Asynchronous consumer-driven contract testing in micro service architecture | |
| US8799230B2 (en) | Method and system for centralized issue tracking | |
| CN108352003A (en) | It is presented by the single choice or multiselect data of Cooperative Database service | |
| US10636086B2 (en) | XBRL comparative reporting | |
| CN108572963A (en) | Information acquisition method and device | |
| CN106575246A (en) | Machine learning service | |
| CN111639480A (en) | Text labeling method based on artificial intelligence, electronic device and storage medium | |
| US20150370674A1 (en) | Tenant provisioning for testing a production multi-tenant service | |
| CN111897890B (en) | Financial business processing method and device | |
| CN110249356A (en) | A kind of sharing method and system of user-defined ERP function | |
| CN111221855A (en) | Data processing method and device | |
| Tai et al. | The construction of a mobile business application system for ERP | |
| CN117454278A (en) | Method and system for realizing digital rule engine of standard enterprise | |
| CN112102099B (en) | Policy data processing method and device, electronic equipment and storage medium | |
| US10289525B2 (en) | Multi-layer design response time calculator | |
| US20210124752A1 (en) | System for Data Collection, Aggregation, Storage, Verification and Analytics with User Interface | |
| CN109815448A (en) | Lantern slide generation method and device | |
| CN117076542B (en) | Data processing method and related device | |
| US11995587B2 (en) | Method and device for managing project by using data merging | |
| CN111209281A (en) | Annuity data processing method, device, medium and electronic equipment | |
| CN110347657A (en) | Data creation method, device, electronic equipment and storage medium | |
| CN106575301B (en) | Automated orchestration of infrastructure service blocks in hosted services |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |