CN111104428A - Flow calculation method, flow calculation device, flow calculation system, and medium - Google Patents
Flow calculation method, flow calculation device, flow calculation system, and medium Download PDFInfo
- Publication number
- CN111104428A CN111104428A CN201911314030.9A CN201911314030A CN111104428A CN 111104428 A CN111104428 A CN 111104428A CN 201911314030 A CN201911314030 A CN 201911314030A CN 111104428 A CN111104428 A CN 111104428A
- Authority
- CN
- China
- Prior art keywords
- message
- stream computing
- data
- source
- stream
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004364 calculation method Methods 0.000 title claims abstract description 62
- 238000000034 method Methods 0.000 claims description 25
- 230000009471 action Effects 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 13
- 238000012163 sequencing technique Methods 0.000 claims description 8
- 230000000694 effects Effects 0.000 abstract description 6
- 238000012545 processing Methods 0.000 description 16
- 238000004891 communication Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 3
- 238000005192 partition Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24568—Data stream processing; Continuous queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a flow calculation method, which comprises the following steps: the stream computing system reads source data; writing the source end data into a target end to produce messages with numbers in the target end, wherein the numbers correspond to the messages one to one and are monotonically increased; and the application terminal performs duplicate removal on the number, and reads the message corresponding to the number in the target terminal according to the number after the duplicate removal. The invention also discloses a stream calculating device and a computer readable storage medium, which achieve the effect of improving the compatibility of stream calculation.
Description
Technical Field
The present invention relates to the field of data processing, and in particular, to a stream calculation method, a stream calculation apparatus, a stream calculation system, and a computer-readable storage medium.
Background
In a conventional data processing flow, data is always collected and then placed in a database. When people need to get answers or perform relevant processing, the data can be queried through the database. This seems reasonable, but the result is very compact and does not solve the problem well, especially in the face of some specific problems in some real-time search application environments. This leads to a new data computation structure-stream computation approach. In the flow calculation process, in order to improve the accuracy of data processing, each input message needs to affect the final result only once, and even if a machine fault or software crash occurs, it is ensured that data is not processed repeatedly.
In the conventional stream calculation process, only one-time processing can be realized based on Flink (frienk, an open source stream processing framework). However, the Flink framework is complex, and cannot solve the problem that a simple independent Java or C + + program is processed only once, and a characteristic downstream component must be used, and cannot support other downstream components such as a database. This has the disadvantage of poor compatibility with stream computations.
Disclosure of Invention
The present invention is directed to a stream calculation method, a stream calculation apparatus, a stream calculation system, and a computer-readable storage medium, which are intended to achieve the effect of improving the compatibility of stream calculation.
To achieve the above object, the present invention provides a flow calculation method including the steps of:
the stream computing system reads source data;
writing the source end data into a target end to produce messages with numbers in the target end, wherein the numbers correspond to the messages one to one and are monotonically increased;
and the application terminal performs duplicate removal on the number, and reads the message corresponding to the number in the target terminal according to the number after the duplicate removal.
Optionally, after the step of writing the source data to the target to produce the numbered message in the target, the method further includes:
judging whether the source end data fails to be written or not;
and when the source end data writing fails, executing the step of writing the source end data into the target end.
Optionally, the step of reading, by the application end, the message corresponding to the number in the target end according to the number includes:
acquiring a first number corresponding to a current reading action and a second number corresponding to a previous reading action;
and reading the message corresponding to the first number when the first number is larger than the second number.
Optionally, after the step of obtaining the first number corresponding to the current reading action and the second number corresponding to the previous reading action, the method further includes:
and when the first number is less than or equal to the second number, ignoring the first number and executing the next reading action.
Optionally, the flow calculation method further includes:
after the stream computing system is abnormally restarted, reading the number corresponding to the last message in the target end;
and determining a starting point when the source data is read according to the serial number, and executing the step of reading the source data by the stream computing system based on the inspiration point.
Optionally, after the step of removing the duplicate of the serial number by the application terminal and reading the message corresponding to the serial number in the target terminal according to the serial number after the removal of the duplicate, the method further includes:
acquiring a service timestamp and/or a service field of each message;
sequencing the messages according to the service timestamps and/or the service fields;
and the application end processes the message according to the sequencing sequence.
Further, to achieve the above object, the present invention also provides a stream calculation apparatus including a memory, a processor, and a control program of the stream calculation apparatus stored on the memory and executable on the processor, the control program of the stream calculation apparatus implementing the steps of the stream calculation method as described above when executed by the processor.
In addition, to achieve the above object, the present invention further provides a stream computing system, where the stream computing system includes a data source end, a target end and an application end, and the stream computing system is configured to read and write source end data of the data source end into a message queue of the target end, so that the application end reads messages in the message queue according to message numbers.
Further, to achieve the above object, the present invention also provides a computer-readable storage medium having stored thereon a control program of a stream calculating apparatus, which when executed by a processor, realizes the steps of the stream calculating method as described above.
In the stream calculation method, the stream calculation device, and the computer readable storage medium according to the embodiments of the present invention, a stream calculation system first reads source data, and then writes the source data to a destination to generate a message with a number in the destination, where the number corresponds to the message one to one and the number monotonically increases, and an application reads the message corresponding to the number in the destination according to the number, and since the number corresponds to the message one to one and monotonically increases, downstream deduplication can be performed according to the number to avoid multiple impacts on output caused by the same input, and meanwhile, since the stream calculation system implements only one-time processing according to the message number, an effect of improving compatibility of stream calculation is achieved.
Drawings
Fig. 1 is a schematic terminal structure diagram of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart diagram illustrating an embodiment of a flow calculation method according to the present invention;
fig. 3 is a schematic flow chart of another embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In a conventional data processing flow, data is always collected and then placed in a database. When people need to get answers or perform relevant processing, the data can be queried through the database. This seems reasonable, but the result is very compact and does not solve the problem well, especially in the face of some specific problems in some real-time search application environments. This leads to a new data computation structure-stream computation approach. In the flow calculation process, in order to improve the accuracy of data processing, each input message needs to affect the final result only once, and even if a machine fault or software crash occurs, it is ensured that data is not processed repeatedly.
In the conventional stream calculation process, only one-time processing can be realized based on Flink (frienk, an open source stream processing framework). However, the Flink framework is complex, and cannot solve the problem that a simple independent Java or C + + program is processed only once, and a characteristic downstream component must be used, and cannot support other downstream components such as a database. This has the disadvantage of poor compatibility with stream computations.
To solve the above-mentioned drawbacks, an embodiment of the present invention mainly provides a flow calculation method, and its main solution is:
the stream computing system reads source data;
writing the source end data into a target end to produce messages with numbers in the target end, wherein the numbers correspond to the messages one to one and are monotonically increased;
and the application terminal performs duplicate removal on the number, and reads the message corresponding to the number in the target terminal according to the number after the duplicate removal.
Because the number is in one-to-one correspondence with the message and is monotonically increased, downstream deduplication can be performed according to the label to avoid multiple influences on output caused by the same input, and meanwhile, the stream computing system realizes only one-time processing according to the message number, thereby achieving the effect of improving the compatibility of stream computing.
As shown in fig. 1, fig. 1 is a schematic terminal structure diagram of a hardware operating environment according to an embodiment of the present invention.
The terminal of the embodiment of the invention can be a computer or a server and other terminal equipment.
As shown in fig. 1, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), a mouse, etc., and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.
Those skilled in the art will appreciate that the terminal structure shown in fig. 1 is not intended to be limiting and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a control program of a stream computing apparatus.
In the terminal shown in fig. 1, the network interface 1004 is mainly used for connecting to a backend server and performing data communication with the backend server; the processor 1001 may be configured to call a control program of the stream calculation apparatus stored in the memory 1005, and perform the following operations:
the stream computing system reads source data;
writing the source end data into a target end to produce messages with numbers in the target end, wherein the numbers correspond to the messages one to one and are monotonically increased;
and the application terminal performs duplicate removal on the number, and reads the message corresponding to the number in the target terminal according to the number after the duplicate removal.
Further, the processor 1001 may call a control program of the stream calculation apparatus stored in the memory 1005, and also perform the following operations:
judging whether the source end data fails to be written or not;
and when the source end data writing fails, executing the step of writing the source end data into the target end.
Further, the processor 1001 may call a control program of the stream calculation apparatus stored in the memory 1005, and also perform the following operations:
acquiring a first number corresponding to a current reading action and a second number corresponding to a previous reading action;
and reading the message corresponding to the first number when the first number is larger than the second number.
Further, the processor 1001 may call a control program of the stream calculation apparatus stored in the memory 1005, and also perform the following operations:
and when the first number is less than or equal to the second number, ignoring the first number and executing the next reading action.
Further, the processor 1001 may call a control program of the stream calculation apparatus stored in the memory 1005, and also perform the following operations:
after the stream computing system is abnormally restarted, reading the number corresponding to the last message in the target end;
and determining a starting point when the source data is read according to the serial number, and executing the step of reading the source data by the stream computing system based on the inspiration point.
Further, the processor 1001 may call a control program of the stream calculation apparatus stored in the memory 1005, and also perform the following operations:
acquiring a service timestamp and/or a service field of each message;
sequencing the messages according to the service timestamps and/or the service fields;
and the application end processes the message according to the sequencing sequence.
Referring to fig. 2, in an embodiment of the stream calculation method of the present invention, the stream calculation method includes the steps of:
step S10, reading source data by the stream computing system;
step S20, writing the source data into a target end to produce messages with numbers in the target end, wherein the numbers correspond to the messages one by one and the numbers are monotonically increased;
and step S30, the application end reads the message corresponding to the number in the target end according to the number.
The execution subject of the stream calculation method is a stream calculation system, wherein the stream calculation system may be a stream calculation system based on a kafka (an open source stream processing platform) framework or other stream calculation framework. The present invention does not limit the flow computing framework of the flow computing system.
In the present embodiment, a flow calculation system based on the kafka flow calculation framework is explained as an execution subject.
The stream computing system can read the source end data, and then hash (hash) the source end data into different partitions (ordered data storage queues), i.e. convert the input with any length (as pre-mapping) into the message with fixed length by the hash algorithm.
Specifically, when source data is read, the read source data may be compressed into a message digest through a hash algorithm, and then the message digest may be stored in the partition of kafka. In the writing process, in order to avoid repeatedly writing data, the data is written in batches, namely, only one batch of data is written at each time.
Further, when the data write is successful, the stream computing system may read the next batch of data and perform a write operation on the next batch of data. When the data writing fails or the data writing is overtime, the writing operation can be executed again on the data which is currently written in failure, so that the aim of upstream deduplication of the stream computing system is fulfilled.
It should be noted that, since the data is hashed to different partitions located on different cluster nodes, each message is recorded with a message number, where the message number is gradually increased along with the increase of the message, that is, the message number is in one-to-one correspondence with the message and monotonically increases. That is, each message will have a number (i.e., the message is good). The application end can inquire and read the message through the number.
Alternatively, since the upstream program does not necessarily mean that data writing fails when a write timeout occurs, the data pits in the message queue have duplicate numbers. In order to realize only one-time processing, the phenomenon that the same message has multiple responses needs to be avoided. Therefore, the message in the target can be read at the application end, and the downstream deduplication is carried out according to the number.
Specifically, when the application terminal can read the message in the application terminal according to the number. When the application terminal obtains the message number sequence, the messages in the message queue of the target terminal can be sequentially obtained according to the corresponding numbers in the message number sequence. Because the messages in the message queue should have monotonically increasing numbering values when the messages are not repeatedly written, but should have no monotonically increasing numbering sequence when repeated writing occurs. Downstream deduplication can be performed according to the number value. When the message is read according to the current number, whether the current number is larger than the previous number is judged, and if yes, the corresponding message is directly read according to the current number. Otherwise, the current number is ignored, and the message is read according to the next number.
For example, when the obtained number sequence is 1, 2, 3, 4, 5, 3, 4, the message corresponding to the number 1 may be read first, and then when the message corresponding to the number 2 is read, it may be determined whether the number 2 is greater than the previous number. Since number 2 is larger than number 1, the message corresponding to number 2 can be read directly. By analogy, the messages corresponding to the number 3, the number 4 and the number 5 can be read in sequence. Then, before the message corresponding to the number 3 is read next time, since the number 3 is smaller than the previous number 5, the second number 3 can be directly ignored, and the message is read according to the next number. Since the next number is 4 and the number 3 before the number 4 is ignored, the number 5 can be used as the previous number of the number 4, and since the number 5 is larger than the number 4, the number 4 can be ignored, i.e. the message corresponding to the number 4 is ignored.
In the technical scheme disclosed in this embodiment, a stream calculation system first reads source end data, and then writes the source end data into a destination end to produce a message with a number in the destination end, where the number corresponds to the message one to one and the number monotonically increases, and an application end reads the message corresponding to the number in the destination end according to the number, and since the number corresponds to the message one to one and monotonically increases, downstream deduplication can be performed according to the number to avoid multiple impacts on output caused by the same input, and meanwhile, since the stream calculation system implements only one-time processing according to the message number, an effect of improving compatibility of stream calculation is achieved.
Referring to fig. 3, based on the foregoing embodiment, in another embodiment, after the step S30, the method further includes:
step S40, acquiring a service time stamp and/or a service field of each message;
step S50, sorting the messages according to the service time stamps and/or the service fields;
and step S60, the application terminal processes the message according to the sorting sequence.
In this embodiment, for a more complex program, it is generally not guaranteed that the result of the program is idempotent, i.e., the results of multiple runs are different. Using the above method alone may result in data duplication or loss. It is therefore desirable to achieve idempotent by ordering the messages. The first is ordering by traffic timestamp, and when the traffic timestamps are equal, by traffic field.
It will be appreciated that the ordering may be based on other fields that ensure that the messages have a unique order.
And then the messages are processed in sequence according to the sequencing result.
In the technical solution disclosed in this embodiment, since the service timestamp and/or the service field of each message can be obtained, then the messages are sequenced according to the service timestamp and/or the service field, and the messages are processed according to the sequencing order by the application end, an idempotent effect of a complex program is achieved.
Furthermore, an embodiment of the present invention further provides a stream computing apparatus, where the stream computing apparatus includes a memory, a processor, and a control program of the stream computing apparatus that is stored in the memory and is executable on the processor, and the control program of the stream computing apparatus implements the steps of the stream computing method according to each of the above embodiments when executed by the processor.
In addition, an embodiment of the present invention further provides a stream computing system, where the stream computing system includes a data source end, a target end and an application end, and the stream computing system is configured to read and write source end data of the data source end into a message queue of the target end, so that the application end reads a message in the message queue according to a message number.
Further, an embodiment of the present invention also proposes a computer-readable storage medium having stored thereon a control program of a stream calculation apparatus, which when executed by a processor, implements the steps of the stream calculation method according to the above respective embodiments.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) as described above and includes several instructions for enabling a terminal device (e.g. computer, server, etc.) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (9)
1. A flow calculation method characterized by comprising the steps of:
the stream computing system reads source data;
writing the source end data into a target end to produce messages with numbers in the target end, wherein the numbers correspond to the messages one to one and are monotonically increased;
and the application terminal performs duplicate removal on the number, and reads the message corresponding to the number in the target terminal according to the number after the duplicate removal.
2. The stream computing method of claim 1, wherein the step of writing the source data to a destination to produce a numbered message in the destination further comprises, after the step of writing the source data to the destination:
judging whether the source end data fails to be written or not;
and when the source end data writing fails, executing the step of writing the source end data into the target end.
3. The stream computing method according to claim 1, wherein the step of the application end reading the message corresponding to the number in the destination end according to the number comprises:
acquiring a first number corresponding to a current reading action and a second number corresponding to a previous reading action;
and reading the message corresponding to the first number when the first number is larger than the second number.
4. The stream computing method according to claim 3, wherein the step of acquiring the first number corresponding to the current reading action and the second number corresponding to the previous reading action is followed by further comprising:
and when the first number is less than or equal to the second number, ignoring the first number and executing the next reading action.
5. The flow calculation method according to claim 1, wherein the flow calculation method further comprises:
after the stream computing system is abnormally restarted, reading the number corresponding to the last message in the target end;
and determining a starting point when the source data is read according to the serial number, and executing the step of reading the source data by the stream computing system based on the inspiration point.
6. The stream computing method according to claim 1, wherein, after the step of the application performing deduplication on the number and reading the message corresponding to the number in the destination according to the number subjected to deduplication, the method further comprises:
acquiring a service timestamp and/or a service field of each message;
sequencing the messages according to the service timestamps and/or the service fields;
and the application end processes the message according to the sequencing sequence.
7. A stream computing apparatus, characterized in that the stream computing apparatus comprises: memory, a processor and a control program of a stream computing apparatus stored on the memory and executable on the processor, the control program of the stream computing apparatus implementing the steps of the stream computing method according to any one of claims 1 to 6 when executed by the processor.
8. A stream computing system comprising a data source, a target and an application, the stream computing system being arranged to read and write source data of the data source to a message queue of the target, such that the application reads messages in the message queue according to message numbers.
9. A computer-readable storage medium, characterized in that a control program of a stream computing apparatus is stored thereon, which when executed by a processor implements the steps of the stream computing method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911314030.9A CN111104428A (en) | 2019-12-18 | 2019-12-18 | Flow calculation method, flow calculation device, flow calculation system, and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911314030.9A CN111104428A (en) | 2019-12-18 | 2019-12-18 | Flow calculation method, flow calculation device, flow calculation system, and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111104428A true CN111104428A (en) | 2020-05-05 |
Family
ID=70422546
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911314030.9A Pending CN111104428A (en) | 2019-12-18 | 2019-12-18 | Flow calculation method, flow calculation device, flow calculation system, and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111104428A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114157730A (en) * | 2021-10-26 | 2022-03-08 | 武汉光迅信息技术有限公司 | Message duplicate removal method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150134795A1 (en) * | 2013-11-11 | 2015-05-14 | Amazon Technologies, Inc. | Data stream ingestion and persistence techniques |
| CN105723679A (en) * | 2013-11-11 | 2016-06-29 | 亚马逊科技公司 | Client-configurable security options for data streams |
| CN106549990A (en) * | 2015-09-18 | 2017-03-29 | 阿里巴巴集团控股有限公司 | A kind of processing method and system of distributed data |
-
2019
- 2019-12-18 CN CN201911314030.9A patent/CN111104428A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150134795A1 (en) * | 2013-11-11 | 2015-05-14 | Amazon Technologies, Inc. | Data stream ingestion and persistence techniques |
| CN105723679A (en) * | 2013-11-11 | 2016-06-29 | 亚马逊科技公司 | Client-configurable security options for data streams |
| CN106549990A (en) * | 2015-09-18 | 2017-03-29 | 阿里巴巴集团控股有限公司 | A kind of processing method and system of distributed data |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114157730A (en) * | 2021-10-26 | 2022-03-08 | 武汉光迅信息技术有限公司 | Message duplicate removal method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9639444B2 (en) | Architecture for end-to-end testing of long-running, multi-stage asynchronous data processing services | |
| CN110502546A (en) | A kind of data processing method and device | |
| CN109669795B (en) | Crash information processing method and device | |
| WO2019019652A1 (en) | Push-information processing method, apparatus, computer device, and storage medium | |
| CN110955409B (en) | Method and device for creating resources on cloud platform | |
| CN110647562B (en) | Data query method and device, electronic equipment and storage medium | |
| CN111443920B (en) | Frame migration method and device | |
| CN110825731A (en) | Data storage method and device, electronic equipment and storage medium | |
| CN111104324A (en) | Test case generation method, test case generation device and computer-readable storage medium | |
| CN110888816A (en) | Program testing method, program testing device, and storage medium | |
| CN108595178B (en) | Hook-based data acquisition method, device and equipment | |
| CN109559121B (en) | Transaction path call exception analysis method, device, equipment and readable storage medium | |
| CN111104428A (en) | Flow calculation method, flow calculation device, flow calculation system, and medium | |
| CN110011845B (en) | Log collection method and system | |
| CN113760242A (en) | Data processing method, device, server and medium | |
| CN111125163B (en) | Method and device for processing data | |
| CN113965522B (en) | Behavior log grouping method, device, storage medium and equipment | |
| CN106469086B (en) | Event processing method and device | |
| CN111078571B (en) | Test method for analog response, terminal equipment and computer readable storage medium | |
| CN114036132A (en) | Object information processing method and device, storage medium and electronic equipment | |
| CN111782479A (en) | Log processing method and device, electronic equipment and computer readable storage medium | |
| CN112181802A (en) | Test method and device | |
| CN115543227B (en) | Cross-system data migration method, system, electronic device and storage medium | |
| CN114090269B (en) | Service scheduling balancing method, device, equipment and storage medium | |
| CN116861455B (en) | Event data processing method, system, electronic device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200505 |
|
| RJ01 | Rejection of invention patent application after publication |