CN110019310A - Data processing method and system, computer system, computer readable storage medium - Google Patents
Data processing method and system, computer system, computer readable storage medium Download PDFInfo
- Publication number
- CN110019310A CN110019310A CN201711477730.0A CN201711477730A CN110019310A CN 110019310 A CN110019310 A CN 110019310A CN 201711477730 A CN201711477730 A CN 201711477730A CN 110019310 A CN110019310 A CN 110019310A
- Authority
- CN
- China
- Prior art keywords
- database
- data
- change data
- stored
- state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
- G06F16/244—Grouping and aggregation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Present disclose provides a kind of data processing methods, this method comprises: obtaining the change data that data manipulation generates；Change data are stored in first database, wherein first database supports data aggregate inquiry；And change data are stored in the second database, wherein the second database also supports that data aggregate is inquired, for the standby database as first database.The disclosure additionally provides a kind of data processing system, a kind of computer system and a kind of computer readable storage medium.
Description
Technical field
This disclosure relates to data processing field, more particularly, to a kind of data processing method and system, department of computer science
System, computer readable storage medium.
Background technique
Internet development swift and violent today, the increment of data is out of imagination, in the tremendous expansion of geometry character, and nothing
By what system, with the development of time and business, data involved in system are also gradually increasing.Previous Oracle or
The payment datas such as person Sqlserver library, with the increase of data database storing, resources costs are also increase accordingly.And it is every
CPU, disk, the memory of platform machine are all limited, and when data volume is very big, database performance can be excessively poor.Therefore increasingly
More enterprises all uses the distributed storage mode that table is divided in point library.
Since data divide by point library the distributed storage form of table, it is stored in breaing up on multiple data base machines, because
This is unable to satisfy the demand of some various dimensions inquiries, but often has certain big datas to need externally to provide inquiry, and inquire
Dimension is also required to diversification, or even including paging demand.
Currently, data point library divide table store after, the prior art provide aggregate query it is general there are two types of: one, pass through timing
Point library is divided the data in table to store into single table by the mode of scheduling according within the scope of certain time, such as a monthly table, user
The aggregate query of single table is carried out when inquiry by screening month；It two, will be within the scope of certain time by way of timer-triggered scheduler
Data store into a database, be used for aggregate query.
However, at least there are the following problems in the related technology for inventor's discovery: looking into during realizing disclosure design
There is the risk that can not be supported in the service of inquiry.
Summary of the invention
In view of this, present disclose provides a kind of data that can reduce aggregate query service and there is the risk that can not be supported
Processing method and data processing system.
An aspect of this disclosure provides a kind of data processing method, comprising: obtains the change number that data manipulation generates
According to；Above-mentioned change data are stored in first database, wherein above-mentioned first database supports data aggregate inquiry；And it will
Above-mentioned change data are stored in the second database, wherein above-mentioned second database also supports data aggregate to inquire, for being used as
State the standby database of first database.
In accordance with an embodiment of the present disclosure, the above method further include: by first thread that the deposit of above-mentioned change data is above-mentioned
In first database；And by being different from above-mentioned first thread and second thread synchronous with above-mentioned first thread is by above-mentioned change
More data are stored in above-mentioned second database.
In accordance with an embodiment of the present disclosure, the above method further include: be sent to above-mentioned change data by Kafka message
State first database first receives server, so that the data that above-mentioned first reception server sends above-mentioned Kafka message
It is stored in above-mentioned first database；And above-mentioned change data are sent to simultaneously by above-mentioned second number by above-mentioned Kafka message
Server is received according to the second of library, so that the data deposit that above-mentioned second reception server sends above-mentioned Kafka message is above-mentioned
In second database.
In accordance with an embodiment of the present disclosure, the above method further include: after obtaining above-mentioned change data, by above-mentioned change data
As in a task record write-in task list；During storing above-mentioned change data, check above-mentioned first database and
Whether the state value of above-mentioned second database is all that state is completed；And in above-mentioned first database and/or above-mentioned second number
State value according to library will be written by above-mentioned Kafka message above-mentioned in above-mentioned task list in the case where state is completed
Above-mentioned change data in task record are sent to the above-mentioned first reception server and/or above-mentioned second of above-mentioned first database
Above-mentioned the second of database receives server, be to guarantee state value not the state that is completed database can successfully be stored in it is above-mentioned
Change data.
In accordance with an embodiment of the present disclosure, the above method further include: will be stored in above-mentioned first database earlier than preset time
Historical data delete.
In accordance with an embodiment of the present disclosure, the above method further include: after obtaining above-mentioned change data, by above-mentioned change data
Point library divides table to be stored in a distributed experiment & measurement system.
Another aspect of the disclosure provides a kind of data processing system, comprising: module is obtained, for obtaining data behaviour
Make the change data generated；First memory module, for above-mentioned change data to be stored in first database, wherein above-mentioned the
One database supports data aggregate inquiry；And second memory module, for above-mentioned change data to be stored in the second database,
Wherein, above-mentioned second database also supports data aggregate to inquire, for the standby database as above-mentioned first database.
In accordance with an embodiment of the present disclosure, above-mentioned first memory module is also used to above-mentioned change data through first thread
It is stored in above-mentioned first database；And above-mentioned second memory module, be also used to by be different from above-mentioned first thread and with it is upper
It states the second synchronous thread of first thread above-mentioned change data are stored in above-mentioned second database.
In accordance with an embodiment of the present disclosure, above-mentioned first memory module is also used to above-mentioned change data through Kafka message
Be sent to above-mentioned first database first receives server, so that above-mentioned first reception server sends out above-mentioned Kafka message
The data sent are stored in above-mentioned first database；And above-mentioned second memory module, it is also used to through above-mentioned Kafka message simultaneously
Second that above-mentioned change data are sent to above-mentioned second database receives server, so that above-mentioned second reception server will be upper
The data for stating the transmission of Kafka message are stored in above-mentioned second database.
In accordance with an embodiment of the present disclosure, above system further include: third memory module, for obtaining above-mentioned change data
Afterwards, using above-mentioned change data as in a task record write-in task list；Enquiry module, for storing above-mentioned change data
During, check whether the state value of above-mentioned first database and above-mentioned second database is all that state is completed；Above-mentioned
One memory module, be also used to the state value of above-mentioned first database and above-mentioned second database be not the state that is completed or on
The state value for stating first database is not that above-mentioned task list will be written by above-mentioned Kafka message in the case where state is completed
In above-mentioned task record in above-mentioned change data be sent to above-mentioned first database it is above-mentioned first receive server, with protect
Card state value is not that the above-mentioned first database for the state that is completed can successfully be stored in above-mentioned change data；And above-mentioned second deposit
Module is stored up, is also used in the state value of above-mentioned first database and above-mentioned second database not be the state that is completed or above-mentioned the
The state value of two databases is not that will be written in above-mentioned task list in the case where state is completed by above-mentioned Kafka message
Above-mentioned change data in above-mentioned task record are sent to the above-mentioned second reception server of above-mentioned second database, to guarantee shape
State value is not that above-mentioned second database for the state that is completed can successfully be stored in above-mentioned change data.
In accordance with an embodiment of the present disclosure, above system further include: removing module, for by above-mentioned first database earlier than
The historical data of preset time deposit is deleted.
In accordance with an embodiment of the present disclosure, above system further include: the 4th memory module, for obtaining above-mentioned change data
Afterwards, table is divided to be stored in a distributed experiment & measurement system in above-mentioned change data point library.
Another aspect of the present disclosure provides a kind of computer system, comprising: one or more processors；Memory is used
In the one or more programs of storage, wherein when said one or multiple programs are executed by said one or multiple processors, make
It obtains said one or multiple processors realizes data processing method described in any of the above embodiments.
Another aspect of the present disclosure provides a kind of computer readable storage medium, is stored thereon with executable instruction, should
Instruction makes processor realize data processing method described in any of the above embodiments when being executed by processor.
By the embodiment of the present disclosure, because using two mutual backup and all support data aggregate query services number
The technological means that change data are stored according to library (such as first database and the second database), is looked into so at least partially overcoming
It askes and services not quietly, there is technical issues that technology that is not available, and then having reached the availability of raising query service
Effect.For example, first database is Elasticsearch database, for saving hot spot data, first database is
MongoDB database can guarantee still may be used when delay machine situation occurs in any database in this way for saving full dose data
To provide aggregate query service, thus it is mutually standby by Elasticsearch database and MongoDB database, it can be improved poly-
Close the availability of query service.
Detailed description of the invention
By referring to the drawings to the description of the embodiment of the present disclosure, the above-mentioned and other purposes of the disclosure, feature and
Advantage will be apparent from, in the accompanying drawings:
Fig. 1 is diagrammatically illustrated can be using the data processing method of the disclosure and the exemplary system architecture of system；
Fig. 2 diagrammatically illustrates the data processing method and systematic difference scene according to the embodiment of the present disclosure；
Fig. 3 diagrammatically illustrates the flow chart of the data processing method according to the embodiment of the present disclosure；
Fig. 4 A diagrammatically illustrates the schematic diagram of the data processing method according to another embodiment of the disclosure；
Fig. 4 B diagrammatically illustrates the schematic diagram of the data processing method according to another embodiment of the disclosure；
Fig. 4 C diagrammatically illustrates the schematic diagram of the data processing method according to another embodiment of the disclosure；
Fig. 4 D diagrammatically illustrates the flow chart of the data processing method according to another embodiment of the disclosure；
Fig. 4 E diagrammatically illustrates the flow chart of the data processing method according to another embodiment of the disclosure；
Fig. 4 F diagrammatically illustrates the schematic diagram of the data processing method according to another embodiment of the disclosure；
Fig. 5 diagrammatically illustrates the block diagram of the data processing system according to the embodiment of the present disclosure；
Fig. 6 A diagrammatically illustrates the block diagram of the data processing system according to the embodiment of the present disclosure；
Fig. 6 B diagrammatically illustrates the block diagram of the data processing system according to the embodiment of the present disclosure；And
Fig. 7 diagrammatically illustrates the department of computer science for being adapted for carrying out data processing method and system according to the embodiment of the present disclosure
The block diagram of system.
Specific embodiment
Hereinafter, will be described with reference to the accompanying drawings embodiment of the disclosure.However, it should be understood that these descriptions are only exemplary
, and it is not intended to limit the scope of the present disclosure.In addition, in the following description, descriptions of well-known structures and technologies are omitted, with
Avoid unnecessarily obscuring the concept of the disclosure.
Term as used herein is not intended to limit the disclosure just for the sake of description specific embodiment.It uses herein
The terms "include", "comprise" etc. show the presence of the feature, step, operation and/or component, but it is not excluded that in the presence of
Or add other one or more features, step, operation or component.
There are all terms (including technical and scientific term) as used herein those skilled in the art to be generally understood
Meaning, unless otherwise defined.It should be noted that term used herein should be interpreted that with consistent with the context of this specification
Meaning, without that should be explained with idealization or excessively mechanical mode.
It, in general should be according to this using statement as " at least one in A, B and C etc. " is similar to
Field technical staff is generally understood the meaning of the statement to make an explanation (for example, " system at least one in A, B and C "
Should include but is not limited to individually with A, individually with B, individually with C, with A and B, with A and C, have B and C, and/or
System etc. with A, B, C).Using statement as " at least one in A, B or C etc. " is similar to, generally come
Saying be generally understood the meaning of the statement according to those skilled in the art to make an explanation (for example, " having in A, B or C at least
One system " should include but is not limited to individually with A, individually with B, individually with C, with A and B, have A and C, have
B and C, and/or the system with A, B, C etc.).It should also be understood by those skilled in the art that substantially arbitrarily indicating two or more
The adversative conjunction and/or phrase of optional project shall be construed as either in specification, claims or attached drawing
A possibility that giving including one of these projects, either one or two projects of these projects.For example, phrase " A or B " should
A possibility that being understood to include " A " or " B " or " A and B ".
Embodiment of the disclosure provides a kind of can reduce at the risk data that aggregate query service presence can not be supported
Reason method and the data processing system that this method can be applied.This method includes the change data for obtaining data manipulation and generating；
Above-mentioned change data are stored in first database, wherein above-mentioned first database supports data aggregate inquiry；And it will be above-mentioned
It changes data to be stored in the second database, wherein above-mentioned second database also supports data aggregate to inquire, for as above-mentioned the
The standby database of one database.
Fig. 1 is diagrammatically illustrated can be using the data processing method of the disclosure and the exemplary system architecture of system.
As shown in Figure 1, system architecture 100 may include terminal device 101,102,103, network according to this embodiment
104 and server 105.Network 104 between terminal device 101,102,103 and server 105 to provide communication link
Medium.Network 104 may include various connection types, such as wired and or wireless communications link etc..
User can be used terminal device 101,102,103 and be interacted by network 104 with server 105, to receive or send out
Send message etc..Various telecommunication customer end applications, such as the application of shopping class, net can be installed on terminal device 101,102,103
(merely illustrative) such as the application of page browsing device, searching class application, instant messaging tools, mailbox client and/or social platform softwares.
Terminal device 101,102,103 can be the various electronic equipments with display screen and supported web page browsing, packet
Include but be not limited to smart phone, tablet computer, pocket computer on knee and desktop computer etc..
Server 105 can be to provide the server of various services, such as utilize terminal device 101,102,103 to user
The website browsed provides the back-stage management server (merely illustrative) supported.Back-stage management server can be to the use received
The data such as family request analyze etc. processing, and by processing result (such as according to user's request or the webpage of generation, believe
Breath or data etc.) feed back to terminal device.
It should be noted that data processing method provided by the embodiment of the present disclosure can generally be executed by server 105.
Correspondingly, data processing system provided by the embodiment of the present disclosure generally can be set in server 105.The embodiment of the present disclosure
Provided data processing method can also by be different from server 105 and can with terminal device 101,102,103 and/or clothes
The server or server cluster that business device 105 communicates execute.Correspondingly, data processing system provided by the embodiment of the present disclosure
It can be set in the service that is different from server 105 and can be communicated with terminal device 101,102,103 and/or server 105
In device or server cluster.
It should be understood that the number of terminal device, network and server in Fig. 1 is only schematical.According to realization need
It wants, can have any number of terminal device, network and server.
Internet development swift and violent today, the increment of data is out of imagination, in the tremendous expansion of geometry character, and nothing
By what system, with the development of time and business, data involved in system are also gradually increasing.Previous Oracle or
The payment datas such as person Sqlserver library, with the increase of data database storing, resources costs are also increase accordingly.And it is every
CPU, disk, the memory of platform machine are all limited, and when data volume is very big, database performance can be excessively poor.Therefore increasingly
More enterprises all uses the distributed storage mode that table is divided in point library, as shown in Fig. 2, the data that data source 204 is generated store
In multiple and different Mysql databases 201.
Since data divide by point library the distributed storage form of table, it is stored in breaing up on multiple data base machines, because
This is unable to satisfy the demand of some various dimensions inquiries, but often has certain big datas to need externally to provide inquiry, and inquire
Dimension is also required to diversification, or even including paging demand.
Currently, data point library divide table store after, the prior art provide aggregate query it is general there are two types of: one, pass through timing
Point library is divided the data in table to store into single table by the mode of scheduling according within the scope of certain time, such as a monthly table, user
The aggregate query of single table is carried out when inquiry by screening month；It two, will be within the scope of certain time by way of timer-triggered scheduler
Data store into a database, be used for aggregate query.
However, at least there are the following problems in the related technology for inventor's discovery: looking into during realizing disclosure design
There is the risk that can not support in the service of inquiry, be based on this, as shown in Fig. 2, the data for generating data source 204 be stored in it is multiple not
While in same Mysql database 201, a hotspot database 202 and a full dose number can also be respectively stored in
According in library 203.Wherein, the hotspot database 202 and the full dose database 203 all support data aggregate to inquire, and look into data
Hotspot database 202 is preferentially used when inquiry, when hotspot database 202 is unavailable, full dose database 203 can be used and replace heat
Point data base 202 provides data aggregate query service.In this way, caused by can avoiding hotspot database 202 unavailable as far as possible
The risk that query service is not supported.
It should be noted that Mysql is a Relational DBMS.Relevant database saves the data in
In different tables of data, rather than all data are placed in one big warehouse, which improves data query speed and are mentioned
The high flexibility of data query.
Fig. 3 diagrammatically illustrates the flow chart of the data processing method according to the embodiment of the present disclosure.
As shown in figure 3, the system includes operation S301~S303, in which:
In operation S301, the change data that data manipulation generates are obtained；
It, will be in change data deposit first database, wherein first database supports data aggregate to look into operation S302
It askes；And
In operation S303, change data are stored in the second database, wherein the second database also supports data aggregate to look into
It askes, for the standby database as first database.
In the embodiments of the present disclosure, without limitation to data manipulation, it can include but is not limited to data newly-increased, modification,
The operation such as deletion.In addition, although first database and the second database can provide data aggregate query service, it is preferred that
, first database can be set to primary database, by and database is set as standby database, under normal circumstances by
First database provides data aggregate query service, when first database leads to unavailable due to exception or delay machine, by
Second database provides data aggregate query service.Based on this, it is generally desirable to first databases to have higher inquiry velocity, and
Support various dimensions inquiry and paging query, therefore, first database can be ElasticSearch database, the second database
It can be MongoDB database.
ElasticSearch is the search server based on Lucene.It provides a distributed multi-user energy
The full-text search engine of power is based on RESTful web interface.Elasticsearch is developed with Java, and as Apache
Open source code publication under license terms, is Enterprise search engine currently popular.Designed for that can reach in cloud computing
Search in real time has and stablizes, reliably, quickly, the features such as installing, is easy to use.
MongoDB is the database based on distributed document storage.It is write by C Plus Plus.It is intended that WEB application mentions
For expansible high-performance data storage solution.
Originally it is opening in embodiment, when data source has data generation, a distributed database can be stored it in
In cluster, to realize that a point library for data divides table to store, meanwhile, in order to realize various dimensions even paged data aggregate query, may be used also
These data to be respectively stored in the database of two mutual backup, i.e. first database and the second database.In this way, working as
One database can enable another database such as the second database to provide data aggregate when for example first database is unavailable
Query service.
And in the related art, for the data that point library divides table to store, when realizing data aggregate inquiry, a kind of mode is
Temporally divide table again, by way of timer-triggered scheduler, according within the scope of certain time by the data that point library is divided in table store to
In single table, a such as monthly table, when user query, carries out the aggregate query of single table by screening month.For this mode,
Since the inquiry operation across time degree is cumbersome, and in the case that data volume is very big, single table can still have query performance and ask
Topic, therefore user experience is bad.Another way is to provide data aggregate query service by Elasticsearch database.
Although Elasticsearch database can provide better query performance, good high availability not can guarantee, once it searches
Rope services delay machine, and external query service is with regard to unavailable, and to occupy resource many for this mode, can not store mass data, this
Outer Elasticsearch database itself the case where there are loss of data, etc..
Based on this, all there are the following problems for above two existing solution: due to the mode of timer-triggered scheduler, data letter
It is lower to cease real-time, is unable to satisfy the demand required to real-time；Single memory space is unable to satisfy the storage of big data
Demand；Not quietly, there are not available risks for query service.
And by the embodiment of the present disclosure, because using two mutual backup and all support data aggregate query service
Database (such as first database and the second database) changes the technological means of data to store, so at least partially overcoming
Query service quietly, not there is technical issues that skill that is not available, and then having reached the availability of raising query service
Art effect.For example, first database is Elasticsearch database, for saving hot spot data, first database is
MongoDB database can guarantee still may be used when delay machine situation occurs in any database in this way for saving full dose data
To provide aggregate query service, thus it is mutually standby by Elasticsearch database and MongoDB database, it can be improved poly-
Close the availability of query service.
Below with reference to Fig. 4 A~Fig. 4 F, system shown in Fig. 3 is described further in conjunction with specific embodiments.
As a kind of optional embodiment, the above method further include: the first number of data deposit will be changed by first thread
According in library；And by being different from first thread and second thread synchronous with first thread will change data the second data of deposit
In library.
As shown in Figure 4 A, the data 401 for needing to store are stored in first database 402 by first thread respectively, are led to
The second thread is crossed to be stored in the second database 403.
Since first thread and the second thread are different thread and are mutually synchronized, it can be improved and deposit using multithreading
The speed for storing up data, the needs of when so as to meet data query to timeliness, and Multi-thread synchronization can then guarantee first
Database 402 and the second database 403 can store same data, so that no matter using first database 402 or second
Database 403 carries out that consistent accurate query result can be obtained when data query.
It should be noted that multithreading refers to the technology for realizing that multiple threads are concurrently executed from software or hardware.
By the embodiment of the present disclosure, data are saved into multi-dimensional data library by multithreading, data can be improved and deposit
The timeliness of storage, so as to provide a kind of higher aggregate query service of real-time accuracy.
As a kind of optional embodiment, the above method further include: be sent to first for data are changed by Kafka message
The first of database receives server, so that the data that the first reception server sends Kafka message are stored in first database
In；And the second reception server that data are sent to the second database will be changed simultaneously by Kafka message, so that second connects
The data that server sends Kafka message are received to be stored in the second database.
As shown in Figure 4 B, data can be stored in by multithreading and Kafka message 406 by 402 He of first database respectively
In second database 403.As shown in Figure 4 C, data can be stored in by Kafka message by first database 402 and second respectively
In database 403.For through multithreading and Kafka message storing data, specifically, as shown in Figure 4 B, it will need first
The data 401 of storage input Kafka message 406 (i.e. in Kafka message queue), then are divided these data by Kafka message 406
The first reception server 404 is not sent to by first thread, the second reception server 405 is sent to by the second thread, so
Server 404 is received by first afterwards, first database 402 is written into these data, counted these by the second reception server 405
According to the second database 403 of write-in.
It should be noted that Kafka be a distribution, subregion, more copies, more subscribers, be based on zookeeper
The distributed information log system (MQ system can also be regarded) of coordination.
By the embodiment of the present disclosure, data can be saved into multi-dimensional database by multithreading and Kafka message, by
Storage operation is carried out using different threads in different data, and Kafka message can provide better handling capacity and higher
Performance, therefore be able to satisfy the requirement to real-time property.
As a kind of optional embodiment, the above method further include: after obtaining change data, data will be changed as one
In task record write-in task list；During data are changed in storage, the shape of first database and the second database is checked
Whether state value is all that state is completed；It and in the state value of first database and/or the second database is not that state is completed
In the case where, the change data in the task record being written in task list are sent to by first database by Kafka message
First receives the second reception server of server and/or the second database, is the number for the state that is completed to guarantee state value not
Change data can be successfully stored according to library.
As shown in Figure 4 D, it during data 401 that storage needs to store, can be added in task list 407 simultaneously
One task record, for recording the data currently stored, in this way, when in first database 402 and the second database 403
When the case where data storage failure occur in any one or two, when storing failure such as first database 402, it can pass through
Data in this task record recorded in task list 407 are sent to the first reception server 404 by Kafka message, in turn
By the first reception server 404 data write operation is executed to first database 402 again, so recycled, until data are write
Until entering successfully.Preferably, after data are written successfully, the task of the correspondence database in synchronized update task list 407 is needed
Completion status.
It specifically, can be same based on tools, timing scan synchronous task table, real-time queries such as timer-triggered scheduler frame Quartz
Each record in task list is walked, judges the E item task (i.e. the store tasks of first database) and M tasks of every record
Whether (i.e. the store tasks of the second database) are that state is completed.If the E item of task record S2 does not complete, pass through
Data A2 representated by S2 is sent to Elasticsearch and receives server (i.e. first receives server) by Kafka message,
Elasticsearch receives server and receives data A2, is updated in Elasticsearch database, updates and completes
Afterwards, the E column in S2 are updated to that state is completed.Equally, if the M item of record S2 does not complete, pass through Kafka message for S2
Representative data A2 is sent to MongoDB and receives server (i.e. second receives server), and MongoDB receives server and receives
It to data A2, is updated in MongoDB database, after the completion of update, the M column in S2 is updated to that state is completed.
By the embodiment of the present disclosure, increasing synchronous task table can be guaranteed all by handling synchronous task table in real time
Data can be complete and accurately write into Elasticsearch database and MongoDB database, and externally provide quasi-
True ground query service.
As a kind of optional embodiment, as shown in Figure 4 E, the above method further include:
In operation S304, the historical data in first database earlier than preset time deposit is deleted.
Specifically, in the embodiments of the present disclosure, with first database for Elasticsearch database and the second database
For MongoDB database, Elasticsearch data can be stored data in based on inquiry temperature and query time
In library and MongoDB database, while externally providing various dimensions query service.Wherein it is possible to pass through Elasticsearch data
Library save hot spot data, by MongoDB database save full dose data, in this way, Elasticsearch database and
MongoDB database mutual backup, it is ensured that in any database delay machine, still can externally provide inquiry clothes
Business, and historical data expired in periodic cleaning Elasticsearch database, it is ensured that Elasticsearch data
The high-performance of library inquiry service provides search efficiency high query service, so that Elasticsearch database and MongoDB
Database is capable of providing the query service of different temperatures.
As a kind of optional embodiment, as illustrated in figure 4f, the above method further include: after obtaining change data, will become
More data point library divides table to be stored in a distributed experiment & measurement system 408, to realize that big data point library divides table to store.
The disclosure is elaborated below by way of a specific embodiment:
Generally, normal database information storage is that the data for increasing newly, modifying, deleting are divided to the rule of table by point library
It updates into corresponding Mysql database, such as newly-increased A data.By technical solution provided by the present disclosure, can be based on multi-thread
Journey, asynchronous one data synchronous task of rapid increasing new records S1 in synchronous task table, this synchronous task table is for guaranteeing number
According to the integrality and accuracy of stored data in library.There are two tasks to arrange for synchronous task table, and E and M column respectively represent storage
Two task items of Elasticsearch database and storage MongoDB database.At the same time, it is based on Kafka message queue
Newly-increased A data are sent to the reception server of the two databases.Due to Kafka message can provide better handling capacity with
And higher performance, therefore it is able to satisfy the requirement to real-time property.
It is for grabbing the message in Kafka in real time and carrying out data write-in processing that Elasticsearch, which receives server,
Service, after the reception server grabs the message of newly-increased A data in real time, by A data store to
In Elasticsearch database, after storing successfully, the E for updating synchronous task table S1 record is classified as the state of being completed.
It is that the service handled is written for grabbing the message in Kafka in real time and carrying out data that MongoDB, which receives server,
After the reception server grabs the message of newly-increased A data in real time, A data are stored into MongoDB database, are deposited
After storing up successfully, the M for updating synchronous task table S1 record is classified as the state of being completed.
According to user using characteristic and the characteristic in storing data library, when Elasticsearch database provides common
Between hot spot data query service in section.The data information of MongoDB database purchase full dose, for doing history number if necessary
According to aggregate query take over the offer of Elasticsearch database and when Elasticsearch database service delay machine
Query service guarantees the high availability of aggregate query service.
By the embodiment of the present disclosure, the synchronous of data information can be carried out with Kafka message based on multithreading, according to inquiry
The characteristics of data, stores data into Elasticsearch and MongoDB respectively, not only can satisfy the requirement of timeliness,
But also it can guarantee the high availability of aggregate query service.
Fig. 5 diagrammatically illustrates the block diagram of the data processing system according to the embodiment of the present disclosure.
As shown in figure 5, the data processing system 500 includes obtaining module 510, the first memory module 520 and the second storage
Module 530.
Module 510 is obtained, for obtaining the change data of data manipulation generation；First memory module 520, for that will change
Data are stored in first database, wherein first database supports data aggregate inquiry；And second memory module 530, it is used for
Change data are stored in the second database, wherein the second database also supports that data aggregate is inquired, for being used as the first data
The standby database in library.
By the embodiment of the present disclosure, because using two mutual backup and all support data aggregate query services number
The technological means that change data are stored according to library (such as first database and the second database), is looked into so at least partially overcoming
It askes and services not quietly, there is technical issues that technology that is not available, and then having reached the availability of raising query service
Effect.For example, first database is Elasticsearch database, for saving hot spot data, first database is
MongoDB database can guarantee still may be used when delay machine situation occurs in any database in this way for saving full dose data
To provide aggregate query service, thus it is mutually standby by Elasticsearch database and MongoDB database, it can be improved poly-
Close the availability of query service.
As a kind of optional embodiment, above-mentioned first memory module is also used to deposit by first thread by data are changed
Enter in first database；And second memory module, it is also used to by being different from first thread and synchronous with first thread the
Two threads will change data and be stored in the second database.
By the embodiment of the present disclosure, data are saved into multi-dimensional data library by multithreading, data can be improved and deposit
The timeliness of storage, so as to provide a kind of higher aggregate query service of real-time accuracy.
As a kind of optional embodiment, above-mentioned first memory module is also used to that data hair will be changed by Kafka message
Give first database first receives server, so that the data deposit that the first reception server sends Kafka message
In one database；And second memory module, it is also used to that change data are sent to the second database simultaneously by Kafka message
Second receive server so that the data that the second reception server sends Kafka message are stored in the second database.
By the embodiment of the present disclosure, data can be saved into multi-dimensional database by multithreading and Kafka message, by
Storage operation is carried out using different threads in different data, and Kafka message can provide better handling capacity and higher
Performance, therefore be able to satisfy the requirement to real-time property.
As a kind of optional embodiment, above system further include: third memory module, for obtaining change data
Afterwards, using change data as in a task record write-in task list；Enquiry module, for the process in storage change data
In, check whether the state value of first database and the second database is all that state is completed；First memory module, is also used to
The state value of first database and the second database is not that the state value of the state that is completed or first database is not completed
In the case where state, the change data in the task record being written in task list are sent to by the first data by Kafka message
The first of library receives server, is that the first database for the state that is completed can successfully be stored in change number to guarantee state value not
According to；And second memory module, be also used to the state value of first database and the second database be not the state that is completed or
The state value of second database is remembered by Kafka message by the task in task list is written in the case where state is completed
What the change data in record were sent to the second database second receives server, is the of the state that is completed to guarantee state value not
Two databases can successfully be stored in change data.
By the embodiment of the present disclosure, increasing synchronous task table can be guaranteed all by handling synchronous task table in real time
Data can be complete and accurately write into Elasticsearch database and MongoDB database, and externally provide quasi-
True ground query service.
As a kind of optional embodiment, as shown in Figure 6A, above system further include: removing module 540 is used for first
Historical data in database earlier than preset time deposit is deleted.
By the embodiment of the present disclosure, hot spot data can be saved by Elasticsearch database, pass through MongoDB
Database saves full dose data, in this way, Elasticsearch database and MongoDB database mutual backup, it is ensured that
In the case of any database delay machine, query service, and periodic cleaning Elasticsearch database still can be externally provided
In expired historical data, it is ensured that it is high to provide search efficiency for the high-performance of Elasticsearch database enquiry services
Query service, so that Elasticsearch database and MongoDB database are capable of providing the query service of different temperatures.
As a kind of optional embodiment, as shown in Figure 6B, above system further include: the 4th memory module 550 is used for
After obtaining change data, table is divided to be stored in a distributed experiment & measurement system in change data point library.
By the embodiment of the present disclosure, the synchronous of data information can be carried out with Kafka message based on multithreading, according to inquiry
The characteristics of data, stores data into Elasticsearch and MongoDB respectively, not only can satisfy the requirement of timeliness,
But also it can guarantee the high availability of aggregate query service.
It is understood that obtaining module 510, the first memory module 520 and second memory module 530 etc. may be incorporated in
It is realized in one module or any one module therein can be split into multiple modules.Alternatively, one in these modules
At least partly function of a or multiple modules can be combined at least partly function of other modules, and real in a module
It is existing.According to an embodiment of the invention, obtaining in module 510, the first memory module 520 and second memory module 530 etc. at least
One can at least be implemented partly as hardware circuit, such as field programmable gate array (FPGA), programmable logic array
(PLA), system on chip, the system on substrate, the system in encapsulation, specific integrated circuit (ASIC), or can with to circuit into
Row is integrated or the hardware such as any other rational method of encapsulation or firmware are realized, or with software, hardware and three kinds of firmware
The appropriately combined of implementation is realized.Alternatively, obtaining module 510, the first memory module 520 and second memory module 530 etc.
At least one of can at least be implemented partly as computer program module, can be with when the program is run by computer
Execute the function of corresponding module.
Fig. 7 diagrammatically illustrates the frame of the computer system for being adapted for carrying out data processing method according to the embodiment of the present disclosure
Figure.Computer system shown in Fig. 7 is only an example, should not function to the embodiment of the present disclosure and use scope bring and appoint
What is limited.
As shown in fig. 7, include processor 701 according to the computer system 700 of the embodiment of the present disclosure, it can be according to storage
It is loaded into random access storage device (RAM) 703 in the program in read-only memory (ROM) 702 or from storage section 708
Program and execute various movements appropriate and processing.Processor 701 for example may include general purpose microprocessor (such as CPU), refer to
Enable set processor and/or related chip group and/or special microprocessor (for example, specific integrated circuit (ASIC)), etc..Processing
Device 701 can also include the onboard storage device for caching purposes.Processor 701 may include for executing with reference to Fig. 3, Fig. 4 A
Single treatment unit either multiple processing of the different movements of the system flow according to the embodiment of the present disclosure of~Fig. 4 F description
Unit.
In RAM 703, it is stored with system 700 and operates required various programs and data.Processor 701, ROM 702 with
And RAM 703 is connected with each other by bus 704.Processor 701 is held by executing the program in ROM 702 and/or RAM 703
Row is above with reference to Fig. 3, the various operations of Fig. 4 A~Fig. 4 F description.It is noted that described program also can store except ROM 702
In one or more memories other than RAM 703.Processor 701 can also be stored in one or more of by execution
Program in memory is executed above with reference to Fig. 3, the various operations of Fig. 4 A~Fig. 4 F description.
In accordance with an embodiment of the present disclosure, system 700 can also include input/output (I/O) interface 705, input/output
(I/O) interface 705 is also connected to bus 704.System 700 can also include be connected to I/O interface 705 with one in lower component
Item is multinomial: the importation 706 including keyboard, mouse etc.；Including such as cathode-ray tube (CRT), liquid crystal display (LCD)
Deng and loudspeaker etc. output par, c 707；Storage section 708 including hard disk etc.；And including such as LAN card, modulatedemodulate
Adjust the communications portion 709 of the network interface card of device etc..Communications portion 709 executes communication process via the network of such as internet.
Driver 710 is also connected to I/O interface 705 as needed.Detachable media 711, such as disk, CD, magneto-optic disk, semiconductor
Memory etc. is mounted on as needed on driver 710, in order to be pacified as needed from the computer program read thereon
It is packed into storage section 708.
In accordance with an embodiment of the present disclosure, it may be implemented as computer software journey above with reference to the system of flow chart description
Sequence.For example, embodiment of the disclosure includes a kind of computer program product comprising be carried on computer readable storage medium
Computer program, which includes the program code for system shown in execution flow chart.In such implementation
In example, which can be downloaded and installed from network by communications portion 709, and/or from detachable media 711
It is mounted.When the computer program is executed by processor 701, the above-mentioned function limited in the system of the embodiment of the present disclosure is executed
Energy.In accordance with an embodiment of the present disclosure, system as described above, unit, module, unit etc. can pass through computer program
Module is realized.
It should be noted that computer readable storage medium shown in the disclosure can be computer-readable signal media or
Person's computer readable storage medium either the two any combination.Computer readable storage medium for example can be ---
But be not limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above group
It closes.The more specific example of computer readable storage medium can include but is not limited to: have being electrically connected for one or more conducting wires
Connect, portable computer diskette, hard disk, random access storage device (RAM), read-only memory (ROM), erasable type may be programmed it is read-only
Memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory
Part or above-mentioned any appropriate combination.In the disclosure, computer readable storage medium, which can be, any include or stores
The tangible medium of program, the program can be commanded execution system, device or device use or in connection.And
In the disclosure, computer-readable signal media may include in a base band or the data as the propagation of carrier wave a part are believed
Number, wherein carrying computer-readable program code.The data-signal of this propagation can take various forms, including but not
It is limited to electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be computer
Any computer readable storage medium other than readable storage medium storing program for executing, which can send, propagate or
Person's transmission is for by the use of instruction execution system, device or device or program in connection.It is computer-readable to deposit
The program code for including on storage media can transmit with any suitable medium, including but not limited to: wireless, electric wire, optical cable, RF
Etc. or above-mentioned any appropriate combination.In accordance with an embodiment of the present disclosure, on computer readable storage medium may include
One or more memories other than the ROM 702 and/or RAM 703 and/or ROM 702 and RAM 703 of text description.
Flow chart and block diagram in attached drawing are illustrated according to the system of the various embodiments of the disclosure, system and computer journey
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
A part of one module, program segment or code of table, a part of above-mentioned module, program segment or code include one or more
Executable instruction for implementing the specified logical function.It should also be noted that in some implementations as replacements, institute in box
The function of mark can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are practical
On can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it wants
It is noted that the combination of each box in block diagram or flow chart and the box in block diagram or flow chart, can use and execute rule
The dedicated hardware based systems of fixed functions or operations is realized, or can use the group of specialized hardware and computer instruction
It closes to realize.
As on the other hand, the disclosure additionally provides a kind of computer readable storage medium, the computer-readable storage medium
Matter can be included in equipment described in above-described embodiment；It is also possible to individualism, and without in the supplying equipment.
Above-mentioned computer readable storage medium carries one or more program, when said one or multiple programs by one this set
When standby execution, so that the equipment executes: obtaining the change data that data manipulation generates；Change data are stored in first database
In, wherein first database supports data aggregate inquiry；And change data are stored in the second database, wherein the second number
Also data aggregate is supported to inquire according to library, for the standby database as first database.
Embodiment of the disclosure is described above.But the purpose that these embodiments are merely to illustrate that, and
It is not intended to limit the scope of the present disclosure.Although respectively describing each embodiment above, but it is not intended that each reality
Use cannot be advantageously combined by applying the measure in example.The scope of the present disclosure is defined by the appended claims and the equivalents thereof.It does not take off
From the scope of the present disclosure, those skilled in the art can make a variety of alternatives and modifications, these alternatives and modifications should all fall in this
Within scope of disclosure.
Claims (14)
1. a kind of data processing method, comprising:
Obtain the change data that data manipulation generates；
The change data are stored in first database, wherein the first database supports data aggregate inquiry；And
The change data are stored in the second database, wherein second database also supports data aggregate to inquire, and is used for
Standby database as the first database.
2. according to the method described in claim 1, wherein, the method also includes:
The change data are stored in the first database by first thread；And
By being different from the first thread and second thread synchronous with the first thread is by change data deposit institute
It states in the second database.
3. method according to claim 1 or 2, wherein the method also includes:
The change data are sent to the first of the first database by Kafka message and receive server, so that described
The data that first reception server sends the Kafka message are stored in the first database；And
The change data are sent to the second of second database simultaneously by the Kafka message and receive server,
So that the data that the second reception server sends the Kafka message are stored in second database.
4. according to the method described in claim 3, wherein, the method also includes:
After obtaining the change data, using the change data as in a task record write-in task list；
During storing the change data, check the first database and second database state value whether
It is all that state is completed；And
In the case where the state value of the first database and/or second database is not that state is completed, pass through institute
It states Kafka message and the change data being written in the task record in the task list is sent to first data
Described the first of library receives the second reception server of server and/or second database, to guarantee state value not
It is that the database of the state that is completed can successfully be stored in the change data.
5. according to the method described in claim 1, wherein, the method also includes:
Historical data in the first database earlier than preset time deposit is deleted.
6. according to the method described in claim 1, wherein, the method also includes:
After obtaining the change data, table is divided to be stored in a distributed experiment & measurement system in the change data point library.
7. a kind of data processing system, comprising:
Module is obtained, for obtaining the change data of data manipulation generation；
First memory module, for the change data to be stored in first database, wherein the first database supports number
According to aggregate query；And
Second memory module, for the change data to be stored in the second database, wherein second database is also supported
Data aggregate inquiry, for the standby database as the first database.
8. system according to claim 7, in which:
First memory module is also used to that the change data are stored in the first database by first thread；With
And
Second memory module is also used to by being different from the first thread and second line synchronous with the first thread
The change data are stored in second database by journey.
9. system according to claim 7 or 8, in which:
First memory module is also used to that the change data are sent to the first database by Kafka message
First receives server, so that data deposit first number that the first reception server sends the Kafka message
According in library；And
Second memory module is also used to that the change data are sent to described second simultaneously by the Kafka message
The second of database receives server, so that the data that the second reception server sends the Kafka message are stored in institute
It states in the second database.
10. system according to claim 9, wherein the system also includes:
Third memory module, for being written the change data as a task record after obtaining the change data
In task list；
Enquiry module, for checking the first database and second data during storing the change data
Whether the state value in library is all that state is completed；
First memory module is also used to not be completed in the state value of the first database and second database
The state value of state or the first database is not that will be written in the case where state is completed by the Kafka message
The change data in the task record in the task list are sent to first reception of the first database
Server is that the first database for the state that is completed can successfully be stored in the change data to guarantee state value not；With
And
Second memory module is also used to not be completed in the state value of the first database and second database
The state value of state or second database is not that will be written in the case where state is completed by the Kafka message
The change data in the task record in the task list are sent to second reception of second database
Server is that second database for the state that is completed can successfully be stored in the change data to guarantee state value not.
11. system according to claim 7, wherein the system also includes:
Removing module, for deleting the historical data in the first database earlier than preset time deposit.
12. system according to claim 7, wherein the system also includes:
4th memory module, for dividing table deposit one distributed in the change data point library after obtaining the change data
In data-base cluster.
13. a kind of computer system, comprising:
One or more processors；
Memory, for storing one or more programs,
Wherein, when one or more of programs are executed by one or more of processors, so that one or more of
Processor realizes data processing method described in any one of claims 1 to 6.
14. a kind of computer readable storage medium, is stored thereon with executable instruction, which makes to handle when being executed by processor
Device realizes data processing method described in any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711477730.0A CN110019310A (en) | 2017-12-29 | 2017-12-29 | Data processing method and system, computer system, computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711477730.0A CN110019310A (en) | 2017-12-29 | 2017-12-29 | Data processing method and system, computer system, computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110019310A true CN110019310A (en) | 2019-07-16 |
Family
ID=67187175
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711477730.0A Pending CN110019310A (en) | 2017-12-29 | 2017-12-29 | Data processing method and system, computer system, computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110019310A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111680063A (en) * | 2020-05-25 | 2020-09-18 | 泰康保险集团股份有限公司 | Method and device for querying data in Elasticissearch paging mode |
| CN112950345A (en) * | 2021-03-05 | 2021-06-11 | 北京健康之家科技有限公司 | Business and financial data processing method and device and computer equipment |
| CN113268488A (en) * | 2020-02-14 | 2021-08-17 | 北京京东振世信息技术有限公司 | Data persistence method and device |
| CN113961580A (en) * | 2021-12-22 | 2022-01-21 | 联通智网科技股份有限公司 | Data query method, service system and electronic equipment |
| CN115952200A (en) * | 2023-01-17 | 2023-04-11 | 安芯网盾(北京)科技有限公司 | Multi-source heterogeneous data aggregation query method and device based on MPP (maximum power point tracking) architecture |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070124348A1 (en) * | 2005-11-30 | 2007-05-31 | Oracle International Corporation | Database system configured for automatic failover with no data loss |
| CN102299904A (en) * | 2010-06-23 | 2011-12-28 | 阿里巴巴集团控股有限公司 | System and method for realizing service data backup |
| CN103744906A (en) * | 2013-12-26 | 2014-04-23 | 乐视网信息技术（北京）股份有限公司 | System, method and device for data synchronization |
| CN104834558A (en) * | 2015-05-19 | 2015-08-12 | 北京京东尚科信息技术有限公司 | Method and system for processing data |
| CN106294672A (en) * | 2016-08-08 | 2017-01-04 | 杭州玳数科技有限公司 | The method and system that a kind of daily record represents in real time and inquires about |
| CN106326469A (en) * | 2016-08-31 | 2017-01-11 | 无锡雅座在线科技发展有限公司 | Synchronization method and device of data |
-
2017
- 2017-12-29 CN CN201711477730.0A patent/CN110019310A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070124348A1 (en) * | 2005-11-30 | 2007-05-31 | Oracle International Corporation | Database system configured for automatic failover with no data loss |
| CN102299904A (en) * | 2010-06-23 | 2011-12-28 | 阿里巴巴集团控股有限公司 | System and method for realizing service data backup |
| CN103744906A (en) * | 2013-12-26 | 2014-04-23 | 乐视网信息技术（北京）股份有限公司 | System, method and device for data synchronization |
| CN104834558A (en) * | 2015-05-19 | 2015-08-12 | 北京京东尚科信息技术有限公司 | Method and system for processing data |
| CN106294672A (en) * | 2016-08-08 | 2017-01-04 | 杭州玳数科技有限公司 | The method and system that a kind of daily record represents in real time and inquires about |
| CN106326469A (en) * | 2016-08-31 | 2017-01-11 | 无锡雅座在线科技发展有限公司 | Synchronization method and device of data |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113268488A (en) * | 2020-02-14 | 2021-08-17 | 北京京东振世信息技术有限公司 | Data persistence method and device |
| CN113268488B (en) * | 2020-02-14 | 2023-11-03 | 北京京东振世信息技术有限公司 | Method and device for data persistence |
| CN111680063A (en) * | 2020-05-25 | 2020-09-18 | 泰康保险集团股份有限公司 | Method and device for querying data in Elasticissearch paging mode |
| CN111680063B (en) * | 2020-05-25 | 2023-08-18 | 泰康保险集团股份有限公司 | Method and device for paging query data by elastic search |
| CN112950345A (en) * | 2021-03-05 | 2021-06-11 | 北京健康之家科技有限公司 | Business and financial data processing method and device and computer equipment |
| CN113961580A (en) * | 2021-12-22 | 2022-01-21 | 联通智网科技股份有限公司 | Data query method, service system and electronic equipment |
| CN115952200A (en) * | 2023-01-17 | 2023-04-11 | 安芯网盾(北京)科技有限公司 | Multi-source heterogeneous data aggregation query method and device based on MPP (maximum power point tracking) architecture |
| CN115952200B (en) * | 2023-01-17 | 2023-06-27 | 安芯网盾(北京)科技有限公司 | MPP architecture-based multi-source heterogeneous data aggregation query method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10162874B2 (en) | Related table notifications | |
| AU2018204273B2 (en) | Auto discovery of configuration items | |
| CN110019310A (en) | Data processing method and system, computer system, computer readable storage medium | |
| CN109413127A (en) | A kind of method of data synchronization and device | |
| CN109308214A (en) | Data task processing method and system | |
| CN110209677A (en) | The method and apparatus of more new data | |
| CN109388654A (en) | A kind of method and apparatus for inquiring tables of data | |
| US11386264B2 (en) | Configuring complex tables in a client experience framework | |
| US20130085895A1 (en) | High throughput global order promising system | |
| CN109961331A (en) | Page processing method and its system, computer system and readable storage medium storing program for executing | |
| CN109241033A (en) | The method and apparatus for creating real-time data warehouse | |
| CN110221829A (en) | Information processing method and its system, computer system and computer-readable medium | |
| CN111258988B (en) | Asset management method, device, electronic equipment and medium | |
| CN109960212A (en) | Task sending method and device | |
| CN110399397A (en) | A kind of data query method and system | |
| CN110020271A (en) | Method and system for cache management | |
| US10942866B1 (en) | Priority-based cache | |
| CN110879818B (en) | Method, device, medium and electronic equipment for acquiring data | |
| CN110019525A (en) | A kind of method and apparatus of data-base capacity-enlarging | |
| US11586604B2 (en) | In-memory data structure for data access | |
| CN104715349A (en) | Method and system for calculating e-commerce freight | |
| CN111695749A (en) | Method and device for generating grouping tasks | |
| CN113127416A (en) | Data query method and device | |
| US10769027B1 (en) | Queued scheduler | |
| CN113821519B (en) | Data processing method and domain drive design architecture |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190716 |