CN116401270A

CN116401270A - Data query method, device, computer equipment and storage medium

Info

Publication number: CN116401270A
Application number: CN202310285471.0A
Authority: CN
Inventors: 孙敏茜; 孙韶; 杨乐; 苏志康
Original assignee: Industrial and Commercial Bank of China Ltd ICBC
Current assignee: Industrial and Commercial Bank of China Ltd ICBC
Priority date: 2023-03-22
Filing date: 2023-03-22
Publication date: 2023-07-07

Abstract

The application relates to a data query method, a data query device, computer equipment and a storage medium, and relates to the technical field of computers. Can be used in the field of financial science and technology or other related fields. The method comprises the following steps: responding to a data query request initiated aiming at a target order receiving service record, and determining a data identifier corresponding to the data query request information; determining a target database storing target order receiving service records according to the data identification; acquiring a first time difference between service processing time corresponding to a target acquiring service record and current system time, and determining a data table query strategy aiming at a target database according to the first time difference; and inquiring the target order receiving service record in the data table set according to the data table inquiring sequence. By adopting the method, the data table query strategy can be determined based on the time difference, and the target data can be conveniently queried in the corresponding data table to be queried according to the data table query sequence in the data table query strategy, so that the data query efficiency is improved.

Description

Data query method, device, computer equipment and storage medium

Technical Field

The present application relates to the field of computer technology, and in particular, to a data query method, apparatus, computer device, storage medium, and computer program product.

Background

The bank card order collection serves as a fund settlement service provided by the contracted bank to the merchant, plays an important role between the merchant and the contracted bank, and the merchant can complete data management and transaction processing of the order collection business through corresponding operation in a corresponding bank order collection system.

After a user initiates a consumption transaction, the serial number of the merchant can be stored in a bank order receiving system as a unique retrieval number, but as the transaction amount of the bank order receiving system is continuously increased, the amount of the order receiving transaction data is gradually increased, and when the merchant inquires the transaction data, the inquiry time of the transaction data is correspondingly increased, so that the efficiency of data inquiry is not improved.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a data query method, apparatus, computer device, computer-readable storage medium, and computer program product that enable data query efficiency.

In a first aspect, the present application provides a data query method, the method including:

responding to a data query request initiated aiming at a target order receiving service record, and determining a data identifier corresponding to the data query request information; the target order receiving service record is used for representing service processing data in a service order receiving system;

Determining a target database storing the target order receiving service records according to the data identification;

acquiring a first time difference between service processing time corresponding to the target order receiving service record and current system time, and determining a data table query strategy aiming at the target database according to the first time difference; the data table query strategy is used for representing a data table set to be queried and a data table query sequence aiming at the data table set;

and inquiring the target order receiving service record in the data table set according to the data table inquiring sequence.

In one embodiment, the determining a data table query policy for the target database according to the first time difference includes:

under the condition that the first time difference is equal to a preset first time difference threshold value, adding a log table corresponding to the current system time into the data table set to be queried to obtain a first data table set to be queried; the first data table set to be queried comprises a first log table and a second log table; the data storage time of the order-receiving service record stored in the second log table is longer than that of the order-receiving service record stored in the first log table;

Determining a first data table query sequence corresponding to the first data table set to be queried, and determining the data table query strategy according to the first data table set to be queried and the first data table query sequence; the data table query order characterizes that the query priority of the first log table is greater than the query priority of the second log table.

In one embodiment, the determining, according to the first time difference, a data table query policy for the target database further includes:

under the condition that the first time difference is larger than a first time difference threshold and smaller than or equal to a preset second time difference threshold, adding a history table and a log table corresponding to the current system time into the data table set to be queried to obtain a second data table set to be queried; the second data table set to be queried comprises a history table, a first log table and a second log table;

determining a second data table query sequence corresponding to the second data table set to be queried, and determining the data table query strategy according to the second data table set to be queried and the second data table query sequence; the data table query order characterizes that the query priority of the history table is greater than the query priority of the first log table, and the query priority of the first log table is greater than the query priority of the second log table.

adding a history table, a log table and a data lake table corresponding to the current system time into the data table set to be queried under the condition that the first time difference is larger than a preset second time difference threshold value to obtain a third data table set to be queried; the third data table set to be queried comprises a history table, a first log table, a second log table and a data lake table;

determining a third data table query sequence corresponding to the third data table set to be queried, and determining the data table query strategy according to the third data table set to be queried and the third data table query sequence; the data table query sequence characterizes that the query priority of the history table is greater than the query priority of the first log table, the query priority of the first log table is greater than the query priority of the second log table, and the query priority of the first log table is greater than the query priority of the data lake table.

In one embodiment, the method further comprises:

responding to a data storage request initiated for a to-be-stored order-receiving service record, and acquiring a routing field corresponding to the to-be-stored order-receiving service record; the to-be-stored order-receiving service record is used for representing service processing data in a service order-receiving system;

Determining a data storage address corresponding to the to-be-stored order-receiving service record according to the routing field;

and determining a storage data table corresponding to the to-be-stored order receiving service record according to the data storage address, and storing the to-be-stored order receiving service record in the storage data table.

In one embodiment, the method further comprises:

acquiring service processing time corresponding to the to-be-stored order-receiving service record;

determining a log table to be stored corresponding to the to-be-stored order-receiving service record according to the service processing time and a preset parameter control table;

and storing the to-be-stored order-receiving service record in the to-be-stored log table.

In one embodiment, the method further comprises:

acquiring a second time difference between service processing time corresponding to the to-be-transferred order-receiving service record in the log table to be stored and current system time;

and determining a transfer data table corresponding to the to-be-transferred order receiving service record according to the second time difference, and transferring the to-be-transferred order receiving service record to the transfer data table.

In one embodiment, the method further comprises:

taking the to-be-transferred order receiving service record with the second time difference larger than a preset third time difference threshold value as a first to-be-transferred order receiving service record, and determining a to-be-transferred history table corresponding to the first to-be-transferred order receiving service record according to a preset data slicing rule;

And transferring the first to-be-transferred order receiving service record in a first time interval preset in the log table to be stored to the to-be-transferred history table.

In one embodiment, the method further comprises:

acquiring a third time difference between service processing time corresponding to the to-be-transferred order-receiving service record in the to-be-transferred history table and current system time;

and taking the to-be-transferred order receiving service record with the third time difference larger than a preset fourth time difference threshold as an order receiving service record to be placed in the lake, and transferring the to-be-placed order receiving service record to a data lake table.

In a second aspect, the present application further provides a data query device, the device including:

the data identification determining module is used for responding to a data query request initiated for a target order receiving service record and determining a data identification corresponding to the data query request information; the target order receiving service record is used for representing service processing data in a service order receiving system;

the database determining module is used for determining a target database storing the target order receiving service records according to the data identification;

the query strategy determining module is used for obtaining a first time difference between the service processing time corresponding to the target order receiving service record and the current system time, and determining a data table query strategy for the target database according to the first time difference; the data table query strategy is used for representing a data table set to be queried and a data table query sequence;

And the record inquiring module is used for inquiring the target order receiving service record in the data table set according to the data table inquiring sequence.

In a third aspect, the present application also provides a computer device. The computer device comprises a memory and a processor, the memory storing a computer program which, when executed by the processor, implements the steps of the method described above.

In a fourth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the method described above.

In a fifth aspect, the present application also provides a computer program product. The computer program product comprising a computer program which, when executed by a processor, implements the steps of the method described above.

The data query method, the device, the computer equipment, the storage medium and the computer program product, the data identification corresponding to the data query request information is determined by responding to the data query request initiated for the target receipt service record, thereby determining the data identification of the data query request information when the target receipt service record in the target receipt service system is queried, determining the target database storing the target receipt service record according to the data identification, thereby determining the database in which the target receipt service record is located, narrowing the data retrieval range, acquiring the first time difference between the service processing time corresponding to the target receipt service record and the current system time, determining the data table query strategy for the target database according to the first time difference, thereby utilizing the first time difference between the service processing time and the current system time, determining the data table set formed by the data tables possibly existing for the target receipt service record, determining the data table query sequence corresponding to the data table set, and querying the target receipt service record in the data table set according to the data table query sequence, determining the data table corresponding to the data query request information, determining the data table identification corresponding to the data query request information, narrowing the data table corresponding to the data record in the data table query request, determining the data table query strategy corresponding to the data table query strategy according to the data query request record, and the data table query sequence corresponding to the target receipt service record, thereby determining the data table set corresponding to the data table query policy can be queried, and conveniently inquiring the target data in the corresponding data table according to the corresponding inquiry sequence, thereby improving the data inquiry efficiency.

Drawings

FIG. 1 is an application environment diagram of a data query method in one embodiment;

FIG. 2 is a flow chart of a data query method according to an embodiment;

FIG. 3 is a business framework diagram of a acquirer access platform according to one embodiment;

FIG. 4 is a flow diagram of order taking platform transaction processing, in one embodiment;

FIG. 5 is a schematic diagram of storing merchant acquirement water transaction data in one embodiment;

FIG. 6 is a schematic diagram of a backup merchant acquiring single stream transaction data in one embodiment;

FIG. 7 is a schematic diagram of querying data of a consumer transaction using merchant orders in one embodiment;

FIG. 8 is a block diagram of a data querying device in one embodiment;

fig. 9 is an internal structural diagram of a computer device in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.

The data query method provided by the embodiment of the application can be applied to an application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated on the server 104 or may be located on a cloud or other network server. The server 104 responds to a data query request initiated for the target order receiving service record, and determines a data identifier corresponding to the data query request information; the target order receiving service record is used for representing service processing data in a service order receiving system; the server 104 determines a target database storing target order receiving service records according to the data identification; the server 104 obtains a first time difference between service processing time corresponding to the target order service record and current system time, and the server 104 determines a data table query strategy for the target database according to the first time difference; the data table query strategy is used for representing a data table set to be queried and a data table query sequence aiming at the data table set; the server 104 queries the target acquiring business record in the data table set according to the data table query sequence. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things devices, and portable wearable devices, where the internet of things devices may be smart speakers, smart televisions, etc. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The server 104 may be implemented as a stand-alone server or as a server cluster of multiple servers.

It will be appreciated that the method may also be applied to a server, and may also be applied to a system comprising a terminal and a server, and implemented by interaction of the terminal and the server.

In some embodiments, as shown in fig. 2, a data query method is provided, where this embodiment is applied to a server for illustration, and it is understood that the method may also be applied to a terminal, and may also be applied to a system including a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the steps of:

step S202, a data query request initiated for a target order receiving service record is responded, and a data identifier corresponding to the data query request information is determined.

The target order service record may be a record for characterizing service processing data in a service order system, and in practical application, the target order service record may include a merchant order stream in a bank order system.

As an example, after receiving a data query request initiated for a target acquiring service record, the server responds to the data query request initiated for the target acquiring service record, and determines a data identifier corresponding to the data query request information according to the data query request information, for example: and the server responds to a data query request initiated for the target order receiving service record, and queries an internal order number in a banking system corresponding to the commercial tenant transaction running water in a preset query relation comparison table according to the commercial tenant transaction running water (unique index).

Step S204, determining a target database storing target order receiving business records according to the data identification.

The data identifier may be an identifier for distinguishing the order-receiving service record, and in practical application, the data identifier may include a merchant transaction stream and an internal order number.

As an example, the server searches the comparison relation according to the merchant transaction flow (unique index), finds the internal order number stored in the banking system, and performs a hash algorithm according to the internal order number to determine the target database storing the target order service record.

Step S206, a first time difference between the service processing time corresponding to the target order-receiving service record and the current system time is obtained, and a data table query strategy for the target database is determined according to the first time difference.

The data table query strategy is used for representing a data table set to be queried and a data table query sequence aiming at the data table set.

The service processing time may refer to a transaction time corresponding to the order-receiving service record.

As an example, a server obtains a service processing time corresponding to a target order-receiving service record, the server compares the service processing time with a current system time, the server calculates a first time difference between the service processing time and the current system time according to the service processing time and the current system time, the first time difference may be represented as D1, and the server determines a data table query policy for the target database according to a relationship between the first time difference and a preset time difference threshold, where the data table query policy is used to characterize a data table set to be queried and a data table query sequence for the data table set.

Step S208, according to the data table inquiring sequence, inquiring the target order receiving service record in the data table set.

As an example, after the server obtains the data table query policy for the target database, the server performs sub-table query in the data table set to be queried according to the data table query sequence in the data table query policy, and obtains the target order receiving service record.

In the data query method, the data identifier corresponding to the data query request information is determined by responding to the data query request initiated for the target receipt service record, so that when the target receipt service record in the target receipt service record is queried, the data identifier of the data query request information is determined, the target database storing the target receipt service record is determined according to the data identifier, so that the database storing the target receipt service record is determined, the data retrieval range is narrowed, the first time difference between the service processing time corresponding to the target receipt service record and the current system time is acquired, the data table query strategy for the target database is determined according to the first time difference, the data table set formed by the data tables possibly existing in the target receipt service record is determined by utilizing the first time difference between the service processing time and the current system time, the data table query sequence for the data table set is determined, and the target receipt service record is queried in the data table query sequence, the target receipt service record is determined by the data table query sequence, the data database corresponding to the data query request information is determined by responding to the data query request, the data query range of the target receipt service record is determined, the data table query strategy for the data table corresponding to the target receipt service record is determined, the data table query strategy can be queried according to the data table query policy is reduced, the data table query policy can be queried according to the data table query sequence corresponding to the data table query sequence, and conveniently inquiring the target data in the corresponding data table according to the corresponding inquiry sequence, thereby improving the data inquiry efficiency.

In some embodiments, determining a data table query policy for the target database based on the first time difference comprises: under the condition that the first time difference is equal to a preset first time difference threshold value, adding the log table corresponding to the current system time into the data table set to be queried to obtain a first data table set to be queried, determining a first data table query sequence corresponding to the first data table set to be queried, and determining a data table query strategy according to the first data table set to be queried and the first data table query sequence.

The first data table set to be queried may include a first log table and a second log table, where in practical application, the data storage time of the order-receiving service record stored in the second log table is longer than the data storage time of the order-receiving service record stored in the first log table.

The data table query sequence may represent that the query priority of the first log table is greater than the query priority of the second log table, and in practical application, the first log table may include a log a table, and the second log table may include a log B table.

Wherein the first time difference threshold may comprise 0 days.

As an example, the server compares a first time difference between the service processing time and the current system time with a preset first time difference threshold, and when the first time difference is equal to the preset first time difference threshold, the server adds a first log table and a second log table corresponding to the current system time into a data table set to be queried to obtain a first data table set to be queried, the server determines a first data table query sequence for the first data table set to be queried, and the server determines a data table query strategy according to the first data table set to be queried and the first data table query sequence, for example: when the server detects that the first time difference is equal to a preset first time difference threshold (for example, 0 days), the server adds the first log table and the second log table into a data table set to be queried to obtain a first data table set to be queried, the server determines a first data table query sequence aiming at the first log table and the second log table in the first data table set to be queried, wherein the first data table query sequence can specifically represent that the query priority of the first log table is greater than that of the second log table, namely, the server firstly performs data query on the first log table, and if a target order-receiving service record is not queried in the first log table, the server performs data query on the second log table until a target order-receiving service record is queried.

In this embodiment, by adding the log table corresponding to the current system time to the data table set to be queried to obtain the first data table set to be queried under the condition that the first time difference is equal to the preset first time difference threshold value, determining the first data table query sequence corresponding to the first data table set to be queried, and determining the data table query policy according to the first data table set to be queried and the first data table query sequence, the corresponding data table set to be queried and the corresponding data table query policy can be determined under the condition that the first time difference is equal to the preset first time difference threshold value based on the relation between the first time difference and the preset time difference threshold value, so as to accurately search the target acquisition service record, thereby improving the data query efficiency.

In some embodiments, determining a data table query policy for the target database based on the first time difference further comprises: and under the condition that the first time difference is larger than a preset first time difference threshold and smaller than or equal to a preset second time difference threshold, adding a history table and a log table corresponding to the current system time into the data table set to be queried to obtain a second data table set to be queried, determining a second data table query sequence corresponding to the second data table set to be queried, and determining a data table query strategy according to the second data table set to be queried and the second data table query sequence.

The second set of data tables to be queried may include a history table, a first log table, and a second log table.

The data table query order may characterize that the query priority of the history table is greater than the query priority of the first log table, and the query priority of the first log table is greater than the query priority of the second log table.

Wherein the second time difference threshold may comprise 14 days.

The history table may include history table 1, history table 2, history table 3, history table 4, history table 5, history table 6, and history table 7, among others.

As an example, the server compares a first time difference between the service processing time and the current system time with a preset first time difference threshold and a preset second time difference threshold, and when the first time difference is greater than the preset first time difference threshold and less than or equal to the preset second time difference threshold, the server adds a history table, a first log table and a second log table corresponding to the current system time to the data table set to be queried to obtain a second data table set to be queried, the server determines a second data table query sequence for the second data table set to be queried, and the server determines a data table query strategy according to the second data table set to be queried and the second data table query sequence, for example: when the server detects that the first time difference is greater than a preset first time difference threshold (for example, 0 day) and is less than or equal to a preset second time difference threshold (for example, 14 days), the server adds the history table, the first log table and the second log table into a data table set to be queried to obtain a second data table set to be queried, the server determines a second data table query sequence aiming at the history table, the first log table and the second log table in the second data table set, the first data table query sequence can specifically represent that the query priority of the history table is greater than the query priority of the first log table, and the query priority of the first log table is greater than the query priority of the second log table, namely, the server firstly queries the history table, if the server does not query the target receipt service record in the history table, the server queries the first log table, and if the target receipt service record is not queried in the first log table, the server queries the second log table until the target receipt service record is queried.

In this embodiment, by adding the history table and the log table corresponding to the current system time to the data table set to be queried to obtain the second data table set to be queried under the condition that the first time difference is greater than the preset first time difference threshold and less than or equal to the preset second time difference threshold, determining the second data table query sequence corresponding to the second data table set to be queried, and determining the data table query strategy according to the second data table set to be queried and the second data table query sequence, the corresponding data table set to be queried and the corresponding data table query strategy can be determined under the condition that the first time difference is greater than the preset first time difference threshold and less than or equal to the preset second time difference threshold, so as to accurately search the target order receiving service record, thereby improving the data query efficiency.

In some embodiments, determining a data table query policy for the target database based on the first time difference further comprises: under the condition that the first time difference is larger than a preset second time difference threshold value, adding a history table, a log table and a data lake table corresponding to the current system time into a data table set to be queried to obtain a third data table set to be queried; and determining a third data table query sequence corresponding to the third data table set to be queried, and determining a data table query strategy according to the third data table set to be queried and the third data table query sequence.

The data table query sequence may characterize that the query priority of the history table is greater than the query priority of the first log table, and the query priority of the first log table is greater than the query priority of the second log table, and the query priority of the first log table is greater than the query priority of the data lake table.

The third set of data tables to be queried may include a history table, a first log table, a second log table, and a data lake table.

As an example, the server compares a first time difference between the service processing time and the current system time with a preset second time difference threshold, and when the first time difference is greater than the preset second time difference threshold, the server adds a history table, a first log table, a second log table and a data lake table corresponding to the current system time to a data table set to be queried to obtain a third data table set to be queried, the server determines a third data table query sequence for the third data table set to be queried, and the server determines a data table query strategy according to the third data table set to be queried and the third data table query sequence, for example: when the server detects that the first time difference is larger than a preset second time difference threshold (for example, 14 days), the server adds the history table, the first log table, the second log table and the data lake table into a data table set to be queried to obtain a third data table set to be queried, the server determines a third data table query sequence aiming at the history table, the first log table, the second log table and the data lake table in the third data table set to be queried, the third data table query sequence specifically indicates that the query priority of the history table is larger than the query priority of the first log table, the query priority of the first log table is larger than the query priority of the data lake table, namely, the server queries the history table first, if the server does not query a target receipt service record in the history table, the server queries the first log table, if the second log table queries the target receipt service record in the first log table, the server queries the second log table is not queried in the target receipt service record, the target receipt service record is not queried in the second log table, the target receipt service record is not queried in the target receipt table, and the target receipt service record is not queried in the target receipt table, the service record is queried in the target receipt table, the target receipt table is queried in the service table is not queried in the target table, the service table is queried in the target table is queried when the target table is queried in the target table is not is queried in the service table is queried.

In this embodiment, when the first time difference is greater than a preset second time difference threshold, adding a history table, a log table and a data lake table corresponding to the current system time into a data table set to be queried to obtain a third data table set to be queried; determining a third data table query sequence corresponding to the third data table set to be queried, determining a data table query strategy according to the third data table set to be queried and the third data table query sequence, and determining the corresponding data table set to be queried and the corresponding data table query strategy under the condition that the first time difference is larger than a preset second time difference threshold based on the relation between the first time difference and the preset time difference threshold so as to accurately search the target acquisition business record and improve the data query efficiency.

In some embodiments, the above method further comprises: and responding to a data storage request initiated for the to-be-stored order receiving service record, acquiring a routing field corresponding to the to-be-stored order receiving service record, determining a data storage address corresponding to the to-be-stored order receiving service record according to the routing field, determining a storage data table corresponding to the to-be-stored order receiving service record according to the data storage address, and storing the to-be-stored order receiving service record in the storage data table.

The to-be-stored order receiving service record can represent service processing data in a service order receiving system.

The data storage address may include a system position of a data table storing the to-be-stored order-receiving service record in the related service system, and in practical application, the data storage address may include a database server address.

The stored data tables may include, among other things, log tables, history tables, and data lake tables.

As an example, after the server obtains a data storage request initiated for the to-be-stored order service record, the server responds to the data storage request initiated for the to-be-stored order service record, the server obtains an order number corresponding to the to-be-stored order service record, the server performs a hash algorithm according to the order number to calculate a data storage address of the to-be-stored order service record, the server determines a storage data table corresponding to the to-be-stored order service record according to the data storage address, and the server stores the to-be-stored order service record in the storage data table.

In an exemplary embodiment, after a server obtains a data storage request initiated for a to-be-stored order receiving service record, the server responds to the data storage request initiated for the to-be-stored order receiving service record, the server obtains a value of a fragment routing field corresponding to the to-be-stored order receiving service record, the server performs a consistent hash algorithm according to the value of the fragment routing field to obtain fragments and a data storage address to which the to-be-stored order receiving service record belongs, the server determines a storage database and a storage data table corresponding to the to-be-stored order receiving service record according to the data storage address, and the server stores the to-be-stored order receiving service record in the storage data table.

In this embodiment, a routing field corresponding to a to-be-stored order-receiving service record is obtained in response to a data storage request initiated for the to-be-stored order-receiving service record, a data storage address corresponding to the to-be-stored order-receiving service record is determined according to the routing field, a storage data table corresponding to the to-be-stored order-receiving service record is determined according to the data storage address, and the to-be-stored order-receiving service record is stored in the storage data table.

In some embodiments, the above method further comprises: acquiring service processing time corresponding to a to-be-stored order-receiving service record; determining a log table to be stored corresponding to the to-be-stored order-receiving service record according to the service processing time and a preset parameter control table; and storing the to-be-stored order-receiving service record in a to-be-stored log table.

The parameter control table may include a system parameter control table, and in practical application, the parameter control table may be used to store a comparison relationship between the service processing time and the log table, for example, the service processing time on the day corresponds to the log a table, and the service processing time on the alternate day corresponds to the log B table.

As an example, in the same type of table of the same database partition in the bank order receiving system, the server obtains the service processing time corresponding to the order receiving service record to be stored, and because the parameter control table can store the comparison relationship between the service processing time corresponding to the order receiving service record to be stored and the log table, the server can determine the log table to be stored corresponding to the order receiving service record to be stored according to the service processing time and the preset parameter control table, and then the server stores the order receiving service record to be stored in the log table to be stored.

In this embodiment, the service processing time corresponding to the to-be-stored order-receiving service record is obtained; determining a log table to be stored corresponding to the to-be-stored order-receiving service record according to the service processing time and a preset parameter control table; the to-be-stored order-receiving service records are stored in the to-be-stored log table, the to-be-stored log table corresponding to the to-be-stored order-receiving service records can be determined based on the service processing time and the parameter control table corresponding to the to-be-stored order-receiving service records, and the to-be-stored order-receiving service records are stored in the to-be-stored log table, so that the order-receiving service records are classified and inquired, and the data inquiring efficiency is improved.

In some embodiments, the above method further comprises: acquiring a second time difference between service processing time corresponding to the to-be-transferred order-receiving service record in the log table to be stored and the current system time; and determining a transfer data table corresponding to the to-be-transferred order receiving service record according to the second time difference, and transferring the to-be-transferred order receiving service record to the transfer data table.

The transfer data table may refer to a data table where the transferred order service record is located.

As an example, in order to reduce the storage pressure of the log table and the history table, the data needs to be subjected to sub-table processing, the server performs batch processing on the order-receiving service records at the end of each day, the server obtains a second time difference between the service processing time corresponding to the order-receiving service record to be transferred in the log table to be stored and the current system time, the server determines a transfer data table corresponding to the order-receiving service record to be transferred according to the second time difference, and the server transfers the order-receiving service record to be transferred to the transfer data table.

In this embodiment, a second time difference between a service processing time corresponding to a to-be-transferred order-receiving service record in a log table to be stored and a current system time is obtained; according to the second time difference, a transfer data table corresponding to the to-be-transferred order receiving service record is determined, the to-be-transferred order receiving service record is transferred to the transfer data table, the transfer data table corresponding to the to-be-transferred order receiving service record can be determined based on the second time difference between the service processing time corresponding to the to-be-transferred order receiving service record in the to-be-stored log table and the current system time, and the to-be-transferred order receiving service record is transferred to the transfer data table, so that the classified storage of the order receiving service record is realized, the classified inquiry of the order receiving service record is conveniently carried out, and the data inquiry efficiency is improved.

In some embodiments, the above method further comprises: taking the to-be-transferred order receiving service record with the second time difference being larger than a preset third time difference threshold value as a first to-be-transferred order receiving service record, and determining a to-be-transferred history table corresponding to the first to-be-transferred order receiving service record according to a preset data slicing rule; and transferring the first to-be-transferred order-receiving business record in a first time interval preset in the log table to be stored into the to-be-transferred history table.

Wherein the third time difference threshold may comprise 2 days.

Wherein the first time interval may comprise one week.

As an example, in order to reduce the storage pressure of the log table, data migration needs to be performed on the log table, the log table data migration is to perform whole table migration on the log a table and the log B table, the server uses the to-be-transferred order-receiving service record with the second time difference being greater than a preset third time difference threshold (for example, 2 days) as the first to-be-transferred order-receiving service record, and the server determines, according to a preset data slicing rule, a to-be-transferred history table corresponding to the first to-be-transferred order-receiving service record, for example, the server may transfer the first to-be-transferred order-receiving service record to a corresponding 0-9 history table according to an internal order number of the bank system; the server transfers a first to-be-transferred order-receiving service record in a preset first time interval (such as a week) in a log table to be stored into a to-be-transferred history table, namely, the server takes the to-be-transferred order-receiving service record with a second time difference larger than a preset third time difference threshold (such as 2 days) as a first to-be-transferred order-receiving service record, the server conducts data migration on the first to-be-transferred order-receiving service record, the server exports data in a log A table and a log B table in a week unit and generates files, the server imports the generated files into the history table to store, and then the server conducts data cleaning on the log A table and the log B table, wherein the data stored in the log A table and the log B table are guaranteed to be unique.

In this embodiment, the to-be-transferred history table corresponding to the first to-be-transferred acquiring service record is determined according to a preset data slicing rule by taking the to-be-transferred acquiring service record with the second time difference being greater than a preset third time difference threshold as the first to-be-transferred acquiring service record; and transferring the first to-be-transferred order-receiving service record in a first time interval preset in the log table to be stored into the to-be-transferred history table, and performing data transfer on the order-receiving service record based on the second time difference and a preset time difference threshold value to reduce the data storage pressure of the log table.

In some embodiments, the above method further comprises: acquiring a third time difference between service processing time corresponding to the to-be-transferred order-receiving service record in the to-be-transferred history table and the current system time; and taking the to-be-transferred order receiving service record with the third time difference larger than the preset fourth time difference threshold as the to-be-in-lake order receiving service record, and transferring the to-be-in-lake order receiving service record into the data lake table.

Wherein the fourth time difference threshold may comprise 14 days.

As an example, in order to reduce the storage pressure of the history table, the history table data migration is to perform whole table migration on the history table, the server uses the to-be-transferred order-receiving service record with the third time difference being greater than a preset fourth time difference threshold (for example, 14 days) as the to-be-in-lake order-receiving service record, the server determines a data lake table corresponding to the to-be-in-lake order-receiving service record according to a preset data slicing rule, the server transfers the to-be-in-lake order-receiving service record in the to-be-stored log table to the data lake table, that is, the server uses the to-be-transferred order-receiving service record with the third time difference being greater than the preset fourth time difference threshold (for example, 14 days) as the to-be-in-lake order-receiving service record, the server performs data migration on the to-be-in-lake order-receiving service record, the server exports the data in the history table and generates a file, the server imports the generated file into the data lake table for storage, and then the server performs data cleaning on the history table.

In this embodiment, a third time difference between the service processing time corresponding to the to-be-transferred order-receiving service record in the to-be-transferred history table and the current system time is obtained; and taking the to-be-transferred order receiving service record with the third time difference larger than the preset fourth time difference threshold value as the to-be-in-lake order receiving service record, and transferring the to-be-in-lake order receiving service record into the data lake table, so that the data migration can be performed on the order receiving service record based on the third time difference and the preset time difference threshold value, and the data storage pressure of the history table is reduced.

For the convenience of understanding of those skilled in the art, fig. 3 exemplarily proposes a service framework diagram of a receipt access platform, where, as shown in fig. 3, the receipt access platform is opened to a non-banking user, and provides data processing service for member institutions to develop a bank card receipt service, and the member institutions complete management and transaction processing of the receipt service through the platform. The access party 301 refers to that a user can perform service processing (including online service, batch service, merchant settlement, error processing, etc.) on a POS terminal, MIS system, phone payment terminal, or third party system through other media such as magnetic stripe card, fingerprint payment, etc.; the order receiving platform 302 is composed of an interface accessed by a member mechanism and an order receiving core processing system.

In an exemplary embodiment, as shown in fig. 4, a flowchart of transaction processing of a acquirer platform is provided, 401 is an interface accessed by a member institution, and mainly includes: (1) merchant management and service system: the management function of the merchant and the terminal is realized; (2) risk management system: the risk grade assessment and risk early warning functions of merchants are realized; (3) a clearing system: the functions of merchant settlement processing, report generation, file transmission and the like are realized; (4) an error handling system: error processing among mechanisms is realized; 402 is a acquiring core processing system, which mainly comprises four parts: (1) The front end access subsystem is mainly responsible for verifying the validity of the transaction, analyzing the message and transferring the communication to the transaction processing subsystem; (2) The transaction processing subsystem is mainly responsible for calculating a transaction route to which service processing layer according to the value of the data fragment to perform transaction logic processing; (3) The data storage module is mainly responsible for storing important data such as merchant running water, order details and the like of receipts after transaction logic processing; (4) The background management subsystem is mainly responsible for settlement processing of merchants, error processing among institutions, data statistics analysis report presentation and the like.

In order to facilitate understanding of those skilled in the art, fig. 5 schematically proposes a schematic diagram for storing data of a merchant order-receiving running water transaction, and as shown in fig. 5, since the transaction amount of a bank order-receiving system is large, the number of access merchants is large, and in order to ensure the stability and dispersion pressure of the system, the data is firstly "classified" according to a certain rule and stored in a plurality of database servers. If the transaction ratio of non-banking customer transactions (such as aggregation business, digital RMB transactions and the like) is higher than that of banking customer transactions (such as bank card transactions and the like) according to the customer group classification, the order receiving business is not suitable for carrying out segmentation by adopting the customer numbers based on the transaction characteristics. If the merchant number is used as the data slicing rule, the transaction flow stored by the database server may be unbalanced due to uneven and dynamic change of the transaction amount of the actual merchant, and the problems of incapability of supporting promotion, second killing or high concurrency of large merchants and the like exist. Based on the above disadvantage, the bank order system preferably adopts order numbers as the slicing rules.

The transaction routing layer is used as a service provider of the order receiving service distributed service access layer and consists of 1-N groups of virtual servers, and in a bank order receiving system, a hash algorithm is carried out according to order numbers to calculate which database server the transaction data is routed to for storage. The service processing layer comprises an online server and a batch server, wherein the online server is responsible for processing online service logic and consists of 1-N groups of containers, virtual servers and database servers; the batch server performs batch business logic processing of date or end of date, such as data migration, data processing and processing, business report generation and the like. In a bank order receiving system, a large amount of data storage can be subjected to scattered processing through transverse sub-warehouse and longitudinal sub-table.

As indicated by 501, the transaction routing layer performs a consistent hash algorithm according to the value of the fragment routing field to obtain fragments to which the transaction data belong, and then routes the transaction to a corresponding database server to store the merchant order-receiving transaction stream;

as indicated at 502, the log surface AB table is selected for storage in the same class of tables in the same database partition based on the current transaction system date. In the bank order receiving system, the system parameter control table is used for storing the relation between the system date and the log surface, for example, the system date of the current day corresponds to the log surface A and the system date of the alternate day corresponds to the log surface B;

as indicated by 503, the batch server performs batch processing business logic at the end of each day, performs data migration for more than 2 days of transaction data storage, exports and generates a file from log surface AB table data in units of weeks, imports the file into a history table for storage, and finally cleans the log surface AB table data;

as indicated at 504, the batch server performs batch processing business logic at the end of each day, performs data migration for transaction data for more than 14 days, exports the history table data into a data lake to generate a file, imports the file into the data lake to store, and finally cleans the history table data.

In an exemplary embodiment, as shown in fig. 6, a schematic diagram of a backup merchant collecting single-stream transaction data is provided, each of the partitioned databases adopts a form of "one master and three slave", three semi-synchronous database servers are hung on one master database server, and four database virtual servers are distributed on two physical servers in an average.

And as indicated by 601, a database semi-synchronous replication mechanism is adopted, after an event is written into a binlog by a Master main library, the event is sent to a Slave standby library to trigger waiting until at least one Slave node of three standby libraries receives binlog content transmitted by the main library and writes the binlog content into a Slave node of the Slave node, the Master main library is informed of the completion of waiting thread operation on the Master main library, and the consistency of data is enhanced. If the Slave delay does not return the information of successful copying due to network delay and other reasons, the half-synchronous copying is degraded into an asynchronous copying mode when the timeout duration set by the Master is exceeded, and a part of efficiency is ensured.

To prevent the database storage from exceeding capacity, the batch server performs batch processing business logic at the end of each day, performs data migration for transaction data for more than 14 days, transfers the data to a data lake for storage, and distributes pressure, as indicated at 602.

In this embodiment, by adopting a data storage backup mode of three semi-synchronous databases of a main database, the consistency of system data can be enhanced by sacrificing small part of efficiency, thereby providing powerful guarantee for data security and improving data security.

In order to facilitate understanding of those skilled in the art, fig. 7 schematically proposes a diagram for querying transaction data using merchant order-receiving flowing water, as shown in fig. 7, in a bank order-receiving system, transactions are accessed from a transaction routing layer, a comparison relation is searched according to merchant transaction flowing water (unique index), an internal order number stored in the bank system is found, then a hash algorithm is performed according to the order number to obtain a database fragment to which the transaction belongs, a horizontal database is used for searching a database, and then a longitudinal data table is used for searching according to a relation between a transaction date and a current system date and time difference, wherein the main steps include: (1) Calculating the current transaction date and current system date time difference D; (2) If d=0 days, judging the log surface to which the current transaction date belongs according to the system parameter control table, and inquiring a corresponding log surface A table or log surface B table; (3) If D < = 14 days, according to the data storage rule, preferentially inquiring the history table, wherein the history table does not look up the log surface table again; (4) If D is more than 14 days, according to the data storage rule, preferentially checking a history table, wherein the history table does not check a log surface table, and the log surface table does not check a data lake table again; and for transactions with high-frequency access, if data is acquired from a data lake, the history table is written back after the details are acquired, so that the next acquisition of the same data is facilitated, and the inquiry time is shortened.

In this embodiment, a write-back mechanism is adopted for the out-of-date data (the transaction time is greater than 14 days) of the high-frequency access in the high-concurrency transaction scene, so that the number of interactions between the system and the data lake can be reduced, the query time can be shortened, the data query efficiency can be improved, and meanwhile, for a large number of data storage, the storage pressure of a single partitioned database can be dispersed and the system stability can be improved by storing in a mode of partitioning the horizontal database and partitioning the longitudinal database.

It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

Based on the same inventive concept, the embodiment of the application also provides a data query device for realizing the above related data query method. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation in the embodiments of one or more data query devices provided below may refer to the limitation of the data query method hereinabove, and will not be repeated herein.

In one embodiment, as shown in fig. 8, there is provided a data query apparatus, including: a data identification determination module 802, a database determination module 804, a query policy determination module 806, and a record query module 808, wherein:

a data identifier determining module 802, configured to determine a data identifier corresponding to the data query request information in response to a data query request initiated for a target acquiring service record; the target order receiving service record is used for representing service processing data in a service order receiving system;

a database determining module 804, configured to determine, according to the data identifier, a target database storing the target acquiring service record;

a query policy determining module 806, configured to obtain a first time difference between a service processing time corresponding to the target acquiring service record and a current system time, and determine a data table query policy for the target database according to the first time difference; the data table query strategy is used for representing a data table set to be queried and a data table query sequence;

And a record querying module 808, configured to query the target acquiring service record in the data table set according to the data table query sequence.

In an exemplary embodiment, the above-mentioned query policy determining module 806 is specifically further configured to add, in the case where the first time difference is equal to a preset first time difference threshold, a log table corresponding to the current system time to the set of data tables to be queried, to obtain a first set of data tables to be queried; the first data table set to be queried comprises a first log table and a second log table; the data storage time of the order-receiving service record stored in the second log table is longer than that of the order-receiving service record stored in the first log table; determining a first data table query sequence corresponding to the first data table set to be queried, and determining the data table query strategy according to the first data table set to be queried and the first data table query sequence; the data table query order characterizes that the query priority of the first log table is greater than the query priority of the second log table.

In an exemplary embodiment, the above-mentioned query policy determining module 806 is specifically further configured to add, when the first time difference is greater than a preset first time difference threshold and less than or equal to a preset second time difference threshold, a history table and a log table corresponding to the current system time to the set of data tables to be queried, to obtain a second set of data tables to be queried; the second data table set to be queried comprises a history table, a first log table and a second log table; determining a second data table query sequence corresponding to the second data table set to be queried, and determining the data table query strategy according to the second data table set to be queried and the second data table query sequence; the data table query order characterizes that the query priority of the history table is greater than the query priority of the first log table, and the query priority of the first log table is greater than the query priority of the second log table.

In an exemplary embodiment, the above-mentioned query policy determining module 806 is specifically further configured to add, when the first time difference is greater than a preset second time difference threshold, a history table, a log table, and a data lake table corresponding to the current system time to the data table set to be queried, to obtain a third data table set to be queried; the third data table set to be queried comprises a history table, a first log table, a second log table and a data lake table; determining a third data table query sequence corresponding to the third data table set to be queried, and determining the data table query strategy according to the third data table set to be queried and the third data table query sequence; the data table query sequence characterizes that the query priority of the history table is greater than the query priority of the first log table, the query priority of the first log table is greater than the query priority of the second log table, and the query priority of the first log table is greater than the query priority of the data lake table.

In an exemplary embodiment, the apparatus further includes a data storage module, where the data storage module is specifically configured to obtain, in response to a data storage request initiated for a to-be-stored order-receiving service record, a routing field corresponding to the to-be-stored order-receiving service record; the to-be-stored order-receiving service record is used for representing service processing data in a service order-receiving system; determining a data storage address corresponding to the to-be-stored order-receiving service record according to the routing field; and determining a storage data table corresponding to the to-be-stored order receiving service record according to the data storage address, and storing the to-be-stored order receiving service record in the storage data table.

In an exemplary embodiment, the apparatus further includes a log table determining module, where the log table determining module is specifically configured to obtain a service processing time corresponding to the to-be-stored order-receiving service record; determining a log table to be stored corresponding to the to-be-stored order-receiving service record according to the service processing time and a preset parameter control table; and storing the to-be-stored order-receiving service record in the to-be-stored log table.

In an exemplary embodiment, the apparatus further includes a first transferring module, where the first transferring module is specifically configured to obtain a second time difference between a service processing time corresponding to the to-be-transferred acquiring service record in the to-be-stored log table and a current system time; and determining a transfer data table corresponding to the to-be-transferred order receiving service record according to the second time difference, and transferring the to-be-transferred order receiving service record to the transfer data table.

In an exemplary embodiment, the apparatus further includes a second transfer module, where the second transfer module is specifically configured to use a to-be-transferred order-receiving service record with the second time difference greater than a preset third time difference threshold as a first to-be-transferred order-receiving service record, and determine, according to a preset data slicing rule, a to-be-transferred history table corresponding to the first to-be-transferred order-receiving service record; and transferring the first to-be-transferred order receiving service record in a first time interval preset in the log table to be stored to the to-be-transferred history table.

In an exemplary embodiment, the apparatus further includes a third transfer module, where the third transfer module is specifically configured to obtain a third time difference between a service processing time corresponding to the to-be-transferred order-receiving service record in the to-be-transferred history table and a current system time; and taking the to-be-transferred order receiving service record with the third time difference larger than a preset fourth time difference threshold as an order receiving service record to be placed in the lake, and transferring the to-be-placed order receiving service record to a data lake table.

The various modules in the data querying device described above may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In one embodiment, a computer device is provided, which may be a terminal, and the internal structure thereof may be as shown in fig. 9. The computer device includes a processor, a memory, an input/output interface, a communication interface, a display unit, and an input means. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a data query method. The display unit of the computer device is used for forming a visual picture, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be a key, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.

It will be appreciated by those skilled in the art that the structure shown in fig. 9 is merely a block diagram of a portion of the structure associated with the present application and is not limiting of the computer device to which the present application applies, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.

In an embodiment, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, carries out the steps of the method embodiments described above.

In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.

It should be noted that, the user information (including, but not limited to, user equipment information, user personal information, etc.) and the data (including, but not limited to, data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data are required to comply with the related laws and regulations and standards of the related countries and regions.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The above examples only represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.

Claims

1. A method of querying data, the method comprising:

2. The method of claim 1, wherein determining a data table query policy for the target database based on the first time difference comprises:

3. The method of claim 1, wherein determining a data table query policy for the target database based on the first time difference further comprises:

under the condition that the first time difference is larger than a preset first time difference threshold value and smaller than or equal to a preset second time difference threshold value, adding a history table and a log table corresponding to the current system time into the data table set to be queried to obtain a second data table set to be queried; the second data table set to be queried comprises a history table, a first log table and a second log table;

4. The method of claim 1, wherein determining a data table query policy for the target database based on the first time difference further comprises:

5. The method according to claim 1, wherein the method further comprises:

6. The method of claim 5, wherein the method further comprises:

7. The method of claim 6, wherein the method further comprises:

8. The method of claim 7, wherein the method further comprises:

9. The method of claim 8, wherein the method further comprises:

10. A data querying device, the device comprising:

11. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 9 when the computer program is executed.

12. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 9.

13. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any one of claims 1 to 9.