CN113377502A - Transaction processing method, device, server, database management system and medium - Google Patents
Transaction processing method, device, server, database management system and medium Download PDFInfo
- Publication number
- CN113377502A CN113377502A CN202110648078.4A CN202110648078A CN113377502A CN 113377502 A CN113377502 A CN 113377502A CN 202110648078 A CN202110648078 A CN 202110648078A CN 113377502 A CN113377502 A CN 113377502A
- Authority
- CN
- China
- Prior art keywords
- transaction
- server node
- target
- data
- serial number
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 26
- 238000012545 processing Methods 0.000 claims abstract description 88
- 238000000034 method Methods 0.000 claims abstract description 31
- 238000011010 flushing procedure Methods 0.000 claims description 39
- 238000004590 computer program Methods 0.000 claims description 6
- 230000004048 modification Effects 0.000 abstract description 27
- 238000012986 modification Methods 0.000 abstract description 27
- 238000007726 management method Methods 0.000 description 17
- 230000007246 mechanism Effects 0.000 description 14
- 230000008569 process Effects 0.000 description 12
- 230000001680 brushing effect Effects 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000002159 abnormal effect Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 101100153085 Dictyostelium discoideum trxA gene Proteins 0.000 description 4
- 101150102071 TRX1 gene Proteins 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000002618 waking effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/466—Transaction processing
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a transaction processing method, a transaction processing device, a server, a database management system and a medium. The method comprises the following steps: receiving an SQL request, wherein the SQL request is used for requesting to perform specified processing on data of a target transaction; if the target server node is not the current server node, extracting a submitted log serial number and a disk-brushing log serial number of the target server node, wherein the target server node is a server node where a transaction for finally modifying target transaction data is located before the SQL request is received; and performing specified processing on the target transaction data according to the submitted log serial number, the disk-refreshing log serial number and the SQL request. According to the technical scheme, aiming at the modification of other server nodes to the target transaction, whether the log corresponding to the submitted log serial number finishes the disk refreshing or not is determined before the transaction is processed according to the submitted log serial number and the disk refreshing log serial number, so that the correctness of data is ensured, and the reliability of transaction processing is improved.
Description
Technical Field
The embodiment of the invention relates to the technical field of databases, in particular to a transaction processing method, a transaction processing device, a server, a database management system and a medium.
Background
Database Management systems (DBMS) generally use Redo (Redo) logs to ensure the persistence (Durability) of transactions, when a user initiates a transaction Commit (Commit) request, a Redo log is generated by the operation of modifying the transaction state of a Database server, and after the Redo log is written into an online log file (i.e., disk-flushing) of a disk, the Database server responds to the user, and the transaction Commit is completed. After the transaction is submitted, if the database server is abnormally restarted, the data can be restored to the state before the fault according to the Redo log, and the data modification of the transaction is kept effective.
At present, an optimized transaction processing mechanism can respond to a user in advance without waiting for completion of log disk refreshing so as to reduce the disk refreshing conflict and the disk refreshing waiting time of a Redo log, and the mechanism is called Commit nowait. For the Redo logs which are not refreshed, if abnormal restart occurs, the database server ignores the modification corresponding to the Redo logs, rolls back the data to the state before modification, and changes the loss of the data modification into better system performance.
However, in the case that the database management system includes a plurality of database servers, the plurality of database servers are allowed to access or modify data of the same transaction, and in this case, if an abnormal restart occurs, the data may be recovered according to the Redo log which completes the disk refreshing, and may roll back because the Redo log does not refresh the disk, thereby causing data errors and affecting the correct processing of the transaction.
Disclosure of Invention
The invention provides a transaction processing method, a transaction processing device, a server, a database management system, a medium and a system, which are used for ensuring the correctness of transaction data and improving the reliability of transaction processing.
In a first aspect, an embodiment of the present invention provides a transaction processing method, including:
receiving an SQL request, wherein the SQL request is used for requesting to perform specified processing on data of a target transaction;
if the target server node is not the current server node, extracting a submission log serial number and a disk-refreshing log serial number of the target server node, wherein the target server node is a server node where a transaction for modifying the data of the target transaction before the SQL request is received is located;
and performing specified processing on the data of the target transaction according to the submitted log serial number, the disk-flushing log serial number and the SQL request.
Further, performing specified processing on the data of the target transaction according to the commit log serial number, the flushing log serial number and the SQL request, including:
if the submitted log serial number is less than or equal to the disk-flushing log serial number, performing specified processing on the data of the target transaction according to the SQL request.
Further, performing specified processing on the data of the target transaction according to the commit log serial number, the flushing log serial number and the SQL request, including:
if the submitted log serial number is larger than the disk-brushing log serial number, sending a disk-brushing request to the target server node;
and after the target server node finishes the disk refreshing, performing specified processing on the data of the target transaction according to the SQL request.
Further, after receiving the SQL request, the method further includes:
and if the target server node is the current server node, performing specified processing on the data of the target transaction according to the SQL request.
Further, before receiving the SQL request, the method further includes:
when the target transaction is started, recording a submitted log serial number and a disk-brushing log serial number of each server node in the cluster;
and updating the serial number of the flashing log of each server node in real time after the target transaction is started.
Further, after receiving the SQL request, the method further includes:
reading a transaction number of a transaction that last modified data of the target transaction before receiving the SQL request;
determining the target server node according to the remainder of the transaction number for the total number of the server nodes in the cluster
In a second aspect, an embodiment of the present invention provides a transaction processing apparatus, including:
the receiving module is used for receiving an SQL request, and the SQL request is used for requesting to perform specified processing on data of a target transaction;
the extracting module is used for extracting a submission log serial number and a disk-refreshing log serial number of a target server node if the target server node is not a current server node, wherein the target server node is a server node where a transaction for modifying data of the target transaction before the SQL request is received is located;
and the processing module is used for performing specified processing on the data of the target transaction according to the submitted log serial number, the disk-flushing log serial number and the SQL request.
In a third aspect, an embodiment of the present invention provides a database server, including:
one or more processors;
a memory for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the transaction method of the first aspect.
In a fourth aspect, an embodiment of the present invention provides a system, including:
a plurality of database servers, each database server being a server node;
wherein the at least one server node is configured to:
receiving an SQL request, wherein the SQL request is used for requesting to perform specified processing on data of a target transaction;
if the target server node is not the current server node, extracting a submission log serial number and a disk-refreshing log serial number of the target server node, wherein the target server node is a server node where a transaction for modifying the data of the target transaction before the SQL request is received is located;
and performing specified processing on the data of the target transaction according to the submitted log serial number, the disk-flushing log serial number and the SQL request.
In a fifth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the transaction processing method according to the first aspect.
The embodiment of the invention provides a transaction processing method, a transaction processing device, a server, a database management system and a medium. The method comprises the following steps: receiving an SQL request, wherein the SQL request is used for requesting to perform specified processing on data of a target transaction; if the target server node is not the current server node, extracting a submission log serial number and a disk-refreshing log serial number of the target server node, wherein the target server node is the server node where the transaction which finally modifies the target transaction data before receiving the SQL request is located; and performing specified processing on the target transaction data according to the submitted log serial number, the disk-refreshing log serial number and the SQL request. According to the technical scheme, aiming at the modification of other server nodes to the target transaction, whether the log corresponding to the submitted log serial number finishes the disk refreshing or not is determined before the transaction is processed according to the submitted log serial number and the disk refreshing log serial number, so that the correctness of data is ensured, and the reliability of transaction processing is improved.
Drawings
Fig. 1 is a flowchart of a transaction processing method according to an embodiment of the present invention;
fig. 2 is a flowchart of a transaction processing method according to a second embodiment of the present invention;
fig. 3 is a schematic structural diagram of a transaction processing apparatus according to a third embodiment of the present invention;
fig. 4 is a schematic hardware structure diagram of a database server according to a fourth embodiment of the present invention;
fig. 5 is a schematic diagram illustrating a result of a database management system according to a fifth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. In addition, the embodiments and features of the embodiments in the present invention may be combined with each other without conflict. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Before discussing exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart may describe the steps as a sequential process, many of the steps can be performed in parallel, concurrently or simultaneously. In addition, the order of the steps may be rearranged. The process may be terminated when its processing is completed, but may have additional steps not included in the figure. The processes may correspond to methods, functions, procedures, subroutines, and the like.
For a better understanding of embodiments of the present invention, the related art will be described below.
The durability of the transaction can be guaranteed by utilizing the Redo log. Specifically, when receiving a transaction submission request initiated by a user, the database server modifies the transaction into a Committed state, the operation of modifying the transaction state generates a Redo log, starts a log disk-flushing operation, writes the Redo log into an online log file of a disk, and then responds to the user to complete the transaction submission. After the transaction is submitted, if the database server is abnormally restarted, the data can be recovered to the state before the fault according to the Redo log, the transaction state is guaranteed to be Committed, and the data modification of the transaction is maintained to be effective. This mechanism means that the submission of each transaction triggers a Redo log disk-flushing operation, and frequent Redo log disk-flushing requests and disk-flushing waits greatly affect the overall performance of the database management system.
The realization principle of the Commit nowait mechanism is as follows: the transaction is submitted with permission to specify the nowait option, namely, the user is responded in advance before the Redo log generated by the transaction is written into the disk, and the aim is to reduce the disk-flushing conflict and the disk-flushing waiting of the Redo log. However, since these Redo logs have not been flushed, if an abnormal restart occurs, the database server ignores the modifications corresponding to these Redo logs, rolls back the data to the state before modification, and trades for better system performance by allowing these data modifications to be lost.
In the case where the database management system includes multiple database servers, the multiple database servers are allowed to access or modify data of the same transaction, and in this case, the Commit nowait mechanism may cause data errors, which may affect the correct processing of the transaction. For example:
Create table t1(c1 int)；
Create table t2(c1 int)；
Insert into t1 values(1)；
Insert into t2 values(1)；
Commit；
after the above statement is executed, there are two tables in the database, t1 and t2, and there is one data in each of t1 and t2, whose value is 1.
Then, the transaction (denoted as trx0, with 0 being the transaction number or ID number of the transaction) of node number 0 (denoted as EP [0]) performs the following processing:
update t1 set c1 ═ 11; - -the data of the modification table t1 is 11, and the Redo log flush of the modify operation is complete
Commit; modifying Redo log non-flushed disk with transaction status Committed
In addition, the transaction (denoted as trx1, with transaction number or ID number for transaction) for node number 1 (denoted as EP [1]) performs the following processing:
update t1 set c1 ═ 22; -modifying the data of table t1 to 22
update t2 set c1 ═ 22; -modifying the data of table t2 to 22
Commit; the two operations for modifying data and the Redo log with the modified transaction status of Committed complete disk flushing
In this case, if the system is abnormally restarted, the database server will Redo the Redo log that has been completed by the disk-flushing, and then roll back the data of the transaction that has not been submitted. Specifically, EP [1] modifies the Redo log of the transaction state of trx1 to complete the disk-flushing, and trx1 commits to complete, so that the data modification is effective, and the data at t1 and t2 are both restored to 22 at the restart; however, because EP [0] modified the Redo log of the transaction state of trx0 without flushing, trx0 is still active, the data of trx0 transaction rolls back after restart, and the data of t1 table is modified to 1 again. It can be seen that a data error is caused by the loss of the modification to the t1 table and the retention of the data modified to the t2 table by the trx1 transaction of EP [1 ].
The method provided by this embodiment aims to provide a transaction processing method, which can determine whether data is correct and whether the data of a target transaction can be directly processed according to an SQL request by analyzing whether recorded Redo logs (i.e., logs corresponding to submitted log serial numbers) of modified data that has been submitted last time complete disk refreshing, on the premise of ensuring high performance of a system (i.e., Commit nowait mechanism), so as to ensure correctness of data and improve reliability of transaction processing.
Example one
Fig. 1 is a flowchart of a transaction processing method according to an embodiment of the present invention, which is applicable to a database management system for performing processing such as accessing, modifying, adding, deleting, and the like on data of a target transaction. In particular, the transaction processing method may be executed by a transaction processing apparatus, which may be implemented in software and/or hardware and integrated in a database server. It should be noted that the database management system may include a plurality of database servers, each database server is a server node, and the system employs a Commit nowait mechanism, that is, each database server may respond to a user first and then flush a Redo log corresponding to a modified transaction state. The method of the present embodiment may be performed when one database server (i.e., the current database server or the current server node) receives an SQL request initiated by a user.
As shown in fig. 1, the method specifically includes the following steps:
s110, receiving an SQL request, wherein the SQL request is used for requesting to perform specified processing on the data of the target transaction.
Specifically, the SQL request refers to a request initiated by a user and used by a current server node to process data of a target transaction, where the processing may refer to access, add, delete, modify, query, and the like of the data of the target transaction.
And S120, if the target server node is not the current server node, extracting the submission log serial number and the disk-brushing log serial number of the target server node.
Specifically, the target server node is a server node where a transaction that modifies data of the target transaction is located before receiving the SQL request. The target server node may be the current server node or may be another server node in the cluster. If the target server node is the current server node, namely the server node which modifies the data last before receiving the SQL request and the current server node which is about to process the data of the target transaction are the same server node, under the condition, the modification of the data of the target transaction by other server nodes in the cluster is not required to be considered, if the system is restarted due to a fault, the current server node ignores the modification corresponding to the Redo log which is not flushed, and recovers the data according to the Redo log which is flushed, and data errors can not be caused.
If the target server node is not the current server node, that is, before the current server node receives the SQL request, other server nodes modify the data of the target transaction, if the fault restart occurs, after the current server node recovers the data according to the disk-brushing Redo log, because the disk-brushing may not be performed on the Redo log corresponding to the other server nodes, the corresponding data rollback is caused, the data recovered according to the disk-brushing completed Redo log is modified, and the data error is caused. In this embodiment, whether the Redo log of the last modified data of the target server node completes the disk refreshing is determined by analyzing the submitted log serial number and the disk refreshing log serial number of the target server node before receiving the SQL request, and on this basis, the Redo log which is not refreshed can be written into the disk in time to avoid data rollback, and then the data of the target transaction is processed.
The Log Sequence Number (LSN) is an incremental value maintained within the database management system to identify the Redo Log. In this embodiment, each database server has two log serial numbers, which are a Commit log serial number (denoted as Commit _ LSN) and a Commit log serial number (denoted as File _ LSN), respectively, where Commit _ LSN is an LSN of a Redo log generated when a target server node modifies a transaction state to Commit when a target transaction is Committed, which is recorded when the target transaction is started; file _ LSN is the maximum LSN of the Redo log written to the disk, and is recorded when the target transaction is started, and is updated after the target transaction is started, and File _ LSN is incremented by a corresponding value every time a batch of Redo logs complete disk refreshing. Before receiving the SQL request, the target server node modifies the data of the target transaction last, and when the current server node receives the SQL request, it may determine whether the Redo log generated by the target server node when the target transaction is started is finished flushing according to the Commit _ LSN of the target server node and the current latest File _ LSN recorded by the target transaction.
Optionally, Commit _ LSN and File _ LSN are interacted between server nodes in the cluster in real time, so that when a certain server node receives an SQL request, the Commit _ LSN and File _ LSN corresponding to a target server node can be acquired.
S130, performing appointed processing on the target transaction data according to the submitted log serial number, the disk-refreshing log serial number and the SQL request.
Specifically, the current server node determines whether a Redo log generated by the target server node when the target transaction is started has finished the disk refreshing according to the Commit _ LSN and the latest File _ LSN of the target server node recorded by the target transaction, and performs the specified processing on the target transaction data according to the SQL request.
For example, Commit _ LSN is 10, File _ LSN is 12, that is, the log sequence number of the Redo log corresponding to the latest Commit operation when the target transaction is started is 10, the log sequence number of the Redo log that has completed the disk flushing reaches 12, which indicates that the Redo log having the log sequence number of 10 has completed the disk flushing, at this time, the data of the target transaction is correct, and will not be lost even if an abnormal restart occurs, and at this time, the target transaction data may be subjected to the specified processing according to the SQL request. For another example, Commit _ LSN is 10, File _ LSN is 8, that is, the log sequence number of the Redo log that has completed the disk flushing is only 8, which indicates that the Redo log with the log sequence number of 10 has not been flushed, once an abnormal restart occurs, the data modification corresponding to the Redo log with the log sequence number of 10 is lost, the data is rolled back to the state before modification, and a data error is caused, in this case, it is necessary to wait for the target server node to complete the disk flushing of the Redo log and then perform the specified processing on the target transaction data according to the SQL request.
In the transaction processing method provided by the embodiment of the invention, whether the disk refreshing of the Redo log corresponding to the submitted operation of the target server node is completed or not is determined by introducing the Commit _ LSN and the File _ LSN when the target transaction is started, so that the modification loss of the target transaction data by other server nodes is avoided, the data of the target transaction is reprocessed on the basis of ensuring the data correctness, and the target transaction is efficiently and reliably processed under the Commit nowait mechanism of the shared storage cluster database management system.
Example two
Fig. 2 is a flowchart of a transaction processing method according to a second embodiment of the present invention, which is optimized based on the second embodiment, and specifically describes a sequence number extraction process, a process of determining a target server node according to a transaction number, and a process of determining whether to wait for a disk-flushing process. It should be noted that technical details that are not described in detail in the present embodiment may be referred to any of the above embodiments.
Specifically, as shown in fig. 2, the method specifically includes the following steps:
s210, recording a commit log serial number and a flash log serial number of each server node in the cluster when a target transaction is started.
Specifically, starting the target transaction refers to starting a specific operation on the data of the target transaction, such as adding, deleting, querying, modifying, and the like, where the current server node (which may be any one server node in the cluster) records Commit _ LSN and File _ LSN of each other server node for the target transaction.
It can be understood that each server node in the cluster generates a corresponding Commit _ LSN when the transaction status is modified to Commit and a Redo log is generated, and broadcasts the Commit _ LSN to other server nodes in the shared storage database cluster; and after the disk refreshing of the Redo log is completed each time, File _ LSN is broadcasted to other server nodes. Specifically, each server node in the cluster may perform the following operations under the Commit nowait mechanism:
receiving a transaction Commit request of a user;
modifying the transaction state to Committed;
generating a Redo log for modifying the transaction state, and registering Commit _ LSN;
releasing blocked resources of transactions and waking up waiting transactions
Triggering a broadcast process of Commit _ LSN information, sending Commit _ LSN to other server nodes, and broadcasting to each server node to be executed by asynchronous threads);
the transaction commit is completed in response to the user.
And S220, updating the serial number of the flashing log of each server node in real time after the target transaction is started.
Specifically, after the target transaction is started, the Commit _ LSN values of the other server nodes in the target transaction record are not changed, but the current server node updates the corresponding File _ LSN in the target transaction record according to the received File _ LSN broadcasted by the other server nodes.
And S230, receiving the SQL request.
S240, reading the transaction number of the transaction which modifies the data of the target transaction before the SQL request is received.
In this embodiment, the target server node is determined by using the transaction number. Each transaction in the system corresponds to a transaction number (trxID), and each piece of data contains a trxID field that identifies the transaction number that last modified the piece of data.
It should be noted that, in the process of reading the transaction number, visibility analysis needs to be performed on each transaction in the system, that is, whether each transaction is visible to the current server node is analyzed, and the visibility determines whether the modification of the data of the target transaction by one transaction is effective. Illustratively, one historical transaction is to delete data a in a data page, the target transaction is to query all data in the data page, and if the historical transaction is visible, the query result of the target transaction does not include data a; if the historical transaction is not visible, data A is included in the query result of the target transaction. By judging the visibility of each transaction, the consistency of data can be ensured.
In this embodiment, the transaction that last modified the data of the target transaction before the SQL request was received is a visible transaction. Specifically, if the transaction number of one transaction is the same as the transaction number of the target transaction, or the transaction number of one transaction is smaller than the transaction number of the target transaction and does not belong to an active transaction, the transaction is visible to the current server node, the modification of the data of the target transaction by the transaction is effective, and it is required to ensure that the data of the target transaction can be processed according to the SQL request after the completion of the flushing of the Redo log corresponding to the transaction. If a transaction has a transaction number greater than the transaction number of the target transaction or belongs to an active transaction, the transaction is not visible to the current server node, and the modification of the data of the target transaction by the transaction occurs after the target transaction is started and Commit _ LSN is recorded, and is therefore invalid and can be ignored.
And S250, determining a target server node according to the remainder of the transaction number for the total number of the server nodes in the cluster.
Specifically, the target server node where the transaction that modified the data last before the SQL request was received is determined according to the read trxID field. In this embodiment, the total number of the server nodes in the cluster is left by the read transaction number, and the remainder is recorded as the node number of the server node. For example, if the read transaction number is 4, the total number of nodes is 3, and the node number of the current server node is 1, the target server node is the current server node. On the basis, the transaction numbers allocated by different server nodes can be ensured not to conflict, and the target server node can be determined conveniently and rapidly by establishing the mapping relation between the node number and the transaction number.
S260, is the target server node the current server node? If yes, go to S280; otherwise, S270 is executed.
Specifically, if the target server node is the current server node, modification of data of the target transaction by other server nodes in the cluster does not need to be considered under the condition, if the system is abnormally restarted, modification of a Redo log which is not flushed is ignored under a Commit nowait mechanism, data is recovered according to the Redo log which is flushed, and data errors cannot be caused; if the target server node is not the current server node, that is, before the current server node receives the SQL request, other server nodes modify the data of the target transaction, whether the Redo log corresponding to the Commit _ LSN completes the disk refreshing can be determined by analyzing the Commit _ LSN and the File _ LSN of the target server node recorded by the target transaction, and then whether the disk refreshing needs to be waited.
S270, is the commit log sequence number less than the flushing log sequence number? If yes, go to S280; otherwise, S290 is performed.
Specifically, since the Commit _ LSN is collected before the target transaction processing starts, when the modification record of the transaction on other server nodes is accessed, if the Commit _ LSN is less than or equal to the File _ LSN, it indicates that the Redo log corresponding to the Commit _ LSN has completed the disk flushing, and the modification of the data by the transaction of the target server node has been already stored, the data of the target transaction is subjected to the specified processing according to the SQL request. The asynchronous Redo log disk-flushing waiting mechanism between the server nodes can improve the execution efficiency of the Commit nowait mechanism of the shared storage cluster.
And S280, performing specified processing on the data of the target transaction according to the SQL request.
And S290, sending a disk refreshing request to the target server node, and after the target server node completes disk refreshing, performing specified processing on the data of the target transaction according to the SQL request.
Specifically, if Commit _ LSN > File _ LSN, it indicates that the Redo log corresponding to Commit _ LSN is not disked, data will roll back when a fault restarts, and the transaction of the target server node is lost to modify the data, in this case, a disking request needs to be sent to the target server node, and after completing disking of the Redo log corresponding to Commit _ LSN, the data of the target transaction is subjected to specified processing according to the SQL request, and after completing execution, the user can be responded, and the target transaction is completed.
In the transaction processing method provided by the second embodiment of the present invention, optimization is performed on the basis of the above embodiments, and each server node provides a basis for determining whether to wait for refreshing a Redo log by broadcasting and recording Commit _ LSN and File _ LSN; by judging the visibility of each transaction, the data of the target transaction can be effectively modified by other server nodes, the data can be correctly recovered, and the consistency of the data is ensured; by establishing a mapping relation between the node number and the transaction number and taking the read transaction number as the balance of the total number of the nodes, the target server node can be determined efficiently, and the transaction numbers distributed by different server nodes are ensured not to conflict; by means of a Redo log disk-refreshing waiting mechanism between asynchronous server nodes, the execution efficiency of a Commit nowait mechanism of a shared storage cluster can be improved; finally, according to Commit _ LSN and File _ LSN, the log corresponding to Commit _ LSN is guaranteed to complete disk refreshing, and therefore reliability of transaction processing is improved
EXAMPLE III
Fig. 3 is a schematic structural diagram of a transaction processing apparatus according to a third embodiment of the present invention. The transaction processing apparatus provided in this embodiment includes:
a receiving module 310, configured to receive an SQL request, where the SQL request is used to request to perform specified processing on data of a target transaction;
an extracting module 320, configured to extract a commit log serial number and a flash log serial number of a target server node if the target server node is not a current server node, where the target server node is a server node where a transaction that last modifies data of the target transaction before receiving the SQL request is located;
and the processing module 330 is configured to perform specified processing on the data of the target transaction according to the commit log serial number, the flushing log serial number, and the SQL request.
The transaction processing device provided by the third embodiment of the present invention can determine the log submitting progress and the disk brushing progress of the target server node according to the submitted log serial number and the disk brushing log serial number for the modification of the target transaction by other server nodes, and accordingly, whether the log corresponding to the submitted log serial number completes disk brushing is determined before the transaction is processed, thereby ensuring the correctness of the transaction data and improving the reliability of the transaction processing.
On the basis of the above embodiment, the processing module 330 includes:
and the first processing unit is used for performing specified processing on the data of the target transaction according to the SQL request if the submitted log serial number is less than or equal to the disk-flushing log serial number.
On the basis of the above embodiment, the processing module 330 includes:
a request unit, configured to send a disk-flushing request to the target server node if the submitted log serial number is greater than the disk-flushing log serial number;
and the second processing unit is used for performing specified processing on the data of the target transaction according to the SQL request after the target server node finishes the disk refreshing.
On the basis of the above embodiment, the apparatus further includes:
and the third processing unit is used for performing specified processing on the data of the target transaction according to the SQL request after the SQL request is received and if the target server node is the current server node.
On the basis of the above embodiment, the apparatus further includes:
the recording module is used for recording a submitted log serial number and a disk-brushing log serial number of each server node in the cluster when the target transaction is started before the SQL request is received;
and the updating module is used for updating the serial number of the disk-flushing log of each server node in real time after the target transaction is started.
On the basis of the above embodiment, the apparatus further includes:
the reading module is used for reading the transaction number of the transaction which modifies the data of the target transaction before the SQL request is received after the SQL request is received;
and the residue taking module is used for determining the target server node according to the residue of the transaction number taking the residue of the total number of the server nodes in the cluster.
The transaction processing device provided by the third embodiment of the present invention can be used to execute the transaction processing method provided by any of the above embodiments, and has corresponding functions and beneficial effects.
Example four
Fig. 4 is a schematic diagram of a hardware structure of a database server according to a fourth embodiment of the present invention. As shown in fig. 4, the database server provided in the present application includes a memory 42, a processor 41, and a computer program stored in the memory and running on the processor, and when the processor 41 executes the computer program, the transaction processing method described above is implemented.
The database server may also include a memory 42; the processor 41 in the database server may be one or more, and fig. 4 illustrates one processor 41 as an example; the memory 42 is used to store one or more programs; the one or more programs are executed by the one or more processors 41, so that the one or more processors 41 implement the transaction processing method as described in the embodiment of the present application.
The database server further comprises: a communication device 43, an input device 44 and an output device 45.
The processor 41, the memory 42, the communication device 43, the input device 44 and the output device 45 in the database server may be connected by a bus or other means, and the bus connection is exemplified in fig. 4.
The input device 44 may be used to receive entered numeric or character information and to generate key signal inputs relating to user settings and function controls of the database server. The output device 45 may include a display device such as a display screen.
The communication means 43 may comprise a receiver and a transmitter. The communication device 43 is configured to perform information transmission and reception communication in accordance with control of the processor 41.
The memory 42, as a computer-readable storage medium, may be configured to store software programs, computer-executable programs, and modules, such as program instructions/modules (e.g., the receiving module 310, the extracting module 320, and the processing module 330 in the transaction device) corresponding to the transaction processing method according to the embodiment of the present application. The memory 42 may include a program storage area and a data storage area, wherein the program storage area may store a processing system, an application program required for at least one function; the storage data area may store data created according to the use of a database server, and the like. Further, the memory 42 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, memory 42 may further include memory located remotely from processor 41, which may be connected to a database server over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
EXAMPLE five
The fifth embodiment of the invention provides a database management system. Fig. 5 is a schematic diagram of a database management system according to a fifth embodiment of the present invention. As shown in fig. 5, the system includes: a plurality of database servers (e.g., database server 511 and database server 512), each database server being a server node; wherein the at least one server node is configured to: receiving an SQL request, wherein the SQL request is used for requesting to perform specified processing on data of a target transaction; if the target server node is not the current server node, extracting a submission log serial number and a disk-refreshing log serial number of the target server node, wherein the target server node is the server node where the transaction for finally modifying the data of the target transaction is located before the SQL request is received; and performing specified processing on the data of the target transaction according to the submitted log serial number, the disk-refreshing log serial number and the SQL request.
In the database management system provided by the fifth embodiment of the present invention, for the modification of the target transaction by other server nodes, the current server node may determine the log submitting progress and the disk brushing progress of the target server node according to the submitted log serial number and the disk brushing log serial number, and accordingly, it is determined whether the disk brushing of the logs of other server nodes is completed before the transaction is processed, so as to ensure the correctness of the transaction data and improve the reliability of the transaction processing.
Further, performing specified processing on the data of the target transaction according to the commit log serial number, the flushing log serial number and the SQL request, including:
if the submitted log serial number is less than or equal to the disk-flushing log serial number, performing specified processing on the data of the target transaction according to the SQL request.
Further, performing specified processing on the data of the target transaction according to the commit log serial number, the flushing log serial number and the SQL request, including:
if the submitted log serial number is larger than the disk-brushing log serial number, sending a disk-brushing request to the target server node;
and after the target server node finishes the disk refreshing, performing specified processing on the data of the target transaction according to the SQL request.
Further, after receiving the SQL request, the at least one server node is further configured to:
and if the target server node is the current server node, performing specified processing on the data of the target transaction according to the SQL request.
For example, database server 511 is the current server node where the target transaction is located, and the target server node where the transaction that last modified the data of the target transaction before database server 511 received the SQL request may be database server 511 or database server 512. If the transaction is the database server 511, the data of the target transaction is correspondingly processed according to the SQL request without waiting for disk refreshing; if the transaction is the database server 512, whether the disk needs to be flushed needs to be determined according to Commit _ LSN and File _ LSN of the database server 512, and then the data of the target transaction is correspondingly processed according to the SQL request.
Specifically, if the Commit _ LSN corresponding to the database server 512 is not greater than File _ LSN, the data of the target transaction is processed correspondingly according to the SQL request without waiting for disk refreshing; if Commit _ LSN > File _ LSN, then the data of the target transaction is processed according to the SQL request after the completion of the disk refreshing of the log of the database server 512.
Further, before receiving the SQL request, the at least one server node is further configured to:
when the target transaction is started, recording a submitted log serial number and a disk-brushing log serial number of each server node in the cluster;
and updating the serial number of the flashing log of each server node in real time after the target transaction is started.
Further, after receiving the SQL request, the at least one server node is further configured to:
reading a transaction number of a transaction that last modified data of the target transaction before receiving the SQL request;
and determining the target server node according to the remainder of the transaction number for the total number of the server nodes in the cluster.
The database management system provided by the fifth embodiment can be used for implementing the transaction processing method provided by any of the above embodiments, and has corresponding functions and beneficial effects.
On the basis of the foregoing embodiments, the present embodiment also provides a computer-readable storage medium, on which a computer program is stored, the program, when executed by a transaction processing apparatus, implementing a transaction processing method in any of the foregoing embodiments of the present invention, the method including: receiving an SQL request, wherein the SQL request is used for requesting to perform specified processing on data of a target transaction; if the target server node is not the current server node, extracting a submission log serial number and a disk-refreshing log serial number of the target server node, wherein the target server node is the server node where the transaction which finally modifies the target transaction data before receiving the SQL request is located; and performing specified processing on the target transaction data according to the submitted log serial number, the disk-refreshing log serial number and the SQL request.
Embodiments of the present invention provide a storage medium including computer-executable instructions, which may take any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer-readable storage medium may be, for example, but is not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM), a flash Memory, an optical fiber, a portable CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. A computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take a variety of forms, including, but not limited to: an electromagnetic signal, an optical signal, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, Radio Frequency (RF), etc., or any suitable combination of the foregoing.
Computer program code for carrying out processes of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the transaction processing method according to the embodiments of the present invention.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A transaction processing method, comprising:
receiving a Structured Query Language (SQL) request, wherein the SQL request is used for requesting to perform specified processing on data of a target transaction;
if the target server node is not the current server node, extracting a submission log serial number and a disk-refreshing log serial number of the target server node, wherein the target server node is a server node where a transaction for modifying the data of the target transaction before the SQL request is received is located;
and performing specified processing on the data of the target transaction according to the submitted log serial number, the disk-flushing log serial number and the SQL request.
2. The method of claim 1, wherein performing specified processing on the data of the target transaction according to the commit log sequence number, the onload log sequence number, and the SQL request comprises:
if the submitted log serial number is less than or equal to the disk-flushing log serial number, performing specified processing on the data of the target transaction according to the SQL request.
3. The method of claim 1, wherein performing specified processing on the data of the target transaction according to the commit log sequence number, the onload log sequence number, and the SQL request comprises:
if the submitted log serial number is larger than the disk-brushing log serial number, sending a disk-brushing request to the target server node;
and after the target server node finishes the disk refreshing, performing specified processing on the data of the target transaction according to the SQL request.
4. The method of claim 1, after receiving the SQL request, further comprising:
and if the target server node is the current server node, performing specified processing on the data of the target transaction according to the SQL request.
5. The method of claim 1, prior to receiving the SQL request, further comprising:
when the target transaction is started, recording a submitted log serial number and a disk-brushing log serial number of each server node in the cluster;
and updating the serial number of the flashing log of each server node in real time after the target transaction is started.
6. The method of claim 1, after receiving the SQL request, further comprising:
reading a transaction number of a transaction that last modified data of the target transaction before receiving the SQL request;
and determining the target server node according to the remainder of the transaction number for the total number of the server nodes in the cluster.
7. A transaction processing apparatus, comprising:
the receiving module is used for receiving an SQL request, and the SQL request is used for requesting to perform specified processing on data of a target transaction;
the extracting module is used for extracting a submission log serial number and a disk-refreshing log serial number of a target server node if the target server node is not a current server node, wherein the target server node is a server node where a transaction for modifying data of the target transaction before the SQL request is received is located;
and the processing module is used for performing specified processing on the data of the target transaction according to the submitted log serial number, the disk-flushing log serial number and the SQL request.
8. A database server, comprising:
one or more processors;
a memory for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the transaction processing method of any of claims 1-6.
9. A database management system is characterized by comprising a plurality of database servers, wherein each database server is a server node;
wherein the at least one server node is configured to:
receiving an SQL request, wherein the SQL request is used for requesting to perform specified processing on data of a target transaction;
if the target server node is not the current server node, extracting a submission log serial number and a disk-refreshing log serial number of the target server node, wherein the target server node is a server node where a transaction for modifying the data of the target transaction before the SQL request is received is located;
and performing specified processing on the data of the target transaction according to the submitted log serial number, the disk-flushing log serial number and the SQL request.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a transaction processing method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110648078.4A CN113377502B (en) | 2021-06-10 | 2021-06-10 | Transaction processing method, device, server, database management system and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110648078.4A CN113377502B (en) | 2021-06-10 | 2021-06-10 | Transaction processing method, device, server, database management system and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113377502A true CN113377502A (en) | 2021-09-10 |
| CN113377502B CN113377502B (en) | 2024-06-14 |
Family
ID=77573608
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110648078.4A Active CN113377502B (en) | 2021-06-10 | 2021-06-10 | Transaction processing method, device, server, database management system and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113377502B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100502501B1 (en) * | 2004-08-05 | 2005-07-20 | 주식회사 알티베이스 | Method for real-time remote logging and recovery of database system |
| US20070143299A1 (en) * | 2005-12-19 | 2007-06-21 | Huras Matthew A | Commitment of transactions in a distributed system |
| CN103678608A (en) * | 2013-12-16 | 2014-03-26 | 华为技术有限公司 | Log management method and device |
| CN107025242A (en) * | 2016-02-02 | 2017-08-08 | 南京中兴新软件有限责任公司 | The clone method and device of data between database |
| US20170300552A1 (en) * | 2016-04-18 | 2017-10-19 | Amazon Technologies, Inc. | Versioned hierarchical data structures in a distributed data store |
| CN109657007A (en) * | 2018-12-17 | 2019-04-19 | 武汉达梦数据库有限公司 | Database in phase based on asynchronous submission executes method and apparatus parallel |
| US20190129894A1 (en) * | 2016-06-30 | 2019-05-02 | Huawei Technologies Co., Ltd. | Database Transaction Processing Method, Client, and Server |
| CN109828720A (en) * | 2019-01-21 | 2019-05-31 | 上海达梦数据库有限公司 | Date storage method, device, server and storage medium |
| CN110196759A (en) * | 2018-06-20 | 2019-09-03 | 腾讯科技（深圳）有限公司 | Distributed transaction processing method and device, storage medium and electronic device |
| CN111240810A (en) * | 2020-01-20 | 2020-06-05 | 上海达梦数据库有限公司 | Transaction management method, device, equipment and storage medium |
| CN111444147A (en) * | 2020-03-30 | 2020-07-24 | 上海达梦数据库有限公司 | Data page creating method and device, terminal equipment and storage medium |
| CN112015713A (en) * | 2019-05-30 | 2020-12-01 | 阿里巴巴集团控股有限公司 | Database task processing method and device, electronic equipment and readable medium |
-
2021
- 2021-06-10 CN CN202110648078.4A patent/CN113377502B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100502501B1 (en) * | 2004-08-05 | 2005-07-20 | 주식회사 알티베이스 | Method for real-time remote logging and recovery of database system |
| US20070143299A1 (en) * | 2005-12-19 | 2007-06-21 | Huras Matthew A | Commitment of transactions in a distributed system |
| CN103678608A (en) * | 2013-12-16 | 2014-03-26 | 华为技术有限公司 | Log management method and device |
| CN107025242A (en) * | 2016-02-02 | 2017-08-08 | 南京中兴新软件有限责任公司 | The clone method and device of data between database |
| US20170300552A1 (en) * | 2016-04-18 | 2017-10-19 | Amazon Technologies, Inc. | Versioned hierarchical data structures in a distributed data store |
| US20190129894A1 (en) * | 2016-06-30 | 2019-05-02 | Huawei Technologies Co., Ltd. | Database Transaction Processing Method, Client, and Server |
| CN110196759A (en) * | 2018-06-20 | 2019-09-03 | 腾讯科技（深圳）有限公司 | Distributed transaction processing method and device, storage medium and electronic device |
| CN109657007A (en) * | 2018-12-17 | 2019-04-19 | 武汉达梦数据库有限公司 | Database in phase based on asynchronous submission executes method and apparatus parallel |
| CN109828720A (en) * | 2019-01-21 | 2019-05-31 | 上海达梦数据库有限公司 | Date storage method, device, server and storage medium |
| CN112015713A (en) * | 2019-05-30 | 2020-12-01 | 阿里巴巴集团控股有限公司 | Database task processing method and device, electronic equipment and readable medium |
| CN111240810A (en) * | 2020-01-20 | 2020-06-05 | 上海达梦数据库有限公司 | Transaction management method, device, equipment and storage medium |
| CN111444147A (en) * | 2020-03-30 | 2020-07-24 | 上海达梦数据库有限公司 | Data page creating method and device, terminal equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 王嘉豪;蔡鹏;钱卫宁;周傲英;: "集群数据库系统的日志复制和故障恢复", 软件学报, no. 03 * |
| 飞鸡壳壳: "mysql日志刷新到磁盘_MySQL数据库刷日志的方法", Retrieved from the Internet <URL:https://www.blog.csdn.net/weixin_30580403/article/113555748> * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113377502B (en) | 2024-06-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10691722B2 (en) | Consistent query execution for big data analytics in a hybrid database | |
| CN109739935B (en) | Data reading method and device, electronic equipment and storage medium | |
| US9348641B2 (en) | System and method for performing a transaction in a massively parallel processing database | |
| US10747745B2 (en) | Transaction execution commitment without updating of data row transaction status | |
| CN108363806B (en) | Multi-version concurrency control method and device for database, server and storage medium | |
| CN102073540B (en) | Distributed affair submitting method and device thereof | |
| CN110196856B (en) | Distributed data reading method and device | |
| US20160335310A1 (en) | Direct-connect functionality in a distributed database grid | |
| US20150293966A1 (en) | System and Method for Massively Parallel Processing Database | |
| CN111444147B (en) | Data page creation method and device, terminal equipment and storage medium | |
| CN117234670B (en) | Distributed transaction processing method, system, computer equipment and storage medium | |
| CN110413687B (en) | Distributed transaction fault processing method and related equipment based on node interaction verification | |
| CN106354732A (en) | Offline data version conflict resolution method for supporting concurrent cooperation | |
| CN111475480A (en) | Log processing method and system | |
| CN108090056B (en) | Data query method, device and system | |
| CN114036195A (en) | Data request processing method, device, server and storage medium | |
| CN113377502B (en) | Transaction processing method, device, server, database management system and medium | |
| CN114661690A (en) | Multi-version concurrency control and log clearing method, node, equipment and medium | |
| CN112800060A (en) | Data processing method and device, computer readable storage medium and electronic equipment | |
| CN111240810B (en) | Transaction management method, device, equipment and storage medium | |
| CN112612647B (en) | Log parallel replay method, device, equipment and storage medium | |
| WO2023111910A1 (en) | Rolling back database transaction | |
| CN114218236A (en) | Database cluster metadata management method | |
| CN113032437B (en) | Caching method and device based on distributed database, medium and electronic equipment | |
| CN112286992B (en) | Query method, distributed system, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |