WO2024193138A1 - Transaction processing method and related device - Google Patents
Transaction processing method and related device Download PDFInfo
- Publication number
- WO2024193138A1 WO2024193138A1 PCT/CN2023/139803 CN2023139803W WO2024193138A1 WO 2024193138 A1 WO2024193138 A1 WO 2024193138A1 CN 2023139803 W CN2023139803 W CN 2023139803W WO 2024193138 A1 WO2024193138 A1 WO 2024193138A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- transaction
- distributed
- branch
- write
- abnormal
- Prior art date
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 38
- 238000000034 method Methods 0.000 claims abstract description 111
- 230000008569 process Effects 0.000 claims abstract description 99
- 230000002159 abnormal effect Effects 0.000 claims abstract description 63
- 238000012545 processing Methods 0.000 claims abstract description 32
- 230000007246 mechanism Effects 0.000 claims abstract description 21
- 230000004044 response Effects 0.000 claims abstract description 7
- 230000015654 memory Effects 0.000 claims description 24
- 230000005856 abnormality Effects 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 11
- 238000005096 rolling process Methods 0.000 claims description 11
- 238000012958 reprocessing Methods 0.000 claims description 4
- 230000006870 function Effects 0.000 description 17
- 238000010586 diagram Methods 0.000 description 15
- 238000004891 communication Methods 0.000 description 13
- 238000007251 Prelog reaction Methods 0.000 description 6
- 206010047289 Ventricular extrasystoles Diseases 0.000 description 5
- 238000013500 data storage Methods 0.000 description 5
- 238000011084 recovery Methods 0.000 description 5
- 238000005129 volume perturbation calorimetry Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 101001121408 Homo sapiens L-amino-acid oxidase Proteins 0.000 description 2
- 102100026388 L-amino-acid oxidase Human genes 0.000 description 2
- 101100012902 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) FIG2 gene Proteins 0.000 description 2
- 101100233916 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) KAR5 gene Proteins 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000002085 persistent effect Effects 0.000 description 2
- 238000000638 solvent extraction Methods 0.000 description 2
- 101000827703 Homo sapiens Polyphosphoinositide phosphatase Proteins 0.000 description 1
- 102100023591 Polyphosphoinositide phosphatase Human genes 0.000 description 1
- 101150058910 RDS1 gene Proteins 0.000 description 1
- 101100219167 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) BUL1 gene Proteins 0.000 description 1
- 101100140267 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) RDS2 gene Proteins 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000004549 pulsed laser deposition Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
Definitions
- the present application relates to the field of computer technology, and in particular to a transaction processing method and related equipment.
- this transaction can be called a distributed transaction.
- the mainstream transaction submission protocol is usually a two-phase submission protocol (2PC). Since distributed transactions involve the scheduling and coordination of multiple transaction participants, distributed transaction processing is more complicated than traditional single-machine transactions.
- the existing distributed failed transaction retry mechanism generally performs failed transaction retry based on the transaction log records in memory.
- failed transaction retry based on the transaction log records in memory may not guarantee that all failed transactions can be processed correctly. For example, if the transaction coordinator fails, pending failed transactions may be left on the transaction participants.
- an embodiment of the present application discloses a transaction processing method, comprising: in response to a received operation instruction, creating a distributed transaction for the operation instruction, the distributed transaction including multiple transaction branches; committing the distributed transaction based on a preset transaction commit mechanism, and generating a pre-write log of the distributed transaction, the commit of each transaction branch in the multiple transaction branches including multiple process nodes, the preset transaction commit mechanism including a checkpoint for detecting the execution status of the multiple process nodes and the transaction status of each transaction branch; if it is determined that the distributed transaction includes an abnormal transaction branch, processing the abnormal transaction branch based on the pre-write log and the checkpoint, the abnormal transaction branch is caused by an abnormality in the device where the transaction coordinator of the distributed transaction is located, or an abnormality in the device where the transaction participant of the distributed transaction is located.
- the submission of a distributed transaction may be left in a pending state due to an exception in the device where the transaction coordinator of the distributed transaction is located, or in the device where the transaction participant of the distributed transaction is located (device crash, power outage, network disconnection, restart, etc.).
- the transaction submission process is recorded by using a pre-write log with persistent storage capability during the transaction submission phase, and checkpoints are set at multiple process nodes in the transaction submission process to check the execution status and transaction status.
- checkpoints are set at multiple process nodes in the transaction submission process to check the execution status and transaction status.
- the transaction can be re-promoted from the specified process node based on log playback, providing reliable transaction recovery capabilities for transaction exceptions, ensuring that transactions can be handled correctly, and avoiding pending transactions.
- creating a distributed transaction for an operation instruction includes: if data processed by the operation instruction involves multiple sub-databases or multiple sub-tables, creating a distributed transaction for the operation instruction.
- the transaction commit mechanism is preset as a two-phase commit protocol, and a write-ahead log of the distributed transaction is generated, including: before each transaction branch is pre-committed, a first write-ahead log is generated, and the first write-ahead log is used to record transaction-related information of the first phase in the two-phase commit protocol; before each transaction branch is committed, a second write-ahead log is generated, and the second write-ahead log is used to record transaction-related information of the second phase in the two-phase commit protocol.
- the transaction-related information of the first phase and the transaction-related information of the second phase both include the transaction status information of the distributed transaction, the transaction status information of the transaction branch, the operation data, and the structured query language (SQL) statement for processing the operation data.
- a write-ahead log with persistent storage capability is used in the first and second phases of the transaction to record the transaction status information of the global transaction, the transaction status information of the transaction branch, the operation data, and the SQL statement used to process the operation data, thereby providing a reliable transaction recovery capability for transaction exceptions, ensuring that abnormal transactions can be correctly processed based on log playback, and avoiding pending transactions.
- the transaction processing method further includes: if the generation of the first write-ahead log or the second write-ahead log fails, controlling each transaction branch to roll back.
- the submission of each transaction branch includes submitting a transaction process or rolling back a transaction process, and the submitting transaction process or rolling back a transaction process both include multiple process nodes, and each of the multiple process nodes is provided with a checkpoint.
- both the transaction submission process and the transaction rollback process can include multiple process nodes.
- By setting multiple checkpoints in the transaction submission process and the transaction rollback process to check the execution status of each process node it is possible to re-submit the transaction or roll back the transaction from the specified process node based on log playback after the device abnormality is recovered, thereby providing reliable transaction recovery capabilities for transaction abnormalities, ensuring that transactions can be processed correctly, and avoiding pending transactions.
- the method further includes: determining a transaction branch that is not in a preset transaction state as an abnormal transaction branch, where the preset transaction state includes a Rollbacked state or a Committed state.
- a transaction branch that is not in the Rollbacked state or the Committed state is determined as an abnormal transaction branch.
- the transaction can be re-committed or rolled back from the specified process node based on log playback until it is in the Committed state or the Rollbacked state, ensuring that the transaction can be processed correctly and avoiding pending transactions.
- an abnormal transaction branch is processed based on a write-ahead log and a checkpoint, including: replaying the write-ahead log corresponding to the abnormal transaction branch, determining a process node for reprocessing the abnormal transaction branch; and processing the abnormal transaction branch with the determined process node as the starting process node.
- the pre-written log corresponding to the abnormal transaction branch can be replayed after the device abnormality is recovered, thereby improving the log playback efficiency.
- the transaction can be re-promoted from the specified process node, providing reliable transaction recovery capabilities for transaction exceptions, ensuring that the transaction can be processed correctly, and avoiding pending transactions.
- an embodiment of the present application provides a distributed middleware, including: a creation module, used to create a distributed transaction for the operation instruction in response to a received operation instruction, the distributed transaction including multiple transaction branches; a submission module, used to submit the distributed transaction based on a preset transaction submission mechanism, and generate a pre-write log of the distributed transaction, the submission of each transaction branch in the multiple transaction branches includes multiple process nodes, and the preset transaction submission mechanism includes a checkpoint for detecting the execution status of the multiple process nodes and the transaction status of each transaction branch; a processing module, used to process the abnormal transaction branch based on the pre-write log and the checkpoint when it is determined that the distributed transaction contains an abnormal transaction branch, the abnormal transaction branch is caused by an abnormality in the device where the transaction coordinator of the distributed transaction is located, or an abnormality in the device where the transaction participant of the distributed transaction is located.
- the submission of distributed transactions may be in a pending state due to an abnormality of the device where the transaction coordinator of the distributed transaction is located, or the device where the transaction participant of the distributed transaction is located (device downtime, power outage, network disconnection, restart, etc.).
- the write-ahead log records the transaction submission process, and checks the execution status and transaction status by setting checkpoints at multiple process nodes in the transaction submission process.
- the transaction can be re-promoted from the specified process node based on log playback, providing reliable transaction recovery capabilities for transaction exceptions, ensuring that transactions can be handled correctly, and avoiding pending transactions.
- an embodiment of the present application provides a computer-readable storage medium, comprising computer program instructions.
- the computer program instructions When executed by a computing device cluster, the computing device cluster executes the transaction processing method as described in the first aspect.
- an embodiment of the present application provides a computing device cluster, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the transaction processing method described in the first aspect.
- an embodiment of the present application provides a computer program product.
- the computer program product is executed by a computing device cluster
- the computing device cluster executes the transaction processing method as described in the first aspect.
- a device wherein the device has the function of implementing the computing device cluster behavior in the method provided in the first aspect.
- the function can be implemented by hardware, or by hardware executing corresponding software.
- the hardware or software includes one or more modules corresponding to the above functions.
- the computer-readable storage medium described in the third aspect, the computing device cluster described in the fourth aspect, the computer program product described in the fifth aspect, and the apparatus described in the sixth aspect all correspond to the method of the first aspect. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.
- FIG1 is a schematic diagram of the architecture of a distributed transaction submission system provided by an embodiment of the present application.
- FIG2 is a schematic diagram of an application environment of a transaction processing method provided by an embodiment of the present application.
- FIG3 is a schematic diagram of the overall execution flow of a distributed transaction provided by an embodiment of the present application.
- FIG4 is a schematic diagram of a process of performing two-phase commit of a distributed transaction provided by an embodiment of the present application
- FIG5 is a schematic diagram of the organizational structure of a write-ahead log in a storage device according to an embodiment of the present application.
- FIG6 is a schematic diagram of a process of ending a distributed transaction provided by an embodiment of the present application.
- FIG7 is a schematic diagram of a process of rolling back a distributed transaction provided by an embodiment of the present application.
- FIG8 is a schematic diagram of a process flow of a transaction processing method according to an embodiment of the present application.
- FIG9 is a schematic diagram of a module of a distributed database middleware provided in an embodiment of the present application.
- FIG10 is a schematic diagram of the structure of a computing device provided by an embodiment of the present application.
- FIG. 11 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application.
- words such as “exemplary” or “for example” are used to indicate examples, illustrations or descriptions. Any embodiment or design described as “exemplary” or “for example” in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as “exemplary” or “for example” is intended to present related concepts in a specific way.
- the data in the original centralized database is stored in multiple data storage nodes connected by the network to obtain larger storage capacity and higher concurrency.
- the database and table sharding technology can be used to store massive data shards in each storage node of the distributed database.
- the database and table sharding technology refers to storing the data shards of a large table in each storage node according to the established partitioning strategy, or dividing a large table into business sub-tables with smaller data volumes, and storing each sub-table in each storage node according to the established partitioning strategy.
- Transaction refers to a group of continuous operations, which are combined into a complete logic. After executing this group of operations, the transaction commit system needs to switch from one consistent state to another consistent state. All operations in the above operation combination are either successfully executed or not successfully executed. If some operations in the operation combination are successfully executed and some operations are not successfully executed, the successful operations need to be rolled back to the state before the unsuccessful execution.
- XA protocol A distributed transaction processing specification proposed by the X/Open organization, which mainly defines the interface between the transaction manager (TM) and the local resource manager (RM).
- the transaction manager also known as the transaction coordinator, is responsible for allocating transaction unique identifiers, monitoring the progress of transaction execution, and committing and rolling back transactions.
- the local resource manager can be a database, file system, etc., which can provide a way to access resources.
- the application Application Program
- Two-phase commit protocol based on XA protocol The transaction commit process is divided into the prepare phase, which is the first phase in the two-phase commit protocol, and the commit phase, which is the second phase in the two-phase commit protocol.
- the transaction coordinator can coordinate all transaction participants. If all transaction participants (RM) can commit the transaction in the first phase, the transaction branch on each RM is committed in the second phase. Otherwise, the transaction coordinator rolls back the transaction branch on each RM.
- SQL Structured query language
- FIG1 a schematic diagram of the architecture of a distributed transaction submission system provided in an embodiment of the present application is exemplarily introduced.
- the distributed transaction submission system may include a transaction coordinator 110 and multiple transaction participants 120.
- the transaction participants 120 are used to process distributed transactions.
- the transaction coordinator 110 is used to coordinate and manage multiple transaction participants 120 to process distributed transactions.
- a distributed transaction may also be called a global transaction.
- a distributed transaction may correspond to multiple transaction branches, and multiple transaction branches correspond to multiple transaction participants 120 one by one.
- the transaction coordinator 110 may be selected from a plurality of transaction participants 120.
- the transaction coordinator 110 and one of the plurality of transaction participants 120 may be the same node.
- the transaction coordinator 110 and the plurality of transaction participants 120 may also be different nodes, which is not limited in the embodiments of the present application.
- the specific implementation of a node may be a server or a virtual machine, which is not limited in the embodiments of the present application.
- a node may refer to a data storage node in a distributed database.
- the transaction coordinator 110 may also be deployed in a middleware used to provide functional extension services for a distributed database.
- the transaction coordinator 110 may be deployed in a distributed database middleware (DDM) 12 shown in FIG. 2 .
- DDM distributed database middleware
- FIG2 it is a schematic diagram of an application environment of a transaction processing method provided in an embodiment of the present application.
- This embodiment takes the distributed transaction involving the operation of multiple database shards as an example, and includes an application 11, a DDM 12, and a database 13.
- the application 11 can be a cloud application for tenants, or other types of applications, which are not limited in this application.
- the database 13 can be a cloud database, or other types of databases, which are not limited in this application.
- the application does not limit this.
- DDM 12 can provide database 13 with capabilities such as sub-library and sub-table, read-write separation, and elastic expansion.
- the application 11 can be used to receive the user's operation instructions and create a transaction tr1.
- the data involved in transaction tr1 may be distributed in different database shards on multiple relational database service (RDS) instances.
- RDS relational database service
- a series of operations on the data table in the same database shard on each RDS instance is equivalent to a transaction branch (local transaction), that is, transaction tr1 can be equivalent to a distributed transaction composed of transaction branches on multiple RDS instances, and these transaction branches are either all executed successfully or all failed to execute successfully.
- DDM 12 plays the role of transaction coordinator in transaction tr1, and each RDS instance plays the role of transaction participant.
- the creation of transaction tr1 can also be executed by DDM 12.
- an overall execution flow chart of a distributed transaction provided in an embodiment of the present application is exemplarily introduced.
- DDM 12 can transparently encapsulate the two-phase commit of transactions.
- the application side does not need to care whether the underlying layer is a distributed transaction. It can use the ordinary transaction instructions BEGIN/COMMIT to perform distributed transaction operations.
- DDM 12 can automatically handle the two-phase commit logic of distributed transactions. If the transaction only involves one sub-library or one database, DDM 12 can also automatically downgrade the transaction to a local transaction committed in one phase.
- DDM 12 can map distributed transaction semantics with common transaction semantics. DDM 12 can determine whether the transaction is a local transaction or a distributed transaction by identifying whether the data of the transaction operation involves multiple sub-databases or multiple sub-tables. If multiple sub-databases or multiple sub-tables are involved, the transaction is determined to be a distributed transaction and a two-phase commit is performed. If only one sub-database or one sub-table is involved, the transaction is determined to be a local transaction and a one-phase commit is performed.
- the 3 takes a distributed transaction including two transaction branches (hereinafter referred to as a first transaction branch and a second transaction branch) as an example.
- the first transaction branch corresponds to a first RDS instance RDS1
- the second transaction branch corresponds to a second RDS instance RDS2.
- the first phase based on the XA protocol includes: starting an XA transaction (through the XA START instruction), executing business SQL (business SQL may include multiple SQL statements), ending the XA transaction (through the XA END instruction) and pre-committing the XA transaction (through the XA PREPARE instruction), thus completing the preparation operation for the business SQL.
- the second phase based on the XA protocol includes: committing an XA transaction or rolling back an XA transaction. Execute the XA COMMIT instruction to commit the XA transaction, and only then will the business SQL be truly committed to the database. Execute the XA ROLLBACK instruction to roll back the XA transaction, complete the rollback of the business SQL, and release the database lock resources.
- an XA transaction branch can be started through the XA START instruction, and the XA transaction branch is placed in the ACTIVE state.
- the SQL statements that constitute the XA transaction branch are executed, the boundary of the XA transaction branch can be specified, and then the XA END instruction is executed to put the XA transaction branch in the IDLE state, that is, to end the boundary of the XA transaction branch.
- the SQL statements between the XA START instruction and the XA END instruction constitute a transaction scope of this XA transaction branch.
- the XA transaction branch can be placed in the PREPARED state by executing the XA PREPARE instruction.
- the XA COMMIT instruction can be executed to commit, or the XA ROLLBACK instruction can be executed to roll back.
- XID globally unique distributed transaction number
- ID the transaction sequence number within the node
- timestamp the time stamp
- An XA transaction branch can be opened through the XA START "XID + DBNAME" instruction.
- the XID of the distributed transaction shown in Figure 3 is: 34287.DDM.1.20210421024418, where "34287” is the transaction sequence number in the node, "1" is the node ID, and "20210421024418" is the timestamp.
- the sub-library name corresponding to the first transaction branch is db_sysbench_0003
- the sub-library name corresponding to the second transaction branch is db_sysbench_0004.
- the XA_ID of the first transaction branch is: 34287.DDM.1.20210421024418.db_sysbench_0003
- the XA_ID of the second transaction branch is: 34287.DDM.1.20210421024418. db_sysbench_0004.
- multiple checkpoints are set in the first and second stages (to detect whether each process node in the transaction commit or rollback process is executed successfully or failed), and a write-ahead log (WAL) technology with persistence capability is used to record the transaction commit and rollback, so that after an exception occurs in a transaction, the unfinished distributed transaction can be continued to be promoted through log playback until the transaction is committed or rolled back, thereby avoiding pending transactions.
- Transaction exceptions are generally caused by an exception in the device where the transaction coordinator is located, or an exception in the device where the transaction participant is located, such as a device crash, power outage, network disconnection, restart, etc. After the device abnormality is recovered, the unfinished distributed transaction is continued to be promoted based on log playback until the transaction is committed or rolled back.
- the pre-commit process of a distributed transaction may include:
- the transaction coordinator writes a pre-log to record the transaction-related information of the first phase.
- the write-ahead log may be persisted, and the transaction-related information of the first phase may include transaction status information, SQL statements, and operation data.
- the transaction status information may include the XID of the distributed transaction, the status of the distributed transaction, the XA_ID of each transaction branch, and the status of each transaction branch.
- the operation data may be one or more items, and the operation data may be distributed in one sub-library or in multiple sub-libraries. The operation data may be identified by the table number and the primary key.
- the SQL statement is used to indicate what kind of processing is performed on the operation data.
- the write-ahead log is stored in a data structure of a key-value pair.
- FIG. 5 shows the organizational structure of the write-ahead log in a storage device. As can be seen from the organizational structure shown in FIG. 5, the write-ahead log is stored in a tree diagram organizational structure. The tree diagram includes multiple root blocks and multiple leaf blocks, and each root block corresponds to a leaf block.
- the leaf block is used to store the write-ahead log, and the execution result of the transaction (distributed transaction or transaction branch) recorded in the write-ahead log, wherein when the execution result is completed, it is used to indicate that the transaction recorded in the write-ahead log is successfully executed, and when the execution result is incomplete, it is used to indicate that the transaction recorded in the write-ahead log is not successfully executed, and log playback is required, so as to facilitate the retry (continue execution or rollback) of the transaction that has not been successfully executed based on the replayed write-ahead log.
- the root block is used to store the information of the leaf block, for example, the value range of the key of the write-ahead log in the leaf block stored in the root block and the storage address of the corresponding leaf block.
- the transaction coordinator if the transaction coordinator successfully writes the pre-log, the transaction coordinator records each transaction participant as being in the CONN state (also called the connection state), and notifies each transaction participant to set its own state to the PREPARING state (also called the preparing state).
- CONN state also called the connection state
- PREPARING state also called the preparing state
- each transaction participant in the first stage, by setting a checkpoint to determine whether the write-ahead log is successful, each transaction participant performs a pre-commit after the write-ahead log is successful, to avoid the inability to subsequently advance an abnormal transaction branch to a COMMITED state or a ROLLBACKED state based on log playback.
- An abnormal transaction branch may refer to a transaction branch whose final state is not a COMMITED state or a ROLLBACKED state.
- each transaction participant executes the "XA END" instruction and the "XA PREPARE" instruction.
- a checkpoint is set during the process of a transaction participant executing an "XA END" instruction and an "XA PREPARE" instruction, and a transaction participant that fails to execute the instruction is rolled back in advance.
- the transaction participant rolling back may refer to the transaction participant executing an "XA ROLLBACK" instruction.
- the transaction participant disconnects the network connection with the transaction coordinator, which may mean disconnecting the connection established based on the network or inter-process communication (IPC) mechanism, while the transaction coordinator The SESSION connection between the coordinator and the transaction participant is not disconnected. After the transaction participant disconnects the network connection with the transaction coordinator, the lock resources used by the transaction participant in processing the transaction branch can be released.
- IPC inter-process communication
- a checkpoint is set during the rollback process after a transaction participant fails to prepare, so that the rollback status of the transaction participant (whether the rollback is successful or failed) can be known in a timely manner.
- the transaction coordinator will not be able to receive feedback from the transaction participant that it is in the PREPARED state, that is, the transaction coordinator will not receive feedback from all transaction participants that they are in the PREPARED state.
- the transaction coordinator can send a notification to the transaction participants, so that each transaction participant can commit the transaction branch.
- the submission process of the distributed transaction may include:
- the transaction coordinator writes a pre-log to record the transaction-related information of the second phase.
- the write-ahead log may be persisted, and the transaction-related information of the second phase may also include transaction status information, SQL statements, and operation data.
- each transaction participant in the second stage, by setting a checkpoint to determine whether the write-ahead log is successful, each transaction participant commits the transaction after the write-ahead log is successful, thereby avoiding the inability to advance the abnormal transaction branch to the COMMITED state or the ROLLBACKED state based on log playback.
- a checkpoint is set during the process of a transaction participant executing an "XA COMMIT" instruction, and the transaction participant for which the instruction execution fails is set to the COMMIT_FAILED state, so that it can be known in a timely manner whether all transaction participants are in the COMMITED state.
- step S4_19 after the distributed transaction enters the COMMIT_FAILED state, the distributed transaction commit is retried, that is, jumping to step S4_10 to re-execute the distributed transaction commit.
- a checkpoint is set during a distributed transaction commit retry process to determine whether to perform log replay.
- the distributed transaction submission operation is re-executed until the number of failures is equal to a preset threshold, which can be set according to actual needs and is not limited in this application.
- a preset threshold which can be set according to actual needs and is not limited in this application.
- performing log playback may refer to playing back the write-ahead log of the transaction branch with the exception, determining the cause of failure of the abnormal transaction branch (for example, determining which process node caused the transaction branch to be unable to enter the COMMITED state or the ROLLBACKED state based on the checkpoint set above), and re-starting from the determined process node until the final state of the abnormal transaction branch is the COMMITED state or the ROLLBACKED state.
- the specific execution process of ending an XA transaction includes:
- each transaction participant executes the "XA END" instruction.
- a checkpoint is set during the process of a transaction participant executing an "XA END" instruction, and a rollback can be performed in advance for transaction participants whose execution fails.
- each transaction participant executes the "XA ROLLBACK" instruction.
- a checkpoint is set during the process of a transaction participant executing an "XA ROLLBACK" instruction, and a transaction participant that fails in the execution can attempt to roll back again.
- S7_8 If the distributed transaction cannot be rolled back again, or after the distributed transaction is rolled back again, all transaction participants are still not in the ROLLBACKED state, the transaction coordinator disconnects the SESSION connection (also called session connection) with each transaction participant, and writes the transaction participant information in the ROLLBACK_FAILED state into the log.
- SESSION connection also called session connection
- the transaction processing method may include:
- the data processed by the operation instruction involves multiple sub-libraries or multiple sub-tables, it indicates that there are multiple transaction participants.
- a sub-library or a sub-table can serve as a transaction participant.
- a first write-ahead log may be generated, and the transaction-related information of the first phase in the two-phase commit protocol may be recorded through the first write-ahead log; before each transaction branch is committed, a second write-ahead log may be generated, and the transaction-related information of the second phase in the two-phase commit protocol may be recorded through the second write-ahead log, so that when an abnormal transaction branch occurs, the abnormal transaction branch can be continued from the first phase or the second phase based on log playback.
- each transaction branch can be directly controlled to be rolled back, avoiding the problem that the abnormal transaction branch cannot be continued from the first phase or the second phase based on log playback.
- the submission of each transaction branch includes submitting a transaction process or rolling back a transaction process, and the submitting transaction process or rolling back a transaction process includes multiple process nodes, and each of the multiple process nodes is provided with a checkpoint, so that for an abnormal transaction branch, the transaction can be resubmitted or rolled back from a specified process node based on log playback.
- An abnormal transaction branch may refer to a transaction branch that is not in a preset transaction state. For example, if a transaction branch is not in a Rollbacked state or a Committed state, the transaction branch may be considered an abnormal transaction branch.
- Step S83 If it is determined that the distributed transaction contains an abnormal transaction branch, the abnormal transaction branch is processed based on the write-ahead log and the checkpoint.
- the abnormal transaction branch is caused by an abnormality in the device where the transaction coordinator is located, or an abnormality in the device where the transaction participant is located.
- the pre-written log corresponding to the abnormal transaction branch is played back to determine the process node for reprocessing the abnormal transaction branch, and the abnormal transaction branch is processed with the determined process node as the starting process node, so as to realize the re-advancement of the transaction for the abnormal transaction branch from the specified process node.
- a transaction participant of an abnormal transaction branch may send a request to re-advance the transaction to the transaction coordinator, which will replay the write-ahead log corresponding to the abnormal transaction branch, determine the process node for reprocessing the abnormal transaction branch, and notify the transaction participant (transaction participant of the abnormal transaction branch) to re-advance the transaction (commit or roll back the transaction) for the abnormal transaction branch using the determined process node as the starting process node.
- the distributed middleware 30 may include a creation module 301 , a submission module 302 and a processing module 303 .
- the creation module 301 is used to create a distributed transaction for the operation instruction in response to the received operation instruction, and the distributed transaction includes multiple transaction branches.
- the submission module 302 is used to submit distributed transactions based on a preset transaction submission mechanism and generate a pre-written log of the distributed transaction.
- the submission of each transaction branch in the multiple transaction branches includes multiple process nodes.
- the preset transaction submission mechanism includes a checkpoint for detecting the execution status of the multiple process nodes and the transaction status of each transaction branch.
- the processing module 303 is used to process the abnormal transaction branch based on the write-ahead log and the checkpoint when it is determined that the distributed transaction contains an abnormal transaction branch.
- the abnormal transaction branch is caused by an abnormality in the device where the transaction coordinator of the distributed transaction is located, or an abnormality in the device where the transaction participant of the distributed transaction is located.
- the creation module 301, the submission module 302 and the processing module 303 can all be implemented by software, or can be implemented by hardware.
- the creation module 301 is taken as an example to introduce the implementation of the creation module 301.
- the implementation of the submission module 302 and the processing module 303 can refer to the implementation of the creation module 301.
- the creation module 301 may include code running on a computing instance.
- the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more.
- the creation module 301 may include code running on multiple hosts/virtual machines/containers. It should be noted that the multiple hosts/virtual machines/containers used to run the code can be distributed in the same region (region) or in different regions.
- the multiple hosts/virtual machines/containers used to run the code can be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with close geographical locations. Among them, usually a region can include multiple AZs.
- VPC virtual private cloud
- multiple hosts/virtual machines/containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs.
- VPC virtual private cloud
- a VPC is set up in a region.
- a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.
- the creation module 301 may include at least one computing device, such as a server, etc.
- the creation module 301 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD).
- the PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.
- the multiple computing devices included in the creation module 301 can be distributed in the same region or in different regions.
- the multiple computing devices included in the creation module 301 can be distributed in the same AZ or in different AZs.
- the multiple computing devices included in the creation module 301 can be distributed in the same VPC or in multiple VPCs.
- the multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.
- the creation module 301 can be used to execute any step in the transaction processing method
- the submission module 302 can be used to execute any step in the transaction processing method
- the processing module 303 can be used to execute any step in the transaction processing method.
- the steps that the creation module 301, the submission module 302 and the processing module 303 are responsible for implementing can be specified as needed.
- the creation module 301, the submission module 302 and the processing module 303 respectively implement different steps in the transaction processing method to achieve all the functions of the decision-making device 30.
- a computing device 100 is provided in an embodiment of the present application, and the computing device 100 includes: a bus 102, a processor 104, a memory 106, and a communication interface 108.
- the processor 104, the memory 106, and the communication interface 108 communicate with each other through the bus 102.
- the computing device 100 may be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 100.
- the bus 102 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus.
- PCI peripheral component interconnect
- EISA extended industry standard architecture
- the lines can be divided into address bus, data bus, control bus, etc. For ease of representation, only one line is used in FIG10, but it does not mean that there is only one bus or one type of bus.
- Bus 104 may include a path for transmitting information between various components of computing device 100 (e.g., memory 106, processor 104, communication interface 108).
- the processor 104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP) or a digital signal processor (DSP).
- processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP) or a digital signal processor (DSP).
- CPU central processing unit
- GPU graphics processing unit
- MP microprocessor
- DSP digital signal processor
- the memory 106 may include a volatile memory, such as a random access memory (RAM).
- the processor 104 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).
- ROM read-only memory
- HDD hard disk drive
- SSD solid state drive
- the memory 106 stores executable program codes
- the processor 104 executes the executable program codes to respectively implement the functions of the aforementioned creation module 301, submission module 302, and processing module 303, thereby implementing the transaction processing method. That is, the memory 106 stores instructions for executing the transaction processing method.
- the processor 104 executes the executable program codes to respectively implement the functions of the aforementioned creation module 301, submission module 302, and processing module 303 as an example.
- the communication interface 108 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 100 and other devices or a communication network.
- a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 100 and other devices or a communication network.
- the embodiment of the present application also provides a computing device cluster 10.
- the computing device cluster 10 includes at least one computing device.
- the computing device may be a server for data storage, or a local server in a local data center.
- the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.
- the computing device cluster 10 includes at least one computing device 100.
- the memory 106 in one or more computing devices 100 in the computing device cluster 10 may store the same instructions for executing the transaction processing method.
- the memory 106 of one or more computing devices 100 in the computing device cluster 10 may also store partial instructions for executing the transaction processing method.
- the combination of one or more computing devices 100 may jointly execute instructions for executing the transaction processing method.
- the memory 106 in different computing devices 100 in the computing device cluster 10 can store different instructions, which are respectively used to execute part of the functions of the data access device. That is, the instructions stored in the memory 106 in different computing devices 100 can implement the functions of one or more modules among the creation module 301, the submission module 302 and the processing module 303.
- one or more computing devices in the computing device cluster 10 can be connected via a network.
- the network may be a wide area network or a local area network, etc.
- FIG. 11 shows a possible implementation. As shown in FIG. 11 , two computing devices 100A and 100B are connected via a network, taking the functions of one or more modules in the creation module 301, the submission module 302, and the processing module 303 as an example. Specifically, the network is connected via a communication interface in each computing device.
- the memory 106 in the computing device 100A stores instructions for executing the functions of the creation module 301 and the submission module 302. At the same time, the memory 106 in the computing device 100B stores instructions for executing the functions of the processing module 303.
- connection method between the computing device cluster 10 shown in Figure 11 can be considered to be that the transaction processing method provided in this application requires a large amount of data storage and reading, so it is considered to hand over the functions implemented by the processing module 303 to the computing device 100B for execution.
- the functions of the computing device 100A shown in FIG11 may also be completed by multiple computing devices 100.
- the functions of the computing device 100B may also be completed by multiple computing devices 100.
- the embodiment of the present application also provides a computer program product including instructions.
- the computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium.
- the at least one computing device executes a transaction processing method.
- the embodiment of the present application also provides a computer-readable storage medium.
- the computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media.
- the available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk).
- the computer-readable storage medium includes instructions that instruct the computing device to execute the transaction processing method.
- the disclosed devices and methods can be implemented in other ways.
- the device embodiments described above are schematic.
- the division of the modules or units is a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.
- Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
- the units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
- each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
- the above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
- the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium.
- the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the software product is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application.
- the aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Debugging And Monitoring (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present application relates to the technical field of computers, and provides a transaction processing method and a related device. The transaction processing method comprises: in response to a received operation instruction, creating a distributed transaction for the operation instruction, wherein the distributed transaction comprises a plurality of transaction branches; committing the distributed transaction on the basis of a preset transaction commitment mechanism, and generating a write ahead log of the distributed transaction, wherein the commitment of each transaction branch among the plurality of transaction branches comprises a plurality of process nodes, and the preset transaction commitment mechanism comprises a checkpoint for detecting execution conditions of the plurality of process nodes and a transaction state of each transaction branch; and if it is determined that the distributed transaction comprises an abnormal transaction branch, processing the abnormal transaction branch on the basis of the write ahead log and the checkpoint, wherein the abnormal transaction branch may be caused by an anomaly that occurs in a device where a transaction coordinator or a transaction participant of the distributed transaction is located. The present application can avoid pending transactions.
Description
本申请要求于2023年03月17日提交中国专利局,申请号为202310267376.8、申请名称为“事务处理方法及相关设备”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims priority to the Chinese patent application filed with the China Patent Office on March 17, 2023, with application number 202310267376.8 and application name “Transaction Processing Methods and Related Equipment”, the entire contents of which are incorporated by reference into this application.
本申请涉及计算机技术领域,尤其涉及一种事务处理方法及相关设备。The present application relates to the field of computer technology, and in particular to a transaction processing method and related equipment.
当存在多个事务参与者参与处理事务时,这个事务可称为分布式事务。目前,为了保证分布式事务提交的一致性,主流的事务提交协议通常为两阶段提交协议(two phase commitment protocol,2PC)。由于分布式事务涉及多个事务参与者的调度协同,相比传统的单机事务,分布式事务处理更加复杂。When there are multiple transaction participants involved in processing a transaction, this transaction can be called a distributed transaction. At present, in order to ensure the consistency of distributed transaction submission, the mainstream transaction submission protocol is usually a two-phase submission protocol (2PC). Since distributed transactions involve the scheduling and coordination of multiple transaction participants, distributed transaction processing is more complicated than traditional single-machine transactions.
现有的分布式失败事务重试机制,一般是基于内存中的事务日志记录进行失败事务重试。然而,基于内存中的事务日志记录进行失败事务重试可能无法保证失败事务都能得到正确处理,例如,若事务协调者出现故障,可能会在事务参与者上遗留未决的失败事务。The existing distributed failed transaction retry mechanism generally performs failed transaction retry based on the transaction log records in memory. However, failed transaction retry based on the transaction log records in memory may not guarantee that all failed transactions can be processed correctly. For example, if the transaction coordinator fails, pending failed transactions may be left on the transaction participants.
发明内容Summary of the invention
有鉴于此,有必要提供一种事务处理方法,可确保事务能得到正确处理,避免出现悬而未决的事务。In view of this, it is necessary to provide a transaction processing method to ensure that transactions can be processed correctly and avoid pending transactions.
本申请实施例第一方面公开了一种事务处理方法,包括:响应于接收到的操作指令,为操作指令创建分布式事务,分布式事务包括多个事务分支;基于预设事务提交机制提交分布式事务,及生成分布式事务的预写日志,多个事务分支中的每个事务分支的提交包括多个流程节点,预设事务提交机制包括用于检测多个流程节点的执行情况及每个事务分支的事务状态的检查点;若确定分布式事务包含异常的事务分支,基于预写日志及检查点对异常的事务分支进行处理,异常的事务分支由分布式事务的事务协调者所在的设备发生异常,或者分布式事务的事务参与者所在的设备发生异常引起。In a first aspect, an embodiment of the present application discloses a transaction processing method, comprising: in response to a received operation instruction, creating a distributed transaction for the operation instruction, the distributed transaction including multiple transaction branches; committing the distributed transaction based on a preset transaction commit mechanism, and generating a pre-write log of the distributed transaction, the commit of each transaction branch in the multiple transaction branches including multiple process nodes, the preset transaction commit mechanism including a checkpoint for detecting the execution status of the multiple process nodes and the transaction status of each transaction branch; if it is determined that the distributed transaction includes an abnormal transaction branch, processing the abnormal transaction branch based on the pre-write log and the checkpoint, the abnormal transaction branch is caused by an abnormality in the device where the transaction coordinator of the distributed transaction is located, or an abnormality in the device where the transaction participant of the distributed transaction is located.
采用上述技术方案,分布式事务的提交可能会由于分布式事务的事务协调者所在的设备,或者分布式事务的事务参与者所在的设备发生异常(设备宕机、断电、断网、重启等),导致出现悬而未决的状态,通过在事务提交阶段采取带有持久化存储能力的预写日志对事务提交过程进行记录,同时通过在事务提交过程的多个流程节点设置检查点进行执行情况与事务状态的检查,对于出现异常的事务,可实现在设备异常恢复后,基于日志回放可从指定的流程节点重新进行事务推进,为事务异常提供可靠的事务恢复能力,确保事务能得到正确处理,避免出现悬而未决的事务。With the above technical solution, the submission of a distributed transaction may be left in a pending state due to an exception in the device where the transaction coordinator of the distributed transaction is located, or in the device where the transaction participant of the distributed transaction is located (device crash, power outage, network disconnection, restart, etc.). The transaction submission process is recorded by using a pre-write log with persistent storage capability during the transaction submission phase, and checkpoints are set at multiple process nodes in the transaction submission process to check the execution status and transaction status. For transactions with exceptions, after the device exception is recovered, the transaction can be re-promoted from the specified process node based on log playback, providing reliable transaction recovery capabilities for transaction exceptions, ensuring that transactions can be handled correctly, and avoiding pending transactions.
在一些实施例中,为操作指令创建分布式事务,包括:若操作指令处理的数据涉及多个分库或者多个分表,为操作指令创建分布式事务。In some embodiments, creating a distributed transaction for an operation instruction includes: if data processed by the operation instruction involves multiple sub-databases or multiple sub-tables, creating a distributed transaction for the operation instruction.
采用上述技术方案,在确定操作指令处理的数据涉及多个分库或者多个分表,表明存在多个事务参与者(一个分库或者一个分表可作为事务参与者),通过为操作指令创建分布式事务,可实现对位于多个分库或者多个分表内的数据进行一致性处理。By adopting the above technical solution, when it is determined that the data processed by the operation instruction involves multiple sub-libraries or multiple sub-tables, indicating that there are multiple transaction participants (one sub-library or one sub-table can be a transaction participant), by creating a distributed transaction for the operation instruction, consistency processing of the data located in multiple sub-libraries or multiple sub-tables can be achieved.
在一些实施例中,预设事务提交机制为两阶段提交协议,生成所述分布式事务的预写日志,包括:在每个事务分支进行预提交之前,生成第一预写日志,第一预写日志用于记录两阶段提交协议中的第一阶段的事务相关信息;在每个事务分支进行提交之前,生成第二预写日志,第二预写日志用于记录两阶段提交协议中的第二阶段的事务相关信
息;第一阶段的事务相关信息及第二阶段的事务相关信息均包括分布式事务的事务状态信息、事务分支的事务状态信息、操作数据和用于对操作数据进行处理的结构化查询语言(structured query language,SQL)语句。In some embodiments, the transaction commit mechanism is preset as a two-phase commit protocol, and a write-ahead log of the distributed transaction is generated, including: before each transaction branch is pre-committed, a first write-ahead log is generated, and the first write-ahead log is used to record transaction-related information of the first phase in the two-phase commit protocol; before each transaction branch is committed, a second write-ahead log is generated, and the second write-ahead log is used to record transaction-related information of the second phase in the two-phase commit protocol. The transaction-related information of the first phase and the transaction-related information of the second phase both include the transaction status information of the distributed transaction, the transaction status information of the transaction branch, the operation data, and the structured query language (SQL) statement for processing the operation data.
采用上述技术方案,在使用两阶段提交协议提交事务时,通过在事务的第一阶段与第二阶段采取带有持久化存储能力的预写日志对全局事务的事务状态信息、事务分支的事务状态信息、操作数据和用于对操作数据进行处理的SQL语句进行记录,实现为事务异常提供可靠的事务恢复能力,确保异常事务可基于日志回放能得到正确处理,避免出现悬而未决的事务。By adopting the above technical solution, when a transaction is committed using the two-phase commit protocol, a write-ahead log with persistent storage capability is used in the first and second phases of the transaction to record the transaction status information of the global transaction, the transaction status information of the transaction branch, the operation data, and the SQL statement used to process the operation data, thereby providing a reliable transaction recovery capability for transaction exceptions, ensuring that abnormal transactions can be correctly processed based on log playback, and avoiding pending transactions.
在一些实施例中,事务处理方法还包括:若第一预写日志或第二预写日志生成失败,控制每个事务分支进行回滚。In some embodiments, the transaction processing method further includes: if the generation of the first write-ahead log or the second write-ahead log fails, controlling each transaction branch to roll back.
采用上述技术方案,若第一预写日志或第二预写日志生成失败,表明无法通过预写日志对事务的第一阶段或第二阶段进行记录,导致无法基于日志回放对异常的事务从第一阶段或第二阶段进行继续推进,通过控制每个事务分支进行回滚,可避免出现后续无法基于日志回放对异常的事务从第一阶段或第二阶段进行继续推进的问题。By adopting the above technical solution, if the generation of the first write-ahead log or the second write-ahead log fails, it indicates that the first stage or the second stage of the transaction cannot be recorded through the write-ahead log, resulting in the inability to continue to advance the abnormal transaction from the first stage or the second stage based on log playback. By controlling each transaction branch to roll back, the problem of being unable to continue to advance the abnormal transaction from the first stage or the second stage based on log playback can be avoided.
在一些实施例中,每个事务分支的提交包括提交事务流程或回滚事务流程,提交事务流程或回滚事务流程均包括多个流程节点,多个流程节点中的每个流程节点均设置有检查点。In some embodiments, the submission of each transaction branch includes submitting a transaction process or rolling back a transaction process, and the submitting transaction process or rolling back a transaction process both include multiple process nodes, and each of the multiple process nodes is provided with a checkpoint.
采用上述技术方案,提交事务流程与回滚事务流程均可包括多个流程节点,通过在提交事务流程与回滚事务流程中设置多个检查点检查各个流程节点的执行情况,可实现在设备异常恢复后基于日志回放可从指定的流程节点重新进行事务提交或者事务回滚,为事务异常提供可靠的事务恢复能力,确保事务能得到正确处理,避免出现悬而未决的事务。By adopting the above technical solution, both the transaction submission process and the transaction rollback process can include multiple process nodes. By setting multiple checkpoints in the transaction submission process and the transaction rollback process to check the execution status of each process node, it is possible to re-submit the transaction or roll back the transaction from the specified process node based on log playback after the device abnormality is recovered, thereby providing reliable transaction recovery capabilities for transaction abnormalities, ensuring that transactions can be processed correctly, and avoiding pending transactions.
在一些实施例中,基于预设事务提交机制提交分布式事务之后,还包括:将未处于预设事务状态的事务分支确定为异常的事务分支,预设事务状态包括Rollbacked状态或Committed状态。In some embodiments, after the distributed transaction is committed based on a preset transaction commit mechanism, the method further includes: determining a transaction branch that is not in a preset transaction state as an abnormal transaction branch, where the preset transaction state includes a Rollbacked state or a Committed state.
采用上述技术方案,将未处于Rollbacked状态或Committed状态的事务分支确定为异常的事务分支,对于异常的事务分支,可基于日志回放从指定的流程节点重新进行事务提交或者事务回滚,直至处于Committed状态或Rollbacked状态,确保事务能得到正确处理,避免出现悬而未决的事务。By adopting the above technical solution, a transaction branch that is not in the Rollbacked state or the Committed state is determined as an abnormal transaction branch. For the abnormal transaction branch, the transaction can be re-committed or rolled back from the specified process node based on log playback until it is in the Committed state or the Rollbacked state, ensuring that the transaction can be processed correctly and avoiding pending transactions.
在一些实施例中,基于预写日志及检查点对异常的事务分支进行处理,包括:回放与异常的事务分支对应的预写日志,确定重新处理异常的事务分支的流程节点;以确定的流程节点为起始流程节点对异常的事务分支进行处理。In some embodiments, an abnormal transaction branch is processed based on a write-ahead log and a checkpoint, including: replaying the write-ahead log corresponding to the abnormal transaction branch, determining a process node for reprocessing the abnormal transaction branch; and processing the abnormal transaction branch with the determined process node as the starting process node.
采用上述技术方案,对于出现异常的事务,可实现在设备异常恢复后回放与异常的事务分支对应的预写日志,提升日志回放效率,基于日志回放可从指定的流程节点重新进行事务推进,为事务异常提供可靠的事务恢复能力,确保事务能得到正确处理,避免出现悬而未决的事务。By adopting the above technical solution, for abnormal transactions, the pre-written log corresponding to the abnormal transaction branch can be replayed after the device abnormality is recovered, thereby improving the log playback efficiency. Based on the log playback, the transaction can be re-promoted from the specified process node, providing reliable transaction recovery capabilities for transaction exceptions, ensuring that the transaction can be processed correctly, and avoiding pending transactions.
第二方面,本申请实施例提供一种分布式中间件,包括:创建模块,用于响应于接收到的操作指令,为操作指令创建分布式事务,分布式事务包括多个事务分支;提交模块,用于基于预设事务提交机制提交分布式事务,及生成分布式事务的预写日志,多个事务分支中的每个事务分支的提交包括多个流程节点,预设事务提交机制包括用于检测多个流程节点的执行情况及每个事务分支的事务状态的检查点;处理模块,用于在确定分布式事务包含异常的事务分支时,基于预写日志及所述检查点对异常的事务分支进行处理,异常的事务分支由分布式事务的事务协调者所在的设备发生异常,或者分布式事务的事务参与所在的设备发生异常引起。In a second aspect, an embodiment of the present application provides a distributed middleware, including: a creation module, used to create a distributed transaction for the operation instruction in response to a received operation instruction, the distributed transaction including multiple transaction branches; a submission module, used to submit the distributed transaction based on a preset transaction submission mechanism, and generate a pre-write log of the distributed transaction, the submission of each transaction branch in the multiple transaction branches includes multiple process nodes, and the preset transaction submission mechanism includes a checkpoint for detecting the execution status of the multiple process nodes and the transaction status of each transaction branch; a processing module, used to process the abnormal transaction branch based on the pre-write log and the checkpoint when it is determined that the distributed transaction contains an abnormal transaction branch, the abnormal transaction branch is caused by an abnormality in the device where the transaction coordinator of the distributed transaction is located, or an abnormality in the device where the transaction participant of the distributed transaction is located.
采用上述技术方案,分布式事务的提交可能会由于分布式事务的事务协调者所在的设备,或者分布式事务的事务参与者所在的设备发生异常(设备宕机、断电、断网、重启等),导致出现悬而未决的状态,通过在事务提交阶段采取带有持久化存储能力
的预写日志对事务提交过程进行记录,同时通过在事务提交过程的多个流程节点设置检查点进行执行情况与事务状态的检查,对于出现异常的事务,可实现在设备异常恢复后,基于日志回放可从指定的流程节点重新进行事务推进,为事务异常提供可靠的事务恢复能力,确保事务能得到正确处理,避免出现悬而未决的事务。With the above technical solution, the submission of distributed transactions may be in a pending state due to an abnormality of the device where the transaction coordinator of the distributed transaction is located, or the device where the transaction participant of the distributed transaction is located (device downtime, power outage, network disconnection, restart, etc.). The write-ahead log records the transaction submission process, and checks the execution status and transaction status by setting checkpoints at multiple process nodes in the transaction submission process. For abnormal transactions, after the device abnormality is recovered, the transaction can be re-promoted from the specified process node based on log playback, providing reliable transaction recovery capabilities for transaction exceptions, ensuring that transactions can be handled correctly, and avoiding pending transactions.
第三方面,本申请实施例提供一种计算机可读存储介质,包括计算机程序指令,当计算机程序指令由计算设备集群执行时,使得计算设备集群执行如第一方面所述的事务处理方法。In a third aspect, an embodiment of the present application provides a computer-readable storage medium, comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the transaction processing method as described in the first aspect.
第四方面,本申请实施例提供一种计算设备集群,包括至少一个计算设备,每个计算设备包括处理器和存储器;至少一个计算设备的处理器用于执行至少一个计算设备的存储器中存储的指令,以使得计算设备集群执行如第一方面所述的事务处理方法。In a fourth aspect, an embodiment of the present application provides a computing device cluster, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the transaction processing method described in the first aspect.
第五方面,本申请实施例提供一种计算机程序产品,当计算机程序产品被计算设备集群运行时,使得计算设备集群执行如第一方面所述的事务处理方法。In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is executed by a computing device cluster, the computing device cluster executes the transaction processing method as described in the first aspect.
第六方面,提供一种装置,所述装置具有实现上述第一方面所提供的方法中计算设备集群行为的功能。功能可以通过硬件实现,也可以通过硬件执行相应的软件实现。硬件或软件包括一个或多个与上述功能相对应的模块。In a sixth aspect, a device is provided, wherein the device has the function of implementing the computing device cluster behavior in the method provided in the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.
可以理解地,上述提供的第三方面所述的计算机可读存储介质,第四方面所述的计算设备集群,第五方面所述的计算机程序产品,第六方面所述的装置均与上述第一方面的方法对应,因此,其所能达到的有益效果可参考上文所提供的对应的方法中的有益效果,此处不再赘述。It can be understood that the computer-readable storage medium described in the third aspect, the computing device cluster described in the fourth aspect, the computer program product described in the fifth aspect, and the apparatus described in the sixth aspect all correspond to the method of the first aspect. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.
图1为本申请一实施例提供的分布式事务提交系统的架构示意图;FIG1 is a schematic diagram of the architecture of a distributed transaction submission system provided by an embodiment of the present application;
图2为本申请一实施例提供的事务处理方法的应用环境示意图;FIG2 is a schematic diagram of an application environment of a transaction processing method provided by an embodiment of the present application;
图3为本申请一实施例提供的分布式事务的整体执行流程示意图;FIG3 is a schematic diagram of the overall execution flow of a distributed transaction provided by an embodiment of the present application;
图4为本申请一实施例提供的分布式事务进行两阶段提交的流程示意图;FIG4 is a schematic diagram of a process of performing two-phase commit of a distributed transaction provided by an embodiment of the present application;
图5为本申请一实施例提供的预写日志在存储设备中的组织结构示意图;FIG5 is a schematic diagram of the organizational structure of a write-ahead log in a storage device according to an embodiment of the present application;
图6为本申请一实施例提供的结束分布式事务的流程示意图;FIG6 is a schematic diagram of a process of ending a distributed transaction provided by an embodiment of the present application;
图7为本申请一实施例提供的分布式事务进行回滚的流程示意图;FIG7 is a schematic diagram of a process of rolling back a distributed transaction provided by an embodiment of the present application;
图8为本申请一实施例提供的事务处理方法的步骤流程示意图;FIG8 is a schematic diagram of a process flow of a transaction processing method according to an embodiment of the present application;
图9为本申请一实施例提供的分布式数据库中间件的模块示意图;FIG9 is a schematic diagram of a module of a distributed database middleware provided in an embodiment of the present application;
图10为本申请一实施例提供的一种计算设备的结构示意图;FIG10 is a schematic diagram of the structure of a computing device provided by an embodiment of the present application;
图11为本申请一实施例提供的一种计算设备集群的结构示意图。FIG. 11 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application.
需要说明的是,本申请中“至少一个”是指一个或者多个,“多个”是指两个或多于两个。“和/或”,描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B可以表示:单独存在A,同时存在A和B,单独存在B的情况,其中A,B可以是单数或者复数。本申请的说明书和权利要求书及附图中的术语“第一”、“第二”、“第三”、“第四”等(如果存在)是用于区别类似的对象,而不是用于描述特定的顺序或先后次序。It should be noted that in this application, "at least one" means one or more, and "more than one" means two or more than two. "And/or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and/or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and drawings of this application are used to distinguish similar objects, rather than to describe a specific order or sequence.
在本申请实施例中,“示例性的”或者“例如”等词用于表示作例子、例证或说明。本申请实施例中被描述为“示例性的”或者“例如”的任何实施例或设计方案不应被解释为比其它实施例或设计方案更优选或更具优势。确切而言,使用“示例性的”或者“例如”等词旨在以具体方式呈现相关概念。In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.
为便于理解本申请实施例,接下来对本请的应用场景进行介绍,本申请实施例描述的业务场景是为了更加清楚的说明本申请实施例的技术方案,并不构成对于本
申请实施例提供的技术方案的限定,本领域普通技术人员可知,随着新业务场景的出现,本申请实施例提供的技术方案对于类似的技术问题,同样适用。To facilitate understanding of the embodiments of the present application, the application scenarios of the present application are introduced below. The business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute any As for the limitation of the technical solutions provided in the application embodiments, it is known to those skilled in the art that, with the emergence of new business scenarios, the technical solutions provided in the application embodiments are also applicable to similar technical problems.
为了便于理解本申请实施例中的技术方案,下面首先对本申请实施例中所涉及的部分术语进行解释:In order to facilitate understanding of the technical solutions in the embodiments of the present application, some of the terms involved in the embodiments of the present application are first explained below:
分布式数据库:将原来集中式数据库中的数据分散存储到多个通过网络连接的数据存储节点上,以获取更大的存储容量和更高的并发量。其具体可以采用分库分表技术将海量的数据分片存储到分布式数据库的各个存储节点中。概括来讲,分库分表技术是指按照既定的分区策略将大表的数据分片存储到各个存储节点,或者将大表切分成各个数据量较小的业务子表,并将各个子表按照既定的分区策略存储到各个存储节点。Distributed database: The data in the original centralized database is stored in multiple data storage nodes connected by the network to obtain larger storage capacity and higher concurrency. Specifically, the database and table sharding technology can be used to store massive data shards in each storage node of the distributed database. In general, the database and table sharding technology refers to storing the data shards of a large table in each storage node according to the established partitioning strategy, or dividing a large table into business sub-tables with smaller data volumes, and storing each sub-table in each storage node according to the established partitioning strategy.
事务(transaction):是指一组连续的操作,这些操作组合成一段完整的逻辑,执行完后这组操作后,事务提交系统需要从一个一致性(consistent)的状态转换到另一个一致性的状态。上述操作组合中的全部操作要么全部执行成功,要么全部都未执行成功,如果操作组合中部分操作执行成功,部分操作未执行成功,则需要将执行成功的操作回滚(rollback)至未执行成功之前的状态。Transaction: refers to a group of continuous operations, which are combined into a complete logic. After executing this group of operations, the transaction commit system needs to switch from one consistent state to another consistent state. All operations in the above operation combination are either successfully executed or not successfully executed. If some operations in the operation combination are successfully executed and some operations are not successfully executed, the successful operations need to be rolled back to the state before the unsuccessful execution.
XA协议:由X/Open组织提出的分布式事务处理规范,主要定义了事务管理器(transaction manager,TM)和局部资源管理器(Local Resource Manager,RM)之间的接口。事务管理器也可称为事务协调者,用于负责分配事务唯一标识,监控事务的执行进度,并负责事务的提交、回滚等。局部资源管理器可以是数据库、文件系统等,可提供访问资源的方式。应用程序(Application Program),用于定义事务边界(即定义事务的开始和结束),并且在事务边界内对资源进行操作。XA protocol: A distributed transaction processing specification proposed by the X/Open organization, which mainly defines the interface between the transaction manager (TM) and the local resource manager (RM). The transaction manager, also known as the transaction coordinator, is responsible for allocating transaction unique identifiers, monitoring the progress of transaction execution, and committing and rolling back transactions. The local resource manager can be a database, file system, etc., which can provide a way to access resources. The application (Application Program) is used to define the transaction boundaries (that is, define the start and end of the transaction) and operate on resources within the transaction boundaries.
基于XA协议的两阶段提交协议:将事务的提交过程分为准备阶段(prepare phase),即两阶段提交协议中的第一阶段,以及提交阶段(commit phase),即两阶段提交协议中的第二阶段,可由事务协调者来协调所有事务参与者,如果在第一阶段所有事务参与者(RM)都可以提交事务,则进行第二阶段的提交每个RM上的事务分支提交,否则事务协调者回滚每个RM上的事务分支。Two-phase commit protocol based on XA protocol: The transaction commit process is divided into the prepare phase, which is the first phase in the two-phase commit protocol, and the commit phase, which is the second phase in the two-phase commit protocol. The transaction coordinator can coordinate all transaction participants. If all transaction participants (RM) can commit the transaction in the first phase, the transaction branch on each RM is committed in the second phase. Otherwise, the transaction coordinator rolls back the transaction branch on each RM.
结构化查询语言(structured query language,SQL):是一种数据库查询和程序设计语言,用于存取数据以及查询、更新和管理关系数据库系统。Structured query language (SQL): is a database query and programming language used to access data and query, update, and manage relational database systems.
如图1所示,示例性的介绍本申请一实施例提供的分布式事务提交系统的架构示意图。As shown in FIG1 , a schematic diagram of the architecture of a distributed transaction submission system provided in an embodiment of the present application is exemplarily introduced.
在该实施例中,分布式事务提交系统可以包括事务协调者110及多个事务参与者120。事务参与者120用于处理分布式事务。事务协调者110用于协调管理多个事务参与者120处理分布式事务。分布式事务也可称为全局事务,分布式事务可对应多个事务分支,多个事务分支与多个事务参与者120一一对应。In this embodiment, the distributed transaction submission system may include a transaction coordinator 110 and multiple transaction participants 120. The transaction participants 120 are used to process distributed transactions. The transaction coordinator 110 is used to coordinate and manage multiple transaction participants 120 to process distributed transactions. A distributed transaction may also be called a global transaction. A distributed transaction may correspond to multiple transaction branches, and multiple transaction branches correspond to multiple transaction participants 120 one by one.
在一些实施例中,事务协调者110可以是从多个事务参与者120中选出的,例如,事务协调者110和多个事务参与者120中的某一个事务参与者可以为同一个节点。又如,事务协调者110和多个事务参与者120也可以为不同的节点,本申请实施例对此不做限定。节点的具体实现可以是服务器或者虚拟机,本申请实施例对此不作限定。例如,节点可以是指分布式数据库中的数据存储节点。In some embodiments, the transaction coordinator 110 may be selected from a plurality of transaction participants 120. For example, the transaction coordinator 110 and one of the plurality of transaction participants 120 may be the same node. For another example, the transaction coordinator 110 and the plurality of transaction participants 120 may also be different nodes, which is not limited in the embodiments of the present application. The specific implementation of a node may be a server or a virtual machine, which is not limited in the embodiments of the present application. For example, a node may refer to a data storage node in a distributed database.
在一些实施例中,事务协调者110也可以部署在用于为分布式数据库提供功能扩展服务的中间件,例如事务协调者110可部署在图2所示的分布式数据库中间件(distributed database middleware,DDM)12中。In some embodiments, the transaction coordinator 110 may also be deployed in a middleware used to provide functional extension services for a distributed database. For example, the transaction coordinator 110 may be deployed in a distributed database middleware (DDM) 12 shown in FIG. 2 .
如图2所示,示例性的介绍本申请一实施例提供的事务处理方法的应用环境示意图。As shown in FIG2 , it is a schematic diagram of an application environment of a transaction processing method provided in an embodiment of the present application.
该实施例以分布式事务涉及操作多个数据库分片为例,该实施例包括应用程序11、DDM 12及数据库13。应用程序11可以是面向租户的云上应用,也可以是其他类型的应用,本申请对此不作限定。数据库13可以是云数据库,或者其他类型的数据库,本
申请对此不作限定。DDM 12可为数据库13提供分库分表、读写分离、弹性扩容等能力。This embodiment takes the distributed transaction involving the operation of multiple database shards as an example, and includes an application 11, a DDM 12, and a database 13. The application 11 can be a cloud application for tenants, or other types of applications, which are not limited in this application. The database 13 can be a cloud database, or other types of databases, which are not limited in this application. The application does not limit this. DDM 12 can provide database 13 with capabilities such as sub-library and sub-table, read-write separation, and elastic expansion.
应用程序11可用于接收用户的操作指令,创建一个事务tr1,事务tr1涉及的数据可能分布在多个关系型数据库服务(relational database service,RDS)实例上的不同数据库分片内。每个RDS实例上的同一数据库分片内所发生的对分片内数据表的一系列操作相当于一个事务分支(本地事务),即事务tr1可相当于由多个RDS实例上的事务分支所组成的分布式事务,这些事务分支要么全部执行成功,要么全部都未执行成功。DDM 12在事务tr1中作为事务协调者的角色,各个RDS实例作为事务参与者的角色。在一些实施例中,也可以由DDM 12执行事务tr1的创建。The application 11 can be used to receive the user's operation instructions and create a transaction tr1. The data involved in transaction tr1 may be distributed in different database shards on multiple relational database service (RDS) instances. A series of operations on the data table in the same database shard on each RDS instance is equivalent to a transaction branch (local transaction), that is, transaction tr1 can be equivalent to a distributed transaction composed of transaction branches on multiple RDS instances, and these transaction branches are either all executed successfully or all failed to execute successfully. DDM 12 plays the role of transaction coordinator in transaction tr1, and each RDS instance plays the role of transaction participant. In some embodiments, the creation of transaction tr1 can also be executed by DDM 12.
如图3所示,示例性的介绍本申请一实施例提供的分布式事务的整体执行流程图。As shown in FIG3 , an overall execution flow chart of a distributed transaction provided in an embodiment of the present application is exemplarily introduced.
在该实施例中,DDM 12可对事务的两阶段提交进行透明化封装。应用侧使用事务的时候不需要关心底层是否为分布式事务,使用普通事务指令BEGIN/COMMIT即可进行分布式事务操作。DDM 12可自动处理分布式事务的两阶段提交逻辑。如果事务只涉及一个分库或一个数据库,DDM 12还可自动将事务降级为一阶段提交的本地事务。In this embodiment, DDM 12 can transparently encapsulate the two-phase commit of transactions. When using transactions, the application side does not need to care whether the underlying layer is a distributed transaction. It can use the ordinary transaction instructions BEGIN/COMMIT to perform distributed transaction operations. DDM 12 can automatically handle the two-phase commit logic of distributed transactions. If the transaction only involves one sub-library or one database, DDM 12 can also automatically downgrade the transaction to a local transaction committed in one phase.
例如,DDM 12可将分布式事务语义与普通事务语义进行映射,DDM 12可通过识别事务操作的数据是否涉及到多个分库或多个分表确定该事务是本地事务还是分布式事务。若涉及到多个分库或多个分表,确定该事务为分布式事务,进行两阶段提交。若只涉及一个分库或一个分表,确定该事务是本地事务,按照一阶段进行提交。For example, DDM 12 can map distributed transaction semantics with common transaction semantics. DDM 12 can determine whether the transaction is a local transaction or a distributed transaction by identifying whether the data of the transaction operation involves multiple sub-databases or multiple sub-tables. If multiple sub-databases or multiple sub-tables are involved, the transaction is determined to be a distributed transaction and a two-phase commit is performed. If only one sub-database or one sub-table is involved, the transaction is determined to be a local transaction and a one-phase commit is performed.
图3以分布式事务包括两个事务分支(以下称为第一事务分支、第二事务分支)为例。第一事务分支对应第一RDS实例RDS1,第二事务分支对应第二RDS实例RDS2。3 takes a distributed transaction including two transaction branches (hereinafter referred to as a first transaction branch and a second transaction branch) as an example. The first transaction branch corresponds to a first RDS instance RDS1, and the second transaction branch corresponds to a second RDS instance RDS2.
基于XA协议的第一阶段包括:开启XA事务(通过XA START指令)、执行业务SQL(业务SQL可包括多条SQL语句)、结束XA事务(通过XA END指令)及预提交XA事务(通过XA PREPARE指令),这样便完成业务SQL的准备操作。The first phase based on the XA protocol includes: starting an XA transaction (through the XA START instruction), executing business SQL (business SQL may include multiple SQL statements), ending the XA transaction (through the XA END instruction) and pre-committing the XA transaction (through the XA PREPARE instruction), thus completing the preparation operation for the business SQL.
基于XA协议的第二阶段包括:提交XA事务或回滚XA事务。执行XA COMMIT指令,提交XA事务,此时业务SQL才算真正提交至数据库。执行XA ROLLBACK指令,回滚XA事务,完成业务SQL回滚,释放数据库锁资源。The second phase based on the XA protocol includes: committing an XA transaction or rolling back an XA transaction. Execute the XA COMMIT instruction to commit the XA transaction, and only then will the business SQL be truly committed to the database. Execute the XA ROLLBACK instruction to roll back the XA transaction, complete the rollback of the business SQL, and release the database lock resources.
在第一阶段,可通过XA START指令开启一个XA事务分支,该XA事务分支置于ACTIVE状态。对于处于ACTIVE状态的XA事务分支,执行构成XA事务分支的SQL语句,可指定XA事务分支的边界,然后再执行XA END指令,将XA事务分支置于IDLE状态,也即结束XA事务分支边界,在XA START指令和XA END指令之间的SQL语句构成本XA事务分支的一个事务范围。对于处于IDLE状态的XA事务分支,可通过执行XA PREPARE指令,将XA事务分支置于PREPARED状态。在第二阶段,对于处于PREPARED状态的XA事务分支,可以执行XA COMMIT指令进行提交,或者执行XA ROLLBACK指令进行回滚。In the first stage, an XA transaction branch can be started through the XA START instruction, and the XA transaction branch is placed in the ACTIVE state. For the XA transaction branch in the ACTIVE state, the SQL statements that constitute the XA transaction branch are executed, the boundary of the XA transaction branch can be specified, and then the XA END instruction is executed to put the XA transaction branch in the IDLE state, that is, to end the boundary of the XA transaction branch. The SQL statements between the XA START instruction and the XA END instruction constitute a transaction scope of this XA transaction branch. For the XA transaction branch in the IDLE state, the XA transaction branch can be placed in the PREPARED state by executing the XA PREPARE instruction. In the second stage, for the XA transaction branch in the PREPARED state, the XA COMMIT instruction can be executed to commit, or the XA ROLLBACK instruction can be executed to roll back.
对于分布式事务而言,可分配一个全局维度的分布式事务唯一编号(XID),XID由节点内事务序号、节点标识(Identity document,ID)和时间戳组成。XID可再加上对应分库名(也可称为DBNAME)构成实际在物理库上开启的事务分支的唯一编号(XA_ID),即XA_ID=XID+DBNAME,通过XA START“XID+DBNAME”指令可实现开启一个XA事务分支。For distributed transactions, a globally unique distributed transaction number (XID) can be assigned. XID consists of the transaction sequence number within the node, the node identifier (identity document, ID) and the timestamp. XID can be added with the corresponding sub-database name (also called DBNAME) to form the unique number (XA_ID) of the transaction branch actually opened on the physical database, that is, XA_ID = XID + DBNAME. An XA transaction branch can be opened through the XA START "XID + DBNAME" instruction.
假设图3中所示的分布式事务的XID为:34287.DDM.1.20210421024418,其中“34287”为节点内事务序号,“1”为节点ID,“20210421024418”为时间戳,第一事务分支对应的分库名为db_sysbench_0003,第二事务分支对应的分库名为db_sysbench_0004。即第一事务分支的XA_ID为:34287.DDM.1.20210421024418.db_sysbench_0003,第二事务分支的XA_ID为:34287.DDM.1.20210421024418.
db_sysbench_0004。Assume that the XID of the distributed transaction shown in Figure 3 is: 34287.DDM.1.20210421024418, where "34287" is the transaction sequence number in the node, "1" is the node ID, and "20210421024418" is the timestamp. The sub-library name corresponding to the first transaction branch is db_sysbench_0003, and the sub-library name corresponding to the second transaction branch is db_sysbench_0004. That is, the XA_ID of the first transaction branch is: 34287.DDM.1.20210421024418.db_sysbench_0003, and the XA_ID of the second transaction branch is: 34287.DDM.1.20210421024418. db_sysbench_0004.
参照图4所示,示例性的介绍本申请一实施例提供的分布式事务进行两阶段提交的流程图。4 , a flowchart of two-phase commit of a distributed transaction provided by an embodiment of the present application is exemplarily introduced.
在该实施例中,通过在第一阶段与第二阶段中设置多个检查点(检测事务提交或者回滚过程中的各个流程节点是执行成功,还是执行失败),且采用带有持久化能力的预写日志(write ahead log,WAL)技术对事务提交、回滚进行记录,实现在事务出现异常后,通过日志回放可以继续推进未完成的分布式事务,直至完成事务提交或完成事务回滚,避免出现未决的事务。事务出现异常一般是由事务协调者所在的设备,或者事务参与者所在的设备发生异常引起,例如设备出现宕机、断电、断网、重启等,通过在设备异常恢复后,基于日志回放继续推进未完成的分布式事务,直至完成事务提交或完成事务回滚。In this embodiment, multiple checkpoints are set in the first and second stages (to detect whether each process node in the transaction commit or rollback process is executed successfully or failed), and a write-ahead log (WAL) technology with persistence capability is used to record the transaction commit and rollback, so that after an exception occurs in a transaction, the unfinished distributed transaction can be continued to be promoted through log playback until the transaction is committed or rolled back, thereby avoiding pending transactions. Transaction exceptions are generally caused by an exception in the device where the transaction coordinator is located, or an exception in the device where the transaction participant is located, such as a device crash, power outage, network disconnection, restart, etc. After the device abnormality is recovered, the unfinished distributed transaction is continued to be promoted based on log playback until the transaction is committed or rolled back.
在第一阶段,分布式事务的预提交流程可以包括:In the first phase, the pre-commit process of a distributed transaction may include:
S4_0,分布式事务进行预提交。S4_0, distributed transaction pre-committed.
S4_1,事务协调者预写日志,记录第一阶段的事务相关信息。S4_1, the transaction coordinator writes a pre-log to record the transaction-related information of the first phase.
在一些实施例中,预写日志可进行持久化保存,第一阶段的事务相关信息可包括事务状态信息、SQL语句和操作数据。其中,事务状态信息可以包括分布式事务的XID、分布式事务的状态、各个事务分支的XA_ID及各个事务分支的状态。操作数据可以是一项或者多项,操作数据可以分布在一个分库里,也可以分布在多个分库里,操作数据可以通过表编号以及主键进行标识。SQL语句用于表示对操作数据进行何种处理。In some embodiments, the write-ahead log may be persisted, and the transaction-related information of the first phase may include transaction status information, SQL statements, and operation data. Among them, the transaction status information may include the XID of the distributed transaction, the status of the distributed transaction, the XA_ID of each transaction branch, and the status of each transaction branch. The operation data may be one or more items, and the operation data may be distributed in one sub-library or in multiple sub-libraries. The operation data may be identified by the table number and the primary key. The SQL statement is used to indicate what kind of processing is performed on the operation data.
在一些实施例中,预写日志以键值对的数据结构进行存储。图5示出了预写日志在存储设备中的组织结构。从图5所示的组织结构中可以看出,预写日志以树状图的组织结构存储。树状图包括多个根块和多个叶子块,每个根块对应一个叶子块。叶子块用于存储预写日志,以及该预写日志记录的事务(分布式事务或事务分支)的执行结果,其中执行结果为完成时,用于指示预写日志记录的事务被执行成功,执行结果为未完成时,用于指示预写日志记录的事务没有被执行成功,需要进行日志回放,便于基于回放的预写日志对没有被执行成功的事务进行重试(继续执行或者回滚)。根块用于存储叶子块的信息,例如,根块中存储叶子块中预写日志的键的取值范围以及对应的叶子块的存储地址。In some embodiments, the write-ahead log is stored in a data structure of a key-value pair. FIG. 5 shows the organizational structure of the write-ahead log in a storage device. As can be seen from the organizational structure shown in FIG. 5, the write-ahead log is stored in a tree diagram organizational structure. The tree diagram includes multiple root blocks and multiple leaf blocks, and each root block corresponds to a leaf block. The leaf block is used to store the write-ahead log, and the execution result of the transaction (distributed transaction or transaction branch) recorded in the write-ahead log, wherein when the execution result is completed, it is used to indicate that the transaction recorded in the write-ahead log is successfully executed, and when the execution result is incomplete, it is used to indicate that the transaction recorded in the write-ahead log is not successfully executed, and log playback is required, so as to facilitate the retry (continue execution or rollback) of the transaction that has not been successfully executed based on the replayed write-ahead log. The root block is used to store the information of the leaf block, for example, the value range of the key of the write-ahead log in the leaf block stored in the root block and the storage address of the corresponding leaf block.
S4_2,若事务协调者预写日志失败,分布式事务进入回滚流程。S4_2: If the transaction coordinator fails to write the pre-log, the distributed transaction enters the rollback process.
S4_3,若事务协调者预写日志成功,事务协调者记录各个事务参与者为CONN状态(也可称为连接状态),及通知各个事务参与者设置自身状态为PREPARING状态(也可称为准备中状态)。S4_3, if the transaction coordinator successfully writes the pre-log, the transaction coordinator records each transaction participant as being in the CONN state (also called the connection state), and notifies each transaction participant to set its own state to the PREPARING state (also called the preparing state).
在一些实施例中,在第一阶段,通过设置预写日志是否成功的检查点,在预写日志成功之后,各个事务参与者再进行预提交,避免后续无法基于日志回放推进异常的事务分支至COMMITED状态或ROLLBACKED状态。异常的事务分支可以是指事务分支的最终状态不是COMMITED状态或ROLLBACKED状态的事务分支。In some embodiments, in the first stage, by setting a checkpoint to determine whether the write-ahead log is successful, each transaction participant performs a pre-commit after the write-ahead log is successful, to avoid the inability to subsequently advance an abnormal transaction branch to a COMMITED state or a ROLLBACKED state based on log playback. An abnormal transaction branch may refer to a transaction branch whose final state is not a COMMITED state or a ROLLBACKED state.
S4_4,各个事务参与者执行“XA END”指令与“XA PREPARE”指令。S4_4, each transaction participant executes the "XA END" instruction and the "XA PREPARE" instruction.
S4_5,若事务参与者执行“XA END”指令与“XA PREPARE”指令成功,事务参与者设置自身状态为PREPARED状态(也可称为准备成功状态)。S4_5, if the transaction participant executes the "XA END" instruction and the "XA PREPARE" instruction successfully, the transaction participant sets its own status to the PREPARED status (also called the prepared success status).
S4_6,若事务参与者执行“XA END”指令和/或“XA PREPARE”指令失败,事务参与者进行回滚,及断开与事务协调者的网络连接。S4_6: If a transaction participant fails to execute the "XA END" instruction and/or the "XA PREPARE" instruction, the transaction participant rolls back and disconnects from the network connection with the transaction coordinator.
在一些实施例中,在事务参与者执行“XA END”指令与“XA PREPARE”指令过程中设置检查点,对于指令执行失败的事务参与者提前进行回滚。事务参与者进行回滚可以是指事务参与者执行“XA ROLLBACK”指令。In some embodiments, a checkpoint is set during the process of a transaction participant executing an "XA END" instruction and an "XA PREPARE" instruction, and a transaction participant that fails to execute the instruction is rolled back in advance. The transaction participant rolling back may refer to the transaction participant executing an "XA ROLLBACK" instruction.
在一些实施例中,事务参与者断开与事务协调者的网络连接,可以是指断开了基于网络或者进程间通信(inter process communication,IPC)机制建立的连接,而事务协
调者与事务参与者之间的SESSION连接未断开。事务参与者断开与事务协调者的网络连接之后,可释放事务参与者处理事务分支过程中使用的锁资源。In some embodiments, the transaction participant disconnects the network connection with the transaction coordinator, which may mean disconnecting the connection established based on the network or inter-process communication (IPC) mechanism, while the transaction coordinator The SESSION connection between the coordinator and the transaction participant is not disconnected. After the transaction participant disconnects the network connection with the transaction coordinator, the lock resources used by the transaction participant in processing the transaction branch can be released.
S4_7,若事务参与者回滚成功,事务参与者设置自身状态为ROLLBACKED状态(也可称为回滚成功状态),事务协调者记录与其断开连接的事务参与者为CONN_QUIT状态(也可称为断开状态)。S4_7, if the transaction participant rolls back successfully, the transaction participant sets its own state to ROLLBACKED state (also called rollback success state), and the transaction coordinator records the transaction participant disconnected from it as CONN_QUIT state (also called disconnected state).
S4_8,若事务参与者回滚失败,事务参与者设置自身状态为ROLLBACK_FAILED状态(也可称为回滚失败状态),事务协调者记录与其断开连接的事务参与者为CONN_QUIT状态。S4_8, if the transaction participant fails to roll back, the transaction participant sets its own state to ROLLBACK_FAILED state (also called rollback failure state), and the transaction coordinator records the transaction participant disconnected from it as CONN_QUIT state.
在一些实施例中,在事务参与者准备失败之后的回滚过程中设置检查点,可及时获知事务参与者的回滚状态(是回滚成功,还是回滚失败)。In some embodiments, a checkpoint is set during the rollback process after a transaction participant fails to prepare, so that the rollback status of the transaction participant (whether the rollback is successful or failed) can be known in a timely manner.
S4_9若事务协调者接收到所有事务参与者反馈处于PREPARED状态,分布式事务进入PREPARED状态。S4_9 If the transaction coordinator receives feedback from all transaction participants that they are in the PREPARED state, the distributed transaction enters the PREPARED state.
S4_10,若事务协调者未接收到所有事务参与者反馈处于PREPARED状态,分布式事务进入回滚流程。S4_10: If the transaction coordinator does not receive feedback from all transaction participants that the transaction is in the PREPARED state, the distributed transaction enters the rollback process.
在一些实施例中,通过设置检查点检测事务协调者是否接收到所有事务参与者反馈处于PREPARED状态,可及时获知事务参与者是否预提交成功。In some embodiments, by setting a checkpoint to detect whether the transaction coordinator has received feedback from all transaction participants that the transaction is in a PREPARED state, it is possible to promptly learn whether the transaction participants have successfully pre-committed.
例如,某个事务参与者宕机或者失联,将会导致事务协调者无法接收该事务参与者反馈处于PREPARED状态,即会出现事务协调者未接收到所有事务参与者反馈处于PREPARED状态的情形。For example, if a transaction participant crashes or loses connection, the transaction coordinator will not be able to receive feedback from the transaction participant that it is in the PREPARED state, that is, the transaction coordinator will not receive feedback from all transaction participants that they are in the PREPARED state.
分布式事务进入PREPARED状态之后,事务协调者可向事务参与者发出通知,使得各个事务参与者进行事务分支提交。在第二阶段,分布式事务的提交流程可以包括:After the distributed transaction enters the PREPARED state, the transaction coordinator can send a notification to the transaction participants, so that each transaction participant can commit the transaction branch. In the second stage, the submission process of the distributed transaction may include:
S4_11,分布式事务进行提交。S4_11, the distributed transaction is committed.
S4_12,事务协调者预写日志,记录第二阶段的事务相关信息。S4_12, the transaction coordinator writes a pre-log to record the transaction-related information of the second phase.
在一些实施例中,预写日志可进行持久化保存,第二阶段的事务相关信息同样可包括事务状态信息、SQL语句和操作数据。In some embodiments, the write-ahead log may be persisted, and the transaction-related information of the second phase may also include transaction status information, SQL statements, and operation data.
S4_13,若事务协调者预写日志失败,分布式事务进入回滚流程。S4_13: If the transaction coordinator fails to write the pre-log, the distributed transaction enters the rollback process.
S4_14,若事务协调者预写日志成功,各个事务参与者执行“XA COMMIT”指令。S4_14, if the transaction coordinator successfully writes the pre-log, each transaction participant executes the "XA COMMIT" instruction.
在一些实施例中,在第二阶段,通过设置预写日志是否成功的检查点,在预写日志成功之后,各个事务参与者再进行提交,避免后续无法基于日志回放推进异常的事务分支至COMMITED状态或ROLLBACKED状态。In some embodiments, in the second stage, by setting a checkpoint to determine whether the write-ahead log is successful, each transaction participant commits the transaction after the write-ahead log is successful, thereby avoiding the inability to advance the abnormal transaction branch to the COMMITED state or the ROLLBACKED state based on log playback.
S4_15,若事务参与者执行“XA COMMIT”指令成功,事务参与者设置自身状态为COMMITED状态(也可称为提交成功状态)。S4_15: If the transaction participant executes the "XA COMMIT" instruction successfully, the transaction participant sets its own status to COMMITED status (also called successful commit status).
S4_16,若事务参与者执行“XA COMMIT”指令失败,事务参与者设置自身状态为COMMIT_FAILED状态(也可称为提交失败状态)。S4_16: If a transaction participant fails to execute the "XA COMMIT" instruction, the transaction participant sets its own status to COMMIT_FAILED status (also called commit failure status).
在一些实施例中,在事务参与者执行“XA COMMIT”指令过程中设置检查点,对于指令执行失败的事务参与者设置为COMMIT_FAILED状态,可及时获知所有事务参与者是否均处于COMMITED状态。In some embodiments, a checkpoint is set during the process of a transaction participant executing an "XA COMMIT" instruction, and the transaction participant for which the instruction execution fails is set to the COMMIT_FAILED state, so that it can be known in a timely manner whether all transaction participants are in the COMMITED state.
S4_17,若事务协调者接收到所有事务参与者反馈处于COMMITED状态,分布式事务进入COMMITED状态。S4_17: If the transaction coordinator receives feedback from all transaction participants that they are in the COMMITED state, the distributed transaction enters the COMMITED state.
S4_18,若事务协调者未接收到所有事务参与者反馈处于COMMITED状态,分布式事务进入COMMIT_FAILED状态。S4_18: If the transaction coordinator does not receive feedback from all transaction participants that they are in the COMMITED state, the distributed transaction enters the COMMIT_FAILED state.
在一些实施例中,通过设置检查点检测事务协调者是否接收到所有事务参与者反馈处于COMMITED状态,可及时获知事务参与者是否提交成功。In some embodiments, by setting a checkpoint to detect whether the transaction coordinator has received feedback from all transaction participants that they are in the COMMITED state, it is possible to promptly learn whether the transaction participants have successfully committed.
S4_19,在分布式事务进入COMMIT_FAILED状态之后,进行分布式事务提交重试,即跳转至步骤S4_10重新执行一次分布式事务提交。S4_19, after the distributed transaction enters the COMMIT_FAILED state, the distributed transaction commit is retried, that is, jumping to step S4_10 to re-execute the distributed transaction commit.
S4_20,若无法再次进行分布式事务提交,或者再次进行分布式事务提交之后,分布式事务仍进入COMMIT_FAILED状态,事务协调者进行日志回放。
S4_20: If the distributed transaction cannot be committed again, or if the distributed transaction still enters the COMMIT_FAILED state after being committed again, the transaction coordinator performs log replay.
在一些实施例中,通过在分布式事务提交重试的过程中设置检查点,以确定是否进行日志回放。In some embodiments, a checkpoint is set during a distributed transaction commit retry process to determine whether to perform log replay.
在一些实施例中,对于提交异常的分布式事务,重新执行分布式事务提交的操作,直至失败次数等于预设阈值,预设阈值可以根据实际需求进行设定,本申请对此不作限定。在失败次数等于预设阈值的情况下,事务协调者进行日志回放。In some embodiments, for a distributed transaction that submits an exception, the distributed transaction submission operation is re-executed until the number of failures is equal to a preset threshold, which can be set according to actual needs and is not limited in this application. When the number of failures is equal to the preset threshold, the transaction coordinator performs log playback.
在一些实施例中,进行日志回放可以是指对存在异常的事务分支的预写日志进行回放,确定异常事务分支的失败原因(例如基于上述设置的检查点确定是哪个流程节点导致事务分支无法进入COMMITED状态或ROLLBACKED状态),从确定的流程节点开始重新进行推进,直至异常的事务分支的最终状态为COMMITED状态或ROLLBACKED状态。In some embodiments, performing log playback may refer to playing back the write-ahead log of the transaction branch with the exception, determining the cause of failure of the abnormal transaction branch (for example, determining which process node caused the transaction branch to be unable to enter the COMMITED state or the ROLLBACKED state based on the checkpoint set above), and re-starting from the determined process node until the final state of the abnormal transaction branch is the COMMITED state or the ROLLBACKED state.
如图6所示,结束XA事务的具体执行流程包括:As shown in FIG6 , the specific execution process of ending an XA transaction includes:
S6_0,各个事务参与者执行“XA END”指令。S6_0, each transaction participant executes the "XA END" instruction.
S6_1,若事务参与者执行“XA END”指令成功,事务参与者设置自身状态为ENDED状态(结束状态)。S6_1: If the transaction participant executes the "XA END" instruction successfully, the transaction participant sets its own status to ENDED status (end status).
S6_2,若事务参与者执行“XA END”指令失败,事务参与者进行回滚,及断开与事务协调者的网络连接。S6_2: If a transaction participant fails to execute the "XA END" instruction, the transaction participant rolls back and disconnects from the network connection with the transaction coordinator.
在一些实施例中,在事务参与者执行“XA END”指令过程中设置检查点,对于执行失败的事务参与者可提前进行回滚。In some embodiments, a checkpoint is set during the process of a transaction participant executing an "XA END" instruction, and a rollback can be performed in advance for transaction participants whose execution fails.
S6_3,若事务参与者回滚成功,事务参与者设置自身状态为ROLLBACKED状态,事务协调者记录与其断开连接的事务参与者为CONN_QUIT状态。S6_3, if the transaction participant rolls back successfully, the transaction participant sets its own state to ROLLBACKED state, and the transaction coordinator records the transaction participant disconnected from it as CONN_QUIT state.
S6_4,若事务参与者回滚失败,事务参与者设置自身状态为ROLLBACK_FAILED状态,事务协调者记录与其断开连接的事务参与者为CONN_QUIT状态。S6_4, if the transaction participant fails to roll back, the transaction participant sets its own state to ROLLBACK_FAILED state, and the transaction coordinator records the transaction participant disconnected from it as CONN_QUIT state.
参照图7所示,示例性的介绍本申请一实施例提供的分布式事务进行回滚的流程图。7 , a flowchart of rolling back a distributed transaction provided by an embodiment of the present application is exemplarily introduced.
S7_0,分布式事务进行回滚。S7_0, distributed transaction is rolled back.
S7_1,各事务参与者执行“XA ROLLBACK”指令。S7_1, each transaction participant executes the "XA ROLLBACK" instruction.
S7_2,若事务参与者执行“XA ROLLBACK”指令成功,事务参与者将自身状态设置为ROLLBACKED状态。S7_2: If the transaction participant executes the "XA ROLLBACK" instruction successfully, the transaction participant sets its own status to ROLLBACKED status.
S7_3,若事务参与者执行“XA ROLLBACK”指令失败,事务参与者进行回滚,及断开与事务协调者的网络连接。S7_3: If a transaction participant fails to execute the "XA ROLLBACK" instruction, the transaction participant rolls back and disconnects from the network connection with the transaction coordinator.
在一些实施例中,在事务参与者执行“XA ROLLBACK”指令过程中设置检查点,对于执行失败的事务参与者可尝试再次进行回滚。In some embodiments, a checkpoint is set during the process of a transaction participant executing an "XA ROLLBACK" instruction, and a transaction participant that fails in the execution can attempt to roll back again.
S7_4,若事务参与者回滚成功,事务参与者设置自身状态为ROLLBACKED状态,事务协调者记录与其断开连接的事务参与者为CONN_QUIT状态。S7_4: If the transaction participant rolls back successfully, the transaction participant sets its own status to ROLLBACKED, and the transaction coordinator records the disconnected transaction participant as CONN_QUIT.
S7_5,若事务参与者回滚失败,事务参与者设置自身状态为ROLLBACK_FAILED状态,事务协调者记录与其断开连接的事务参与者为CONN_QUIT状态。S7_5: If the transaction participant fails to roll back, the transaction participant sets its own status to ROLLBACK_FAILED, and the transaction coordinator records the disconnected transaction participant as CONN_QUIT.
S7_6,若事务协调者接收到所有事务参与者反馈处于ROLLBACKED状态,分布式事务进入ROLLBACKED状态。S7_6: If the transaction coordinator receives feedback from all transaction participants that the transaction is in the ROLLBACKED state, the distributed transaction enters the ROLLBACKED state.
S7_7,若事务协调者未接收到所有事务参与者反馈处于ROLLBACKED状态,进行分布式事务回滚重试,即跳转至步骤S7_0重新执行一次分布式事务回滚。S7_7, if the transaction coordinator does not receive feedback from all transaction participants that the transaction is in the ROLLBACKED state, the distributed transaction is rolled back and retried, that is, the process jumps to step S7_0 and re-executes the distributed transaction rollback.
S7_8,若无法再次进行分布式事务回滚,或者再次进行分布式事务回滚之后,所有事务参与者仍未全部处于ROLLBACKED状态,事务协调者断开与各事务参与者的SESSION连接(也可称为会话连接),及将处于ROLLBACK_FAILED状态的事务参与者信息写入日志。S7_8: If the distributed transaction cannot be rolled back again, or after the distributed transaction is rolled back again, all transaction participants are still not in the ROLLBACKED state, the transaction coordinator disconnects the SESSION connection (also called session connection) with each transaction participant, and writes the transaction participant information in the ROLLBACK_FAILED state into the log.
S7_9,事务协调者进行日志回放。S7_9, the transaction coordinator plays back the log.
在一些实施例中,事务协调者进行日志回放,确定异常的事务分支在回滚过程中的失败的原因(例如基于上述设置的检查点确定回滚过程中的哪个流程节点导致回滚失
败),从确定的流程节点开始重新进行回滚,直至异常的事务分支的最终状态为ROLLBACKED状态。In some embodiments, the transaction coordinator performs log playback to determine the cause of the failure of the abnormal transaction branch during the rollback process (for example, based on the checkpoints set above, determine which process node in the rollback process caused the rollback failure). If the transaction fails, the rollback is performed again from the determined process node until the final state of the abnormal transaction branch is ROLLBACKED.
参照图8所示,为本申请一实施例提供的一种事务处理方法的步骤流程图。根据不同的需求,所述流程图中步骤的顺序可以改变,某些步骤可以省略。本实施例中,事务处理方法可以包括:Referring to FIG. 8 , a flowchart of the steps of a transaction processing method provided in an embodiment of the present application is shown. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted. In this embodiment, the transaction processing method may include:
步骤S81,响应于接收到的操作指令,为操作指令创建分布式事务,分布式事务包括多个事务分支。Step S81, in response to the received operation instruction, creating a distributed transaction for the operation instruction, the distributed transaction including multiple transaction branches.
在一些实施例中,若操作指令处理的数据涉及多个分库或者多个分表,表明存在多个事务参与者,一个分库或者一个分表可作为一个事务参与者,通过为操作指令创建分布式事务,实现对位于多个分库或者多个分表内的数据进行一致性处理。In some embodiments, if the data processed by the operation instruction involves multiple sub-libraries or multiple sub-tables, it indicates that there are multiple transaction participants. A sub-library or a sub-table can serve as a transaction participant. By creating a distributed transaction for the operation instruction, consistency processing of the data located in multiple sub-libraries or multiple sub-tables can be achieved.
步骤S82,基于预设事务提交机制提交分布式事务,及生成分布式事务的预写日志,多个事务分支中的每个事务分支的提交包括多个流程节点,预设事务提交机制包括用于检测多个流程节点的执行情况及每个事务分支的事务状态的检查点。Step S82, committing the distributed transaction based on the preset transaction commit mechanism, and generating a pre-written log of the distributed transaction, the commit of each transaction branch in the multiple transaction branches includes multiple process nodes, and the preset transaction commit mechanism includes a checkpoint for detecting the execution status of the multiple process nodes and the transaction status of each transaction branch.
在一些实施例中,以预设事务提交机制为2PC为例,可在每个事务分支进行预提交之前,生成第一预写日志,通过第一预写日志记录两阶段提交协议中的第一阶段的事务相关信息;在每个事务分支进行提交之前,生成第二预写日志,通过第二预写日志记录两阶段提交协议中的第二阶段的事务相关信息,便于在出现异常的事务分支时,可基于日志回放对异常的事务分支从第一阶段或第二阶段进行继续推进。In some embodiments, taking the preset transaction commit mechanism as 2PC as an example, before each transaction branch is pre-committed, a first write-ahead log may be generated, and the transaction-related information of the first phase in the two-phase commit protocol may be recorded through the first write-ahead log; before each transaction branch is committed, a second write-ahead log may be generated, and the transaction-related information of the second phase in the two-phase commit protocol may be recorded through the second write-ahead log, so that when an abnormal transaction branch occurs, the abnormal transaction branch can be continued from the first phase or the second phase based on log playback.
在一些实施例中,第一阶段的事务相关信息及第二阶段的事务相关信息均可包括分布式事务的事务状态信息、事务分支的事务状态信息、操作数据和用于对所述操作数据进行处理的SQL语句。In some embodiments, the transaction-related information of the first phase and the transaction-related information of the second phase may include transaction status information of the distributed transaction, transaction status information of the transaction branch, operation data, and SQL statements for processing the operation data.
在一些实施例中,若第一预写日志或第二预写日志生成失败,表明无法通过预写日志对事务的第一阶段或第二阶段进行记录,可直接控制每个事务分支进行回滚,避免出现后续无法基于日志回放对异常的事务分支从第一阶段或第二阶段进行继续推进的问题。In some embodiments, if the generation of the first write-ahead log or the second write-ahead log fails, indicating that the first phase or the second phase of the transaction cannot be recorded through the write-ahead log, each transaction branch can be directly controlled to be rolled back, avoiding the problem that the abnormal transaction branch cannot be continued from the first phase or the second phase based on log playback.
在一些实施例中,每个事务分支的提交包括提交事务流程或回滚事务流程,提交事务流程或回滚事务流程均包括多个流程节点,多个流程节点中的每个流程节点均设置有检查点,进而对于异常的事务分支,基于日志回放可从指定的流程节点重新进行事务提交或者事务回滚。异常的事务分支可以是指未处于预设事务状态的事务分支,例如某一事务分支未处于Rollbacked状态或Committed状态,可认为该事务分支为异常的事务分支。In some embodiments, the submission of each transaction branch includes submitting a transaction process or rolling back a transaction process, and the submitting transaction process or rolling back a transaction process includes multiple process nodes, and each of the multiple process nodes is provided with a checkpoint, so that for an abnormal transaction branch, the transaction can be resubmitted or rolled back from a specified process node based on log playback. An abnormal transaction branch may refer to a transaction branch that is not in a preset transaction state. For example, if a transaction branch is not in a Rollbacked state or a Committed state, the transaction branch may be considered an abnormal transaction branch.
步骤S83,若确定分布式事务包含异常的事务分支,基于预写日志及检查点对异常的事务分支进行处理,异常的事务分支由事务协调者所在的设备发生异常,或者事务参与者所在的设备发生异常引起。Step S83: If it is determined that the distributed transaction contains an abnormal transaction branch, the abnormal transaction branch is processed based on the write-ahead log and the checkpoint. The abnormal transaction branch is caused by an abnormality in the device where the transaction coordinator is located, or an abnormality in the device where the transaction participant is located.
例如,事务协调者所在的设备在某个时间节点发生异常,导致各个事务参与者无法接收到事务协调者的指令,或者某些事务参与者还未来得及接收到事务协调者的指令,导致无法顺利执行提交事务流程或回滚事务流程,进而可能产生一个或多个异常的事务分支;或者,由于某个事务参与者所在的设备发生异常,该事务参与者无法接收到事务协调者的指令,或者在接收到事务协调者的指令后无法顺利执行提交事务流程或回滚事务流程,进而产生异常的事务分支。在设备异常恢复后,通过回放与异常的事务分支对应的预写日志,确定重新处理异常的事务分支的流程节点,以确定的流程节点为起始流程节点对异常的事务分支进行处理,实现从指定的流程节点对异常的事务分支重新进行事务推进。例如,异常的事务分支的事务参与者可发送一个重新进行事务推进的请求给事务协调者,由事务协调者回放与异常的事务分支对应的预写日志,确定重新处理异常的事务分支的流程节点,通知事务参与者(异常的事务分支的事务参与者)以确定的流程节点为起始流程节点对异常的事务分支重新进行事务推进(事务提交或回滚)。
For example, an abnormality occurs in the device where the transaction coordinator is located at a certain time point, resulting in the inability of each transaction participant to receive the instruction of the transaction coordinator, or some transaction participants have not yet received the instruction of the transaction coordinator in time, resulting in the inability to smoothly execute the transaction submission process or rollback transaction process, which may generate one or more abnormal transaction branches; or, due to an abnormality in the device where a transaction participant is located, the transaction participant cannot receive the instruction of the transaction coordinator, or cannot smoothly execute the transaction submission process or rollback transaction process after receiving the instruction of the transaction coordinator, thereby generating an abnormal transaction branch. After the device abnormality is recovered, the pre-written log corresponding to the abnormal transaction branch is played back to determine the process node for reprocessing the abnormal transaction branch, and the abnormal transaction branch is processed with the determined process node as the starting process node, so as to realize the re-advancement of the transaction for the abnormal transaction branch from the specified process node. For example, a transaction participant of an abnormal transaction branch may send a request to re-advance the transaction to the transaction coordinator, which will replay the write-ahead log corresponding to the abnormal transaction branch, determine the process node for reprocessing the abnormal transaction branch, and notify the transaction participant (transaction participant of the abnormal transaction branch) to re-advance the transaction (commit or roll back the transaction) for the abnormal transaction branch using the determined process node as the starting process node.
如图9所示,为本申请一实施例提供一种分布式中间件30。分布式中间件30可以包括创建模块301、提交模块302及处理模块303。As shown in FIG9 , a distributed middleware 30 is provided for an embodiment of the present application. The distributed middleware 30 may include a creation module 301 , a submission module 302 and a processing module 303 .
创建模块301,用于响应于接收到的操作指令,为操作指令创建分布式事务,分布式事务包括多个事务分支。The creation module 301 is used to create a distributed transaction for the operation instruction in response to the received operation instruction, and the distributed transaction includes multiple transaction branches.
提交模块302,用于基于预设事务提交机制提交分布式事务,及生成分布式事务的预写日志,多个事务分支中的每个事务分支的提交包括多个流程节点,预设事务提交机制包括用于检测多个流程节点的执行情况及每个事务分支的事务状态的检查点。The submission module 302 is used to submit distributed transactions based on a preset transaction submission mechanism and generate a pre-written log of the distributed transaction. The submission of each transaction branch in the multiple transaction branches includes multiple process nodes. The preset transaction submission mechanism includes a checkpoint for detecting the execution status of the multiple process nodes and the transaction status of each transaction branch.
处理模块303,用于在确定分布式事务包含异常的事务分支,基于预写日志及检查点对异常的事务分支进行处理,异常的事务分支由分布式事务的事务协调者所在的设备发生异常,或者分布式事务的事务参与所在的设备发生异常引起。The processing module 303 is used to process the abnormal transaction branch based on the write-ahead log and the checkpoint when it is determined that the distributed transaction contains an abnormal transaction branch. The abnormal transaction branch is caused by an abnormality in the device where the transaction coordinator of the distributed transaction is located, or an abnormality in the device where the transaction participant of the distributed transaction is located.
其中,创建模块301、提交模块302及处理模块303均可以通过软件实现,或者可以通过硬件实现。示例性的,接下来以创建模块301为例,介绍创建模块301的实现方式。类似的,提交模块302及处理模块303的实现方式可以参考创建模块301的实现方式。Among them, the creation module 301, the submission module 302 and the processing module 303 can all be implemented by software, or can be implemented by hardware. Exemplarily, the creation module 301 is taken as an example to introduce the implementation of the creation module 301. Similarly, the implementation of the submission module 302 and the processing module 303 can refer to the implementation of the creation module 301.
模块作为软件功能单元的一种举例,创建模块301可以包括运行在计算实例上的代码。其中,计算实例可以包括物理主机(计算设备)、虚拟机、容器中的至少一种。进一步地,上述计算实例可以是一台或者多台。例如,创建模块301可以包括运行在多个主机/虚拟机/容器上的代码。需要说明的是,用于运行该代码的多个主机/虚拟机/容器可以分布在相同的区域(region)中,也可以分布在不同的region中。进一步地,用于运行该代码的多个主机/虚拟机/容器可以分布在相同的可用区(availability zone,AZ)中,也可以分布在不同的AZ中,每个AZ包括一个数据中心或多个地理位置相近的数据中心。其中,通常一个region可以包括多个AZ。As an example of a software functional unit, the creation module 301 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the creation module 301 may include code running on multiple hosts/virtual machines/containers. It should be noted that the multiple hosts/virtual machines/containers used to run the code can be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts/virtual machines/containers used to run the code can be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple data centers with close geographical locations. Among them, usually a region can include multiple AZs.
同样,用于运行该代码的多个主机/虚拟机/容器可以分布在同一个虚拟私有云(virtual private cloud,VPC)中,也可以分布在多个VPC中。其中,通常一个VPC设置在一个region内,同一region内两个VPC之间,以及不同region的VPC之间跨区通信需在每个VPC内设置通信网关,经通信网关实现VPC之间的互连。Similarly, multiple hosts/virtual machines/containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.
模块作为硬件功能单元的一种举例,创建模块301可以包括至少一个计算设备,如服务器等。或者,创建模块301也可以是利用专用集成电路(application-specific integrated circuit,ASIC)实现、或可编程逻辑器件(programmable logic device,PLD)实现的设备等。其中,上述PLD可以是复杂程序逻辑器件(complex programmable logical device,CPLD)、现场可编程门阵列(field-programmable gate array,FPGA)、通用阵列逻辑(generic array logic,GAL)或其任意组合实现。As an example of a hardware functional unit, the creation module 301 may include at least one computing device, such as a server, etc. Alternatively, the creation module 301 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.
创建模块301包括的多个计算设备可以分布在相同的region中,也可以分布在不同的region中。创建模块301包括的多个计算设备可以分布在相同的AZ中,也可以分布在不同的AZ中。同样,创建模块301包括的多个计算设备可以分布在同一个VPC中,也可以分布在多个VPC中。其中,所述多个计算设备可以是服务器、ASIC、PLD、CPLD、FPGA和GAL等计算设备的任意组合。The multiple computing devices included in the creation module 301 can be distributed in the same region or in different regions. The multiple computing devices included in the creation module 301 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the creation module 301 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.
需要说明的是,在其他实施例中,创建模块301可以用于执行事务处理方法中的任意步骤,提交模块302可以用于执行事务处理方法中的任意步骤,处理模块303可以用于执行事务处理方法中的任意步骤,创建模块301、提交模块302及处理模块303负责实现的步骤可根据需要指定,通过创建模块301、提交模块302及处理模块303分别实现事务处理方法中不同的步骤来实现决策装置30的全部功能。It should be noted that, in other embodiments, the creation module 301 can be used to execute any step in the transaction processing method, the submission module 302 can be used to execute any step in the transaction processing method, and the processing module 303 can be used to execute any step in the transaction processing method. The steps that the creation module 301, the submission module 302 and the processing module 303 are responsible for implementing can be specified as needed. The creation module 301, the submission module 302 and the processing module 303 respectively implement different steps in the transaction processing method to achieve all the functions of the decision-making device 30.
如图10所示,为本申请实施例提供的一种计算设备100,计算设备100包括:总线102、处理器104、存储器106和通信接口108。处理器104、存储器106和通信接口108之间通过总线102通信。计算设备100可以是服务器或终端设备。应理解,本申请不限定计算设备100中的处理器、存储器的个数。As shown in FIG10 , a computing device 100 is provided in an embodiment of the present application, and the computing device 100 includes: a bus 102, a processor 104, a memory 106, and a communication interface 108. The processor 104, the memory 106, and the communication interface 108 communicate with each other through the bus 102. The computing device 100 may be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 100.
总线102可以是外设部件互连标准(peripheral component interconnect,PCI)总线或扩展工业标准结构(extended industry standard architecture,EISA)总线等。总
线可以分为地址总线、数据总线、控制总线等。为便于表示,图10中仅用一条线表示,但并不表示仅有一根总线或一种类型的总线。总线104可包括在计算设备100各个部件(例如,存储器106、处理器104、通信接口108)之间传送信息的通路。The bus 102 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The lines can be divided into address bus, data bus, control bus, etc. For ease of representation, only one line is used in FIG10, but it does not mean that there is only one bus or one type of bus. Bus 104 may include a path for transmitting information between various components of computing device 100 (e.g., memory 106, processor 104, communication interface 108).
处理器104可以包括中央处理器(central processing unit,CPU)、图形处理器(graphics processing unit,GPU)、微处理器(micro processor,MP)或者数字信号处理器(digital signal processor,DSP)等处理器中的任意一种或多种。The processor 104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP) or a digital signal processor (DSP).
存储器106可以包括易失性存储器(volatile memory),例如随机存取存储器(random access memory,RAM)。处理器104还可以包括非易失性存储器(non-volatile memory),例如只读存储器(read-only memory,ROM),快闪存储器,机械硬盘(hard disk drive,HDD)或固态硬盘(solid state drive,SSD)。The memory 106 may include a volatile memory, such as a random access memory (RAM). The processor 104 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).
存储器106中存储有可执行的程序代码,处理器104执行该可执行的程序代码以分别实现前述创建模块301、提交模块302及处理模块303的功能,从而实现事务处理方法。也即,存储器106上存有用于执行事务处理方法的指令。在图10中以处理器104执行该可执行的程序代码以分别实现前述创建模块301、提交模块302及处理模块303的功能为例。The memory 106 stores executable program codes, and the processor 104 executes the executable program codes to respectively implement the functions of the aforementioned creation module 301, submission module 302, and processing module 303, thereby implementing the transaction processing method. That is, the memory 106 stores instructions for executing the transaction processing method. In FIG. 10 , the processor 104 executes the executable program codes to respectively implement the functions of the aforementioned creation module 301, submission module 302, and processing module 303 as an example.
通信接口108使用例如但不限于网络接口卡、收发器一类的收发模块,来实现计算设备100与其他设备或通信网络之间的通信。The communication interface 108 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 100 and other devices or a communication network.
本申请实施例还提供了一种计算设备集群10。该计算设备集群10包括至少一台计算设备。该计算设备可以是用于进行数据存储的服务器,或者是本地数据中心中的本地服务器。在一些实施例中,计算设备也可以是台式机、笔记本电脑或者智能手机等终端设备。The embodiment of the present application also provides a computing device cluster 10. The computing device cluster 10 includes at least one computing device. The computing device may be a server for data storage, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.
如图11所示,计算设备集群10包括至少一个计算设备100。计算设备集群10中的一个或多个计算设备100中的存储器106中可以存有相同的用于执行事务处理方法的指令。As shown in Fig. 11, the computing device cluster 10 includes at least one computing device 100. The memory 106 in one or more computing devices 100 in the computing device cluster 10 may store the same instructions for executing the transaction processing method.
在一些可能的实现方式中,该计算设备集群10中的一个或多个计算设备100的存储器106中也可以分别存有用于执行事务处理方法的部分指令。换言之,一个或多个计算设备100的组合可以共同执行用于执行事务处理方法的指令。In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster 10 may also store partial instructions for executing the transaction processing method. In other words, the combination of one or more computing devices 100 may jointly execute instructions for executing the transaction processing method.
需要说明的是,计算设备集群10中的不同的计算设备100中的存储器106可以存储不同的指令,分别用于执行数据访问装置的部分功能。也即,不同的计算设备100中的存储器106存储的指令可以实现创建模块301、提交模块302及处理模块303中的一个或多个模块的功能。It should be noted that the memory 106 in different computing devices 100 in the computing device cluster 10 can store different instructions, which are respectively used to execute part of the functions of the data access device. That is, the instructions stored in the memory 106 in different computing devices 100 can implement the functions of one or more modules among the creation module 301, the submission module 302 and the processing module 303.
在一些可能的实现方式中,计算设备集群10中的一个或多个计算设备可以通过网络连接。其中,所述网络可以是广域网或局域网等等。图11示出了一种可能的实现方式。如图11所示,两个计算设备100A和100B之间通过网络进行连接,以实现创建模块301、提交模块302及处理模块303中的一个或多个模块的功能为例。具体地,通过各个计算设备中的通信接口与所述网络进行连接。在这一类可能的实现方式中,计算设备100A中的存储器106中存有执行创建模块301及提交模块302的功能的指令。同时,计算设备100B中的存储器106中存有执行处理模块303的功能的指令。In some possible implementations, one or more computing devices in the computing device cluster 10 can be connected via a network. The network may be a wide area network or a local area network, etc. FIG. 11 shows a possible implementation. As shown in FIG. 11 , two computing devices 100A and 100B are connected via a network, taking the functions of one or more modules in the creation module 301, the submission module 302, and the processing module 303 as an example. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 106 in the computing device 100A stores instructions for executing the functions of the creation module 301 and the submission module 302. At the same time, the memory 106 in the computing device 100B stores instructions for executing the functions of the processing module 303.
图11所示的计算设备集群10之间的连接方式可以是考虑到本申请提供的事务处理方法需要大量地存储数据和读取数据,因此考虑将处理模块303实现的功能交由计算设备100B执行。The connection method between the computing device cluster 10 shown in Figure 11 can be considered to be that the transaction processing method provided in this application requires a large amount of data storage and reading, so it is considered to hand over the functions implemented by the processing module 303 to the computing device 100B for execution.
应理解,图11中示出的计算设备100A的功能也可以由多个计算设备100完成。同样,计算设备100B的功能也可以由多个计算设备100完成。It should be understood that the functions of the computing device 100A shown in FIG11 may also be completed by multiple computing devices 100. Similarly, the functions of the computing device 100B may also be completed by multiple computing devices 100.
本申请实施例还提供了一种包含指令的计算机程序产品。所述计算机程序产品可以是包含指令的,能够运行在计算设备上或被储存在任何可用介质中的软件或程序产品。当所述计算机程序产品在至少一个计算设备上运行时,使得至少一个计算设备执行事务处理方法。
The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes a transaction processing method.
本申请实施例还提供了一种计算机可读存储介质。所述计算机可读存储介质可以是计算设备能够存储的任何可用介质或者是包含一个或多个可用介质的数据中心等数据存储设备。所述可用介质可以是磁性介质,(例如,软盘、硬盘、磁带)、光介质(例如,DVD)、或者半导体介质(例如固态硬盘)等。该计算机可读存储介质包括指令,所述指令指示计算设备执行事务处理方法。The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the transaction processing method.
通过以上的实施方式的描述,所属领域的技术人员可以清楚地了解到,为描述的方便和简洁,仅以上述各功能模块的划分进行举例说明,实际应用中,可以根据需要而将上述功能分配由不同的功能模块完成,即将装置的内部结构划分成不同的功能模块,以完成以上描述的全部或者部分功能。Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.
在本申请所提供的几个实施例中,应所述理解到,所揭露的装置和方法,可以通过其它的方式实现。例如,以上所描述的装置实施例是示意性的,例如,所述模块或单元的划分,为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个装置,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,装置或单元的间接耦合或通信连接,可以是电性,机械或其它的形式。In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are schematic. For example, the division of the modules or units is a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是一个物理单元或多个物理单元,即可以位于一个地方,或者也可以分布到多个不同地方。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
另外,在本申请各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个可读取存储介质中。基于这样的理解,本申请实施例的技术方案本质上或者说对现有技术做出贡献的部分或者所述技术方案的全部或部分可以以软件产品的形式体现出来,所述软件产品存储在一个存储介质中,包括若干指令用以使得一个设备(可以是单片机,芯片等)或处理器执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器、随机存取存储器、磁碟或者光盘等各种可以存储程序代码的介质。If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the software product is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk.
以上所述,仅为本申请的具体实施方式,但本申请的保护范围并不局限于此,任何在本申请揭露的技术范围内的变化或替换,都应涵盖在本申请的保护范围之内。
The above description is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application.
Claims (11)
- 一种事务处理方法,其特征在于,所述方法包括:A transaction processing method, characterized in that the method comprises:响应于接收到的操作指令,为所述操作指令创建分布式事务,所述分布式事务包括多个事务分支;In response to the received operation instruction, create a distributed transaction for the operation instruction, wherein the distributed transaction includes multiple transaction branches;基于预设事务提交机制提交所述分布式事务,及生成所述分布式事务的预写日志,所述多个事务分支中的每个事务分支的提交包括多个流程节点,所述预设事务提交机制包括用于检测所述多个流程节点的执行情况及所述每个事务分支的事务状态的检查点;Submitting the distributed transaction based on a preset transaction submission mechanism, and generating a write-ahead log of the distributed transaction, wherein the submission of each of the multiple transaction branches includes multiple process nodes, and the preset transaction submission mechanism includes a checkpoint for detecting the execution status of the multiple process nodes and the transaction status of each transaction branch;若确定所述分布式事务包含异常的事务分支,基于所述预写日志及所述检查点对所述异常的事务分支进行处理,所述异常的事务分支由所述分布式事务的事务协调者所在的设备发生异常,或者所述分布式事务的事务参与者所在的设备发生异常引起。If it is determined that the distributed transaction includes an abnormal transaction branch, the abnormal transaction branch is processed based on the write-ahead log and the checkpoint, and the abnormal transaction branch is caused by an abnormality in a device where a transaction coordinator of the distributed transaction is located, or an abnormality in a device where a transaction participant of the distributed transaction is located.
- 如权利要求1所述的事务处理方法,其特征在于,所述为所述操作指令创建分布式事务,包括:The transaction processing method according to claim 1, wherein creating a distributed transaction for the operation instruction comprises:若所述操作指令处理的数据涉及多个分库或者多个分表,为所述操作指令创建分布式事务。If the data processed by the operation instruction involves multiple sub-databases or multiple sub-tables, a distributed transaction is created for the operation instruction.
- 如权利要求1或2所述的事务处理方法,其特征在于,所述预设事务提交机制为两阶段提交协议,所述生成所述分布式事务的预写日志,包括:The transaction processing method according to claim 1 or 2, characterized in that the preset transaction commit mechanism is a two-phase commit protocol, and the generating of the write-ahead log of the distributed transaction comprises:在所述每个事务分支进行预提交之前,生成第一预写日志,所述第一预写日志用于记录所述两阶段提交协议中的第一阶段的事务相关信息;Before each transaction branch is pre-committed, a first write-ahead log is generated, where the first write-ahead log is used to record transaction-related information of the first phase in the two-phase commit protocol;在所述每个事务分支进行提交之前,生成第二预写日志,所述第二预写日志用于记录所述两阶段提交协议中的第二阶段的事务相关信息;所述第一阶段的事务相关信息及所述第二阶段的事务相关信息均包括分布式事务的事务状态信息、事务分支的事务状态信息、操作数据和用于对所述操作数据进行处理的结构化查询语言SQL语句。Before each transaction branch is committed, a second write-ahead log is generated, where the second write-ahead log is used to record transaction-related information of the second phase in the two-phase commit protocol; the transaction-related information of the first phase and the transaction-related information of the second phase both include transaction status information of the distributed transaction, transaction status information of the transaction branch, operation data, and a structured query language SQL statement for processing the operation data.
- 如权利要求3所述的事务处理方法,其特征在于,所述方法还包括:The transaction processing method according to claim 3, characterized in that the method further comprises:若所述第一预写日志或所述第二预写日志生成失败,控制所述每个事务分支进行回滚。If the generation of the first write-ahead log or the second write-ahead log fails, each transaction branch is controlled to be rolled back.
- 如权利要求1至4中任意一项所述的事务处理方法,其特征在于,所述每个事务分支的提交包括提交事务流程或回滚事务流程,所述提交事务流程或所述回滚事务流程均包括多个流程节点,所述多个流程节点中的每个流程节点均设置有所述检查点。The transaction processing method according to any one of claims 1 to 4 is characterized in that the submission of each transaction branch includes submitting a transaction process or rolling back a transaction process, and the submitting transaction process or the rolling back transaction process both include multiple process nodes, and each of the multiple process nodes is provided with the checkpoint.
- 如权利要求5所述的事务处理方法,其特征在于,所述基于预设事务提交机制提交所述分布式事务之后,还包括:The transaction processing method according to claim 5, characterized in that after submitting the distributed transaction based on the preset transaction submission mechanism, it also includes:将未处于预设事务状态的事务分支确定为所述异常的事务分支,所述预设事务状态包括Rollbacked状态或Committed状态。A transaction branch that is not in a preset transaction state is determined as the abnormal transaction branch, where the preset transaction state includes a Rollbacked state or a Committed state.
- 如权利要求6所述的事务处理方法,其特征在于,所述基于所述预写日志及所述检查点对所述异常的事务分支进行处理,包括:The transaction processing method according to claim 6, wherein the processing of the abnormal transaction branch based on the write-ahead log and the checkpoint comprises:回放与所述异常的事务分支对应的预写日志,确定重新处理所述异常的事务分支的流程节点;Replaying the write-ahead log corresponding to the abnormal transaction branch, and determining a process node for reprocessing the abnormal transaction branch;以确定的流程节点为起始流程节点对所述异常的事务分支进行处理。The abnormal transaction branch is processed using the determined process node as the starting process node.
- 一种分布式中间件,其特征在于,包括:A distributed middleware, characterized by comprising:创建模块,用于响应于接收到的操作指令,为所述操作指令创建分布式事务,所述分布式事务包括多个事务分支;A creation module, configured to create a distributed transaction for the operation instruction in response to the received operation instruction, wherein the distributed transaction includes a plurality of transaction branches;提交模块,用于基于预设事务提交机制提交所述分布式事务,及生成所述分布式事务的预写日志,所述多个事务分支中的每个事务分支的提交包括多个流程节点,所述预设事务提交机制包括用于检测多个流程节点的执行情况及所述每个事务分支的事务状态的检查点;A commit module, configured to commit the distributed transaction based on a preset transaction commit mechanism, and generate a write-ahead log of the distributed transaction, wherein the commit of each of the multiple transaction branches includes multiple process nodes, and the preset transaction commit mechanism includes a checkpoint for detecting the execution status of the multiple process nodes and the transaction status of each transaction branch;处理模块,用于在确定所述分布式事务包含异常的事务分支时,基于所述预写日志及所述检查点对所述异常的事务分支进行处理,所述异常的事务分支由所述分布式事务的 事务协调者所在的设备发生异常,或者所述分布式事务的事务参与所在的设备发生异常引起。a processing module, configured to process the abnormal transaction branch based on the write-ahead log and the checkpoint when determining that the distributed transaction includes an abnormal transaction branch, wherein the abnormal transaction branch is processed by the distributed transaction The device where the transaction coordinator is located has an exception, or the device where the transaction participant of the distributed transaction is located has an exception.
- 一种计算设备集群,其特征在于,包括至少一个计算设备,每个计算设备包括处理器和存储器;A computing device cluster, characterized in that it includes at least one computing device, each computing device includes a processor and a memory;所述至少一个计算设备的处理器用于执行所述至少一个计算设备的存储器中存储的指令,以使得所述计算设备集群执行如权利要求1至权利要求7中任一项所述的事务处理方法。The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the transaction processing method according to any one of claims 1 to 7.
- 一种包含指令的计算机程序产品,其特征在于,当所述指令被计算设备集群运行时,使得所述计算设备集群执行如权利要求1至权利要求7中任一项所述的事务处理方法。A computer program product comprising instructions, characterized in that when the instructions are executed by a computing device cluster, the computing device cluster executes the transaction processing method as described in any one of claims 1 to 7.
- 一种计算机可读存储介质,其特征在于,包括计算机程序指令,当所述计算机程序指令由计算设备集群执行时,所述计算设备集群执行如权利要求1至权利要求7中任一项所述的事务处理方法。 A computer-readable storage medium, characterized in that it includes computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the transaction processing method as described in any one of claims 1 to claim 7.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310267376.8A CN118708301A (en) | 2023-03-17 | 2023-03-17 | Transaction processing method and related equipment |
CN202310267376.8 | 2023-03-17 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2024193138A1 true WO2024193138A1 (en) | 2024-09-26 |
Family
ID=92806025
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2023/139803 WO2024193138A1 (en) | 2023-03-17 | 2023-12-19 | Transaction processing method and related device |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN118708301A (en) |
WO (1) | WO2024193138A1 (en) |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160342616A1 (en) * | 2015-05-19 | 2016-11-24 | Vmware, Inc. | Distributed transactions with redo-only write-ahead log |
CN111651243A (en) * | 2020-05-29 | 2020-09-11 | 北京东方通科技股份有限公司 | Distributed transaction guarantee method oriented to memory data grid |
CN112470130A (en) * | 2018-07-25 | 2021-03-09 | 微软技术许可有限责任公司 | Constant time database recovery |
CN114528073A (en) * | 2022-01-24 | 2022-05-24 | 赞同科技股份有限公司 | Distributed transaction processing method, device and medium based on independent transaction coordinator |
-
2023
- 2023-03-17 CN CN202310267376.8A patent/CN118708301A/en active Pending
- 2023-12-19 WO PCT/CN2023/139803 patent/WO2024193138A1/en unknown
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20160342616A1 (en) * | 2015-05-19 | 2016-11-24 | Vmware, Inc. | Distributed transactions with redo-only write-ahead log |
CN112470130A (en) * | 2018-07-25 | 2021-03-09 | 微软技术许可有限责任公司 | Constant time database recovery |
CN111651243A (en) * | 2020-05-29 | 2020-09-11 | 北京东方通科技股份有限公司 | Distributed transaction guarantee method oriented to memory data grid |
CN114528073A (en) * | 2022-01-24 | 2022-05-24 | 赞同科技股份有限公司 | Distributed transaction processing method, device and medium based on independent transaction coordinator |
Also Published As
Publication number | Publication date |
---|---|
CN118708301A (en) | 2024-09-27 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10942823B2 (en) | Transaction processing system, recovery subsystem and method for operating a recovery subsystem | |
US11010262B2 (en) | Database system recovery using preliminary and final slave node replay positions | |
CN107077382B (en) | System and method for transaction recovery in a multi-tenant application server environment | |
US8898109B2 (en) | Automatic transaction retry after session failure | |
KR102072726B1 (en) | Systems and methods for supporting inline delegation of middle-tier transaction logs to database | |
US20130110781A1 (en) | Server replication and transaction commitment | |
KR101993432B1 (en) | Systems and methods for supporting transaction recovery based on a strict ordering of two-phase commit calls | |
WO2019109854A1 (en) | Data processing method and device for distributed database, storage medium, and electronic device | |
CN111753013A (en) | Distributed transaction processing method and device | |
CN115794499B (en) | Method and system for dual-activity replication data among distributed block storage clusters | |
US9348841B2 (en) | Transaction processing method and system | |
US20240211488A1 (en) | Transaction commitment systems, methods, and apparatuses based on distributed database systems | |
US20230376479A1 (en) | Schema and data modification concurrency in query processing pushdown | |
US11003550B2 (en) | Methods and systems of operating a database management system DBMS in a strong consistency mode | |
CN105938446B (en) | The data supported based on RDMA and hardware transactional memory replicate fault-tolerance approach | |
US11797523B2 (en) | Schema and data modification concurrency in query processing pushdown | |
US11522966B2 (en) | Methods, devices and systems for non-disruptive upgrades to a replicated state machine in a distributed computing environment | |
WO2024193138A1 (en) | Transaction processing method and related device | |
WO2023103340A1 (en) | Block data committing method and apparatus | |
CN114816682A (en) | Distributed transaction processing method, system and device | |
US6539434B1 (en) | UOWE's retry process in shared queues environment | |
WO2023193495A1 (en) | Method for processing read request, distributed database and server | |
WO2024199464A1 (en) | Database system and data management method therefor | |
WO2024159742A1 (en) | Database task processing method and apparatus, and database system and disaster recovery system | |
RU2714602C1 (en) | Method and system for data processing |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 23928442 Country of ref document: EP Kind code of ref document: A1 |