Cross-engine transactions #1803

kostja · 2016-09-30T09:01:15Z

Allow to mix Vinyl and MemTX operations in a single transactions.

dyu · 2016-09-30T09:33:10Z

👍

rohitjoshi · 2016-11-03T11:20:23Z

:1

There's no need in using a list for storing registered engines because we never unregister engines. Using an array is more convenient since it allows to quickly find an engine by id, which will be useful when we start implementing cross-engine transactions. Needed for tarantool#1803 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring

@TarantoolBot

Both memtx and vinyl support MVCC. Moreover, the Tarantool transaction manager already allows to mix statements for more than one engine in one transaction, see commit 09bfdd4 ("txn: first step towards cross-engine transactions"). All we need to do to enable full support for cross-engine transactions is to abort and send to read view memtx and vinyl transaction parts synchronously. To achieve that, we add two new engine callbacks - abort_with_conflict and send_to_read_view - and make memtx and vinyl use them instead of their internal methods. There's one more thing: since there are engines that don't support MVCC (memtx without `box.cfg.memtx_use_mvcc_engine` and memcs), we add a new engine flag - ENGINE_SUPPORTS_MVCC. The transaction manager will refuse to begin a cross-engine transaction unless this flag is either set or unset for all participating engines. In particular, the patch allows to simplify the vinyl transaction manager by removing various check that are now performed in the common code (like disallowing to start a write statement in a transaction that has been sent to a read view). Still, we don't drop vy_tx::state just yet because vinyl may send a single-statement transaction, which doesn't actually have a txn object, to a read view, see tarantool#11070. Also note silent removal of the TODO in txn_prepare asking to add engine_rollback. There's no need in calling engine_rollback there because it'll be called by txn_rollback, which is called after a failed txn_prepare anyway. Closes tarantool#1803 @TarantoolBot document Title: Document cross-engine transactions Tarantool now supports mixing statements for different storage engines in the same transaction, for example: ```lua local memtx = box.schema.space.create('memtx', {engine = 'memtx'}) memtx:create_index('primary') local vinyl = box.schema.space.create('vinyl', {engine = 'vinyl'}) vinyl:create_index('primary') memtx:insert({1, 'a'}) vinyl:insert({2, 'b'}) box.begin() memtx:replace(vinyl:get(2)) vinyl:replace(memtx:get(1)) box.commit() ``` Note, since accessing a vinyl space may trigger a fiber yield (to read a file from the disk), MVCC must be enabled in memtx to make use of the new feature: ```lua box.cfg{memtx_use_mvcc_engine = true} ```

Now, when all checks are done in the common transaction manager code, engine_begin_statement doesn't need to fail. Let's remove its error code for consistency with engine_begin. Follow-up tarantool#1803 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring

@TarantoolBot

Both memtx and vinyl support MVCC. Moreover, the Tarantool transaction manager already allows to mix statements for more than one engine in one transaction, see commit 09bfdd4 ("txn: first step towards cross-engine transactions"). All we need to do to enable full support for cross-engine transactions is to abort and send to read view memtx and vinyl transaction parts synchronously. To achieve that, we add two new engine callbacks - abort_with_conflict and send_to_read_view - and make memtx and vinyl use them instead of their internal methods. There's one more thing: since there are engines that don't support MVCC (memtx without `box.cfg.memtx_use_mvcc_engine` and memcs), we add a new engine flag - ENGINE_SUPPORTS_MVCC. The transaction manager will refuse to begin a cross-engine transaction unless this flag is either set or unset for all participating engines. In particular, the patch allows to simplify the vinyl transaction manager by removing various check that are now performed in the common code (like disallowing to start a write statement in a transaction that has been sent to a read view). Still, we don't drop vy_tx::state just yet because vinyl may send a single-statement transaction, which doesn't actually have a txn object, to a read view, see tarantool#11070. Also note silent removal of the TODO in txn_prepare asking to add engine_rollback. There's no need in calling engine_rollback there because it'll be called by txn_rollback, which is called after a failed txn_prepare anyway. Closes tarantool#1803 @TarantoolBot document Title: Document cross-engine transactions Tarantool now supports mixing statements for different storage engines in the same transaction, for example: ```lua local memtx = box.schema.space.create('memtx', {engine = 'memtx'}) memtx:create_index('primary') local vinyl = box.schema.space.create('vinyl', {engine = 'vinyl'}) vinyl:create_index('primary') memtx:insert({1, 'a'}) vinyl:insert({2, 'b'}) box.begin() memtx:replace(vinyl:get(2)) vinyl:replace(memtx:get(1)) box.commit() ``` Note, since accessing a vinyl space may trigger a fiber yield (to read a file from the disk), MVCC must be enabled in memtx to make use of the new feature: ```lua box.cfg{memtx_use_mvcc_engine = true} ```

@TarantoolBot

Both memtx and vinyl support MVCC. Moreover, the Tarantool transaction manager already allows to mix statements for more than one engine in one transaction, see commit 09bfdd4 ("txn: first step towards cross-engine transactions"). All we need to do to enable full support for cross-engine transactions is to abort and send to read view memtx and vinyl transaction parts synchronously. To achieve that, we add two new engine callbacks - abort_with_conflict and send_to_read_view - and make memtx and vinyl use them instead of their internal methods. There's one more thing: since there are engines that don't support MVCC (memtx without `box.cfg.memtx_use_mvcc_engine` and memcs), we add a new engine flag - ENGINE_SUPPORTS_MVCC. The transaction manager will refuse to begin a cross-engine transaction unless this flag is either set or unset for all participating engines. In particular, the patch allows to simplify the vinyl transaction manager by removing various check that are now performed in the common code (like disallowing to start a write statement in a transaction that has been sent to a read view). Still, we don't drop vy_tx::state just yet because vinyl may send a single-statement transaction, which doesn't actually have a txn object, to a read view, see tarantool#11070. Also note silent removal of the TODO in txn_prepare asking to add engine_rollback. There's no need in calling engine_rollback there because it'll be called by txn_rollback, which is called after a failed txn_prepare anyway. Closes tarantool#1803 @TarantoolBot document Title: Document cross-engine transactions Tarantool now supports mixing statements for different storage engines in the same transaction, for example: ```lua local memtx = box.schema.space.create('memtx', {engine = 'memtx'}) memtx:create_index('primary') local vinyl = box.schema.space.create('vinyl', {engine = 'vinyl'}) vinyl:create_index('primary') memtx:insert({1, 'a'}) vinyl:insert({2, 'b'}) box.begin() memtx:replace(vinyl:get(2)) vinyl:replace(memtx:get(1)) box.commit() ``` Note, since accessing a vinyl space may trigger a fiber yield (to read a file from the disk), MVCC must be enabled in memtx to make use of the new feature: ```lua box.cfg{memtx_use_mvcc_engine = true} ```

If a memtx snapshot contains a row for a different space engine, ER_CROSS_ENGINE_TRANSACTION is raised. This is confusing because this errors is normally raised on an attempt to start a cross-engine transaction. Besides, when support for cross-engine transactions is finally added, we'll want to remove this error. So let's introduce a new error code for this case and add a test. Needed for tarantool#1803 NO_DOC=minor NO_CHANGELOG=minor

There's no need in a separate PSN counter anymore now, when we have the PSN counter used by the memtx engine. It's behavior is equivalent to the counter used by vinyl. Using the same PSM counter in vinyl and memtx is important for sending a cross-engine transaction to a consistent read view. Needed for tarantool#1803 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring

First, this is consistent with our current memory allocation policy. Second, it'll make implementation of cross-engine transactions easier since we won't need to handle OOM when we send a transaction to a read view. Needed for tarantool#1803 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring

Currently, we reserve memory quota for a transaction when it's prepared. As a result, the engine_prepare callback may yield. This works fine as long as there are no other engines participating in the transaction, but this may lead to inconsistencies when we introduce cross-engine transactions (consider the situation when memtx is successfully prepared while vinyl is waiting for quota). To address this issue, let's move memory quota reservation to statement execution code, thus making the engine_prepare callback non-yielding. Note that in the scope of this patch we drop vy_tx::write_size, which was updated in the vinyl transaction manager, and instead introduce vy_tx::quota_reserved managed completely at the top level. The write_size was increased every time a statement was inserted into the transaction write set so multi-index statements were accounted once per each index. This actually wasn't quite correct because all indexes of the same space share the memory level. That's why the quota_reserve is updated only once per each statement no matter how many indexes the space has. Needed for tarantool#1803 NO_DOC=minor NO_TEST=already covered NO_CHANGELOG=minor

To send a cross-engine transaction to a read view, we need to start it in all engines first. We'll do it lazily, on conflict. If engine_begin could fail, we'd have to handle errors there, which would complicate the code. Actually, engine_begin never fails. Let's make it officially error-free by removing the return code. While we are at it, let's also remove unused xc wrappers. Needed for tarantool#1803 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring

There's no need in using a list for storing registered engines because we never unregister engines. Using an array is more convenient since it allows to quickly find an engine by id, which will be useful when we start implementing cross-engine transactions. Needed for tarantool#1803 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring

@TarantoolBot

Both memtx and vinyl support MVCC. Moreover, the Tarantool transaction manager already allows to mix statements for more than one engine in one transaction, see commit 09bfdd4 ("txn: first step towards cross-engine transactions"). All we need to do to enable full support for cross-engine transactions is to abort and send to read view memtx and vinyl transaction parts synchronously. To achieve that, we add two new engine callbacks - abort_with_conflict and send_to_read_view - and make memtx and vinyl use them instead of their internal methods. There's one more thing: since there are engines that don't support MVCC (memtx without `box.cfg.memtx_use_mvcc_engine` and memcs), we add a new engine flag - ENGINE_SUPPORTS_MVCC. The transaction manager will refuse to begin a cross-engine transaction unless this flag is either set or unset for all participating engines. In particular, the patch allows to simplify the vinyl transaction manager by removing various check that are now performed in the common code (like disallowing to start a write statement in a transaction that has been sent to a read view). Still, we don't drop vy_tx::state just yet because vinyl may send a single-statement transaction, which doesn't actually have a txn object, to a read view, see tarantool#11070. Also note silent removal of the TODO in txn_prepare asking to add engine_rollback. There's no need in calling engine_rollback there because it'll be called by txn_rollback, which is called after a failed txn_prepare anyway. Closes tarantool#1803 @TarantoolBot document Title: Document cross-engine transactions Tarantool now supports mixing statements for different storage engines in the same transaction, for example: ```lua local memtx = box.schema.space.create('memtx', {engine = 'memtx'}) memtx:create_index('primary') local vinyl = box.schema.space.create('vinyl', {engine = 'vinyl'}) vinyl:create_index('primary') memtx:insert({1, 'a'}) vinyl:insert({2, 'b'}) box.begin() memtx:replace(vinyl:get(2)) vinyl:replace(memtx:get(1)) box.commit() ``` Note, since accessing a vinyl space may trigger a fiber yield (to read a file from the disk), MVCC must be enabled in memtx to make use of the new feature: ```lua box.cfg{memtx_use_mvcc_engine = true} ```

@TarantoolBot

Both memtx and vinyl support MVCC. Moreover, the Tarantool transaction manager already allows to mix statements for more than one engine in one transaction, see commit 09bfdd4 ("txn: first step towards cross-engine transactions"). All we need to do to enable full support for cross-engine transactions is to abort and send to read view memtx and vinyl transaction parts synchronously. To achieve that, we add two new engine callbacks - abort_with_conflict and send_to_read_view - and make memtx and vinyl use them instead of their internal methods. There's one more thing: since there are engines that don't support MVCC (memtx without `box.cfg.memtx_use_mvcc_engine` and memcs), we add a new engine flag - ENGINE_SUPPORTS_MVCC. The transaction manager will refuse to begin a cross-engine transaction unless this flag is either set or unset for all participating engines. In particular, the patch allows to simplify the vinyl transaction manager by removing various check that are now performed in the common code (like disallowing to start a write statement in a transaction that has been sent to a read view). Still, we don't drop vy_tx::state just yet because vinyl may send a single-statement transaction, which doesn't actually have a txn object, to a read view, see tarantool#11070. Also note silent removal of the TODO in txn_prepare asking to add engine_rollback. There's no need in calling engine_rollback there because it'll be called by txn_rollback, which is called after a failed txn_prepare anyway. Closes tarantool#1803 @TarantoolBot document Title: Document cross-engine transactions Tarantool now supports mixing statements for different storage engines in the same transaction, for example: ```lua local memtx = box.schema.space.create('memtx', {engine = 'memtx'}) memtx:create_index('primary') local vinyl = box.schema.space.create('vinyl', {engine = 'vinyl'}) vinyl:create_index('primary') memtx:insert({1, 'a'}) vinyl:insert({2, 'b'}) box.begin() memtx:replace(vinyl:get(2)) vinyl:replace(memtx:get(1)) box.commit() ``` Note, since accessing a vinyl space may trigger a fiber yield (to read a file from the disk), MVCC must be enabled in memtx to make use of the new feature: ```lua box.cfg{memtx_use_mvcc_engine = true} ```

If a memtx snapshot contains a row for a different space engine, ER_CROSS_ENGINE_TRANSACTION is raised. This is confusing because this errors is normally raised on an attempt to start a cross-engine transaction. Besides, when support for cross-engine transactions is finally added, we'll want to remove this error. So let's introduce a new error code for this case and add a test. Needed for #1803 NO_DOC=minor NO_CHANGELOG=minor

There's no need in a separate PSN counter anymore now, when we have the PSN counter used by the memtx engine. It's behavior is equivalent to the counter used by vinyl. Using the same PSM counter in vinyl and memtx is important for sending a cross-engine transaction to a consistent read view. Needed for #1803 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring

First, this is consistent with our current memory allocation policy. Second, it'll make implementation of cross-engine transactions easier since we won't need to handle OOM when we send a transaction to a read view. Needed for #1803 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring

Currently, we reserve memory quota for a transaction when it's prepared. As a result, the engine_prepare callback may yield. This works fine as long as there are no other engines participating in the transaction, but this may lead to inconsistencies when we introduce cross-engine transactions (consider the situation when memtx is successfully prepared while vinyl is waiting for quota). To address this issue, let's move memory quota reservation to statement execution code, thus making the engine_prepare callback non-yielding. Note that in the scope of this patch we drop vy_tx::write_size, which was updated in the vinyl transaction manager, and instead introduce vy_tx::quota_reserved managed completely at the top level. The write_size was increased every time a statement was inserted into the transaction write set so multi-index statements were accounted once per each index. This actually wasn't quite correct because all indexes of the same space share the memory level. That's why the quota_reserve is updated only once per each statement no matter how many indexes the space has. Needed for #1803 NO_DOC=minor NO_TEST=already covered NO_CHANGELOG=minor

To send a cross-engine transaction to a read view, we need to start it in all engines first. We'll do it lazily, on conflict. If engine_begin could fail, we'd have to handle errors there, which would complicate the code. Actually, engine_begin never fails. Let's make it officially error-free by removing the return code. While we are at it, let's also remove unused xc wrappers. Needed for #1803 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring

There's no need in using a list for storing registered engines because we never unregister engines. Using an array is more convenient since it allows to quickly find an engine by id, which will be useful when we start implementing cross-engine transactions. Needed for #1803 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring

@TarantoolBot

Both memtx and vinyl support MVCC. Moreover, the Tarantool transaction manager already allows to mix statements for more than one engine in one transaction, see commit 09bfdd4 ("txn: first step towards cross-engine transactions"). All we need to do to enable full support for cross-engine transactions is to abort and send to read view memtx and vinyl transaction parts synchronously. To achieve that, we add two new engine callbacks - abort_with_conflict and send_to_read_view - and make memtx and vinyl use them instead of their internal methods. There's one more thing: since there are engines that don't support MVCC (memtx without `box.cfg.memtx_use_mvcc_engine` and memcs), we add a new engine flag - ENGINE_SUPPORTS_MVCC. The transaction manager will refuse to begin a cross-engine transaction unless this flag is either set or unset for all participating engines. In particular, the patch allows to simplify the vinyl transaction manager by removing various check that are now performed in the common code (like disallowing to start a write statement in a transaction that has been sent to a read view). Still, we don't drop vy_tx::state just yet because vinyl may send a single-statement transaction, which doesn't actually have a txn object, to a read view, see #11070. Also note silent removal of the TODO in txn_prepare asking to add engine_rollback. There's no need in calling engine_rollback there because it'll be called by txn_rollback, which is called after a failed txn_prepare anyway. Closes #1803 @TarantoolBot document Title: Document cross-engine transactions Tarantool now supports mixing statements for different storage engines in the same transaction, for example: ```lua local memtx = box.schema.space.create('memtx', {engine = 'memtx'}) memtx:create_index('primary') local vinyl = box.schema.space.create('vinyl', {engine = 'vinyl'}) vinyl:create_index('primary') memtx:insert({1, 'a'}) vinyl:insert({2, 'b'}) box.begin() memtx:replace(vinyl:get(2)) vinyl:replace(memtx:get(1)) box.commit() ``` Note, since accessing a vinyl space may trigger a fiber yield (to read a file from the disk), MVCC must be enabled in memtx to make use of the new feature: ```lua box.cfg{memtx_use_mvcc_engine = true} ```

If vinyl_engine_begin_statement() fails, it leaves engine_savepoint unset. As a result, vinyl_engine_rollback_statement() called afterwards by txn_begin_stmt() assumes there were no entries in vy_tx::log before the statement was begun and rolls back everything. If there were statements, an attempt to roll them back triggers a crash. Let's fix this issue by skipping invocation of the rollback_statement engine callback in case begin_statement failed. We achieve that by leaving txn::engine and txn::space unset on failure. Fixes commit b9c5a1d ("txn: enable cross-engine transactions"). Closes tarantool#11187 Follow-up tarantool#1803 NO_DOC=bug fix NO_CHANGELOG=unreleased

If vinyl_engine_begin_statement() fails, it leaves engine_savepoint unset. As a result, vinyl_engine_rollback_statement() called afterwards by txn_begin_stmt() assumes there were no entries in vy_tx::log before the statement was begun and rolls back everything. If there were statements, an attempt to roll them back triggers a crash. Let's fix this issue by skipping invocation of the rollback_statement engine callback in case begin_statement failed. We achieve that by leaving txn::engine and txn::space unset on failure. Fixes commit b9c5a1d ("txn: enable cross-engine transactions"). Closes #11187 Follow-up #1803 NO_DOC=bug fix NO_CHANGELOG=unreleased

The commit updates suppression list of the engine fuzzer - the added errors actually can be raised by Tarantool during fuzzing, but were not added to the list. Also, the commit updates suppresion list according to the new cross-engine transaction behavior. Follows up tarantool#1803 NO_CHANGELOG=fuzzing NO_DOC=fuzzing

The commit updates suppression list of the engine fuzzer - the added errors actually can be raised by Tarantool during fuzzing, but were not added to the list. Also, the commit updates suppresion list according to the new cross-engine transaction behavior. Follows up #1803 NO_CHANGELOG=fuzzing NO_DOC=fuzzing

kostja added the feature A new functionality label Sep 30, 2016

kostja added this to the 1.8.0 milestone Sep 30, 2016

kostja modified the milestones: 1.8.0, 1.8.1 Mar 31, 2017

kostja modified the milestones: 1.8.3, 1.8.2 May 31, 2017

kyukhin modified the milestones: 1.8.2, 1.8.3 Aug 3, 2017

kyukhin mentioned this issue Aug 3, 2017

Expose savepoints to API #2655

Closed

kostja modified the milestones: 1.8.3, 1.8.4, wishlist Oct 27, 2017

alyapunov mentioned this issue Apr 15, 2020

Transaction engine for memtx engine #4897

Closed

kyukhin added backlog labels Dec 25, 2020

alyapunov added 13sp tmp labels Mar 19, 2021

kyukhin modified the milestones: wishlist, 2.9.1 Mar 23, 2021

kyukhin added the epic label Mar 23, 2021

kyukhin modified the milestones: 2.9.1, 2.10.1 Mar 26, 2021

kyukhin removed the tmp label Apr 30, 2021

kyukhin modified the milestones: 2.10.1, wishlist Aug 19, 2021

locker closed this as completed in #11071 Feb 11, 2025

sergos added this to Tarantool Public Roadmap Feb 12, 2025

sergos moved this to Q1 2025 – Jan-Mar in Tarantool Public Roadmap Feb 12, 2025

locker mentioned this issue Feb 26, 2025

txn: do not call engine rollback_statement if begin_statement failed #11205

Merged

locker mentioned this issue Mar 18, 2025

Can't use standalone sequence in yielding transaction #11266

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cross-engine transactions #1803

Cross-engine transactions #1803

Cross-engine transactions #1803

Cross-engine transactions #1803

Comments