-
Notifications
You must be signed in to change notification settings - Fork 37.2k
test: Convert non-wallet tests to use our python MiniWallet #20078
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
…p2p_lock acquires) 5b77f80 test: add p2p_lock acquires in p2p_leak_tx.py (Sebastian Falbesoner) cc8c682 test: use MiniWallet for p2p_leak_tx.py (Sebastian Falbesoner) Pull request description: This PR enables one more of the non-wallet functional tests (p2p_leak_tx.py) to be run even with the Bitcoin Core wallet disabled by using the new MiniWallet instead, as proposed in #20078. It also adds missing p2p_lock acquires that need to be held while modifying internal p2p Interface state (in this case the `last_message` dictionary) to avoid data races. ACKs for top commit: laanwj: Code review ACK 5b77f80 Tree-SHA512: 6661bc6e3491a9af4bf040f379e5955c525136397e99d3eadde92e247580d0d87efff750e6d3b1f6d9a4e578144a433a982f574ef056b44dd6bca33873a1bae6
Hey, I can give this issue a shot. |
Sure. No need to ask on this one, as it is open ended. There should be enough tests for everyone to convert |
…(use MiniWallet, add logging) b128b56 test: add logging for mining_getblocktemplate_longpoll.py (Sebastian Falbesoner) 8ee3536 test: remove unused helpers random_transaction(), make_change() and gather_inputs() (Sebastian Falbesoner) fddce7e test: use MiniWallet for mining_getblocktemplate_longpoll.py (Sebastian Falbesoner) Pull request description: This PR enables one more of the non-wallet functional tests (mining_getblocktemplate_longpoll.py) to be run even with the Bitcoin Core wallet disabled by using the new MiniWallet instead, as proposed in #20078. Also adds missing log messages for the subtests. This was the only functional test that used the `random_transaction` helper in `test_framework/util.py`, hence it is removed, together with other helpers (`make_change` and `gather_inputs`) that were again only used by `random_transaction`. ACKs for top commit: MarcoFalke: ACK b128b56 Tree-SHA512: 09a5fa7b0f5976a47040f7027236d7ec0426d5a4829a082221c4b5fae294470230e89ae3df0bca0eea26833162c03980517f5cc88761ad251c3df4c4a49bca46
…ements (use MiniWallet, add logging) b128b56 test: add logging for mining_getblocktemplate_longpoll.py (Sebastian Falbesoner) 8ee3536 test: remove unused helpers random_transaction(), make_change() and gather_inputs() (Sebastian Falbesoner) fddce7e test: use MiniWallet for mining_getblocktemplate_longpoll.py (Sebastian Falbesoner) Pull request description: This PR enables one more of the non-wallet functional tests (mining_getblocktemplate_longpoll.py) to be run even with the Bitcoin Core wallet disabled by using the new MiniWallet instead, as proposed in bitcoin#20078. Also adds missing log messages for the subtests. This was the only functional test that used the `random_transaction` helper in `test_framework/util.py`, hence it is removed, together with other helpers (`make_change` and `gather_inputs`) that were again only used by `random_transaction`. ACKs for top commit: MarcoFalke: ACK b128b56 Tree-SHA512: 09a5fa7b0f5976a47040f7027236d7ec0426d5a4829a082221c4b5fae294470230e89ae3df0bca0eea26833162c03980517f5cc88761ad251c3df4c4a49bca46
…sabled 21f2433 test: run mempool_spend_coinbase.py even with wallet disabled (Michael Dietz) Pull request description: Run the mempool spend coinbase test even when the wallet was not compiled, as proposed in #20078. ACKs for top commit: laanwj: ACK 21f2433 Tree-SHA512: 301582c04376371cfa8f1ebb2418a4341b42ddcd9ad4f48b58bcf888d867a97bdc409972856b67a8339ac5e60124aefee82a049b4f7fc6bca7a18d7e92e090be
3b064fc test: run mempool_expiry.py even with wallet disabled (Michael Dietz) Pull request description: Run the mempool expiry test even when the wallet was not compiled, as proposed in bitcoin/bitcoin#20078. ACKs for top commit: MarcoFalke: ACK 3b064fc Tree-SHA512: 5860dc021d02bc3752268ec1e859505bec87174953223b34b1af8a8e4ab66d645458fbf9571c0b816a9de891c3ff41314996e580869671fccd6972c093e78154
…abled 3b064fc test: run mempool_expiry.py even with wallet disabled (Michael Dietz) Pull request description: Run the mempool expiry test even when the wallet was not compiled, as proposed in bitcoin#20078. ACKs for top commit: MarcoFalke: ACK 3b064fc Tree-SHA512: 5860dc021d02bc3752268ec1e859505bec87174953223b34b1af8a8e4ab66d645458fbf9571c0b816a9de891c3ff41314996e580869671fccd6972c093e78154
11a3272 test: run mempool_resurrect.py even with wallet disabled (Michael Dietz) Pull request description: Another functional test rewritten as proposed in #20078 **Request for help:** `node.gettransaction(txid)` fails for transactions sent with `wallet.send_self_transfer`. Even though the `txid`s look correct, are added to the mempool correctly, and removed from the mempool when a block is mined - all as expected. However, `node.gettransaction(txid)` throws the error: ```sh Traceback (most recent call last): File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/test_framework.py", line 126, in main self.run_test() File "/Users/michaeldietz/Documents/bitcoin/test/functional/mempool_resurrect.py", line 43, in run_test assert_equal(len(list(filter(lambda txid: node.gettransaction(txid)["confirmations"] > 0, spends_ids))), len(spends_ids)) File "/Users/michaeldietz/Documents/bitcoin/test/functional/mempool_resurrect.py", line 43, in <lambda> assert_equal(len(list(filter(lambda txid: node.gettransaction(txid)["confirmations"] > 0, spends_ids))), len(spends_ids)) File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/coverage.py", line 47, in __call__ return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs) File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/authproxy.py", line 146, in __call__ raise JSONRPCException(response['error'], status) test_framework.authproxy.JSONRPCException: Invalid or non-wallet transaction id (-5) ``` Anyone know what's going wrong / can point me in the right direction if I'm making a mistake, or `MiniWallet` needs to be improved for this to work correctly? ACKs for top commit: MarcoFalke: ACK 11a3272 Tree-SHA512: 13d83a13ec23920db716e99b68670e61329d1cc73b12063d85bc1679ee6425a9951da4d2e392ca1f27760be7be049ccdc6f504e192ed5cd24ed0ba003b66fab3
hi there, does this issue still require work to be done on it? |
@stackman27: As far as I can tell there are around 40 tests from If you look at the above notifications it appears 4 test updates have been merged and 1 test update is open. Marco initially fixed two (p2p_feefilter.py, rpc_txoutproof.py) So there are lots of tests to work on. Please link to this issue if you open a PR as it will make it easier to monitor which ones people are working on. |
How would I go about creating and sending tx using |
@stackman27: The Miniwallet class defines In mempool_limit.py that should be fine for For more info on OP_TRUE addresses (P2SH rather than P2WSH but may be useful): https://bitcoin.stackexchange.com/questions/72773/how-to-create-a-p2sh-transaction-with-a-scriptsig-of-op-true |
Did you answer your question @stackman27? I think you deleted your latest post asking about |
I think so. I just generated utxos using |
@michaelfolkson i'm sorry to bug you so much. But how would i setup |
…abled a7599c8 test: run mempool_compatibility.py even with wallet disabled (Michael Dietz) Pull request description: Another functional test rewritten as proposed in bitcoin/bitcoin#20078 ACKs for top commit: MarcoFalke: review ACK a7599c8 didn't test Tree-SHA512: cc7a617e5489ed27bbdbdee85a82fa08525375061f7f4524577a6b8ecb340396adac88419b51f513be22ca53edd0a3bd5d572d9f43ffc2c18550b0ef9069d238
Concept ACK |
I'll have a go at |
Fixed @danielabrozzoni. Thanks for the context!
Added to the list, sorry, must've missed it. Thanks for pointing it out 👍 |
Thanks @shommel for the list! Didn't realize how much progress had been made, not many left now. I was planning to play around with Taproot in Python at some point so I'll take feature_taproot.py (somewhat surprised it is still there). If someone wants to take it off me feel free though. |
…on.py e895900 test: Use MiniWallet in rpc_rawtransaction.py (Daniela Brozzoni) e93046c MOVEONLY: Move signrawtransactionwithwallet test (Daniela Brozzoni) Pull request description: This PR allows `rpc_rawtransaction.py` to be run even without the Core wallet by using the MiniWallet instead, as proposed in bitcoin/bitcoin#20078. This test was previously run twice, once with `--legacy-wallet` and once with `--descriptors`. Since this would have meant running the same test twice if the wallet wasn't compiled, now we run it just once with the legacy wallet. ACKs for top commit: jonatack: ACK e895900 Tree-SHA512: d1580570a54dad8e30a5df1ab7d03ecb3f824efe6843323e1f3aef63592045d823c7d54fc86321dc7c1d414854a253431a01a7baa9f30426ea9a09ef11ae3a04
1da5e45 test: use MiniWallet for feature_dbcrash.py (Sebastian Falbesoner) Pull request description: This PR enables one more of the non-wallet functional tests (feature_dbcrash.py) to be run even with the Bitcoin Code wallet by using the MiniWallet instead, as proposed in bitcoin#20078.
1da5e45 test: use MiniWallet for feature_dbcrash.py (Sebastian Falbesoner) Pull request description: This PR enables one more of the non-wallet functional tests (feature_dbcrash.py) to be run even with the Bitcoin Code wallet by using the MiniWallet instead, as proposed in bitcoin/bitcoin#20078. ACKs for top commit: laanwj: Code review ACK 1da5e45 brunoerg: crACK 1da5e45 Tree-SHA512: 75ee9a32fd1451254004797d695d18032bd0fcb66ebd88cf737e147e43812525f6e884ec05fcc4f76f566dc71174c8ed7347bcdce16567db6511746ae64cead0
1da5e45 test: use MiniWallet for feature_dbcrash.py (Sebastian Falbesoner) Pull request description: This PR enables one more of the non-wallet functional tests (feature_dbcrash.py) to be run even with the Bitcoin Code wallet by using the MiniWallet instead, as proposed in bitcoin#20078. ACKs for top commit: laanwj: Code review ACK 1da5e45 brunoerg: crACK 1da5e45 Tree-SHA512: 75ee9a32fd1451254004797d695d18032bd0fcb66ebd88cf737e147e43812525f6e884ec05fcc4f76f566dc71174c8ed7347bcdce16567db6511746ae64cead0
…ction.py b167e53 test: refactor: use `create_lots_of_big_transactions` to dedup where possible (Sebastian Falbesoner) 8973eeb test: use MiniWallet for mining_prioritisetransaction.py (Sebastian Falbesoner) Pull request description: This PR enables one more of the non-wallet functional tests (mining_prioritisetransaction.py) to be run even with the Bitcoin Code wallet by using the MiniWallet instead, as proposed in bitcoin#20078. Note that the adapted helper function `create_lots_of_big_transactions` is currently only used in this test, i.e. there was no need to change any others. ACKs for top commit: ayush933: tACK b167e53 danielabrozzoni: tACK b167e53 kouloumos: ACK b167e53 furszy: ACK b167e53 Tree-SHA512: ccae20d7d414a720efdeea9c2ae399aa53a3a0e7db72bff8d0cb75d90621a7ae7c019ba68d24f9d06f7b111f87ff33bb9d8e5aa08b763e606cf10268780e205c
From the description of the issue
I understand why those tests are the obvious candidates for MiniWallet usage, but are those the only tests that could benefit from MiniWallet usage? During the process of converting tests to use the MiniWallet, a number of practical methods have been created thus enabling more readable tests by using the MiniWallet class. I am trying to understand if the usage of the MiniWallet to simplify tests that can already run with the Bitcoin Core wallet disabled (e.g. #25379) is something that could be considered beneficial. |
Considering segwit has been implemented long time ago... Is it worth it to convert |
Yes, anything without |
rpc_psbt looks like a wallet test, so maybe it can be renamed to |
Picking |
Nice. Pull requests welcome, however I am going to close this issue for now. The remaining tests can not be trivially converted without also extending MiniWallet (I think), so it might be better to open specific threads/issues for each test. |
4159ccd test: Run feature_bip68_sequence.py with MiniWallet (Miles Liu) fc0caaf test: Add "include mempool" flag to MiniWallet rescan_utxos (Miles Liu) d0a909a test: Add "include immature coinbase" flag to MiniWallet get_utxos (Miles Liu) e5b9127 test: Add signs P2TR and RAWSCRIPT to MiniWallet sign_tx (Miles Liu) Pull request description: This PR enables one more of the non-wallet functional tests (feature_bip68_sequence.py) to be run even when no wallet is compiled in by using the MiniWallet instead, as proposed in #20078. ACKs for top commit: achow101: ACK 4159ccd MarcoFalke: review ACK 4159ccd 🤸 Tree-SHA512: e891b189381e961c840b45fc30d058363707fd54c1c4bdc3d29623b03309981f1d3ebfac27e6aecf621bdbcd7e31754a3ef7c53f86346f7dd241c137e64c92bd
…llet disabled 21f2433 test: run mempool_spend_coinbase.py even with wallet disabled (Michael Dietz) Pull request description: Run the mempool spend coinbase test even when the wallet was not compiled, as proposed in bitcoin#20078. ACKs for top commit: laanwj: ACK 21f2433 Tree-SHA512: 301582c04376371cfa8f1ebb2418a4341b42ddcd9ad4f48b58bcf888d867a97bdc409972856b67a8339ac5e60124aefee82a049b4f7fc6bca7a18d7e92e090be
…en with wallet disabled 21f2433 test: run mempool_spend_coinbase.py even with wallet disabled (Michael Dietz) Pull request description: Run the mempool spend coinbase test even when the wallet was not compiled, as proposed in bitcoin#20078. ACKs for top commit: laanwj: ACK 21f2433 Tree-SHA512: 301582c04376371cfa8f1ebb2418a4341b42ddcd9ad4f48b58bcf888d867a97bdc409972856b67a8339ac5e60124aefee82a049b4f7fc6bca7a18d7e92e090be
…llet disabled 21f2433 test: run mempool_spend_coinbase.py even with wallet disabled (Michael Dietz) Pull request description: Run the mempool spend coinbase test even when the wallet was not compiled, as proposed in bitcoin#20078. ACKs for top commit: laanwj: ACK 21f2433 Tree-SHA512: 301582c04376371cfa8f1ebb2418a4341b42ddcd9ad4f48b58bcf888d867a97bdc409972856b67a8339ac5e60124aefee82a049b4f7fc6bca7a18d7e92e090be
…disabled 11a3272 test: run mempool_resurrect.py even with wallet disabled (Michael Dietz) Pull request description: Another functional test rewritten as proposed in bitcoin#20078 **Request for help:** `node.gettransaction(txid)` fails for transactions sent with `wallet.send_self_transfer`. Even though the `txid`s look correct, are added to the mempool correctly, and removed from the mempool when a block is mined - all as expected. However, `node.gettransaction(txid)` throws the error: ```sh Traceback (most recent call last): File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/test_framework.py", line 126, in main self.run_test() File "/Users/michaeldietz/Documents/bitcoin/test/functional/mempool_resurrect.py", line 43, in run_test assert_equal(len(list(filter(lambda txid: node.gettransaction(txid)["confirmations"] > 0, spends_ids))), len(spends_ids)) File "/Users/michaeldietz/Documents/bitcoin/test/functional/mempool_resurrect.py", line 43, in <lambda> assert_equal(len(list(filter(lambda txid: node.gettransaction(txid)["confirmations"] > 0, spends_ids))), len(spends_ids)) File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/coverage.py", line 47, in __call__ return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs) File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/authproxy.py", line 146, in __call__ raise JSONRPCException(response['error'], status) test_framework.authproxy.JSONRPCException: Invalid or non-wallet transaction id (-5) ``` Anyone know what's going wrong / can point me in the right direction if I'm making a mistake, or `MiniWallet` needs to be improved for this to work correctly? ACKs for top commit: MarcoFalke: ACK 11a3272 Tree-SHA512: 13d83a13ec23920db716e99b68670e61329d1cc73b12063d85bc1679ee6425a9951da4d2e392ca1f27760be7be049ccdc6f504e192ed5cd24ed0ba003b66fab3
…llet disabled 21f2433 test: run mempool_spend_coinbase.py even with wallet disabled (Michael Dietz) Pull request description: Run the mempool spend coinbase test even when the wallet was not compiled, as proposed in bitcoin#20078. ACKs for top commit: laanwj: ACK 21f2433 Tree-SHA512: 301582c04376371cfa8f1ebb2418a4341b42ddcd9ad4f48b58bcf888d867a97bdc409972856b67a8339ac5e60124aefee82a049b4f7fc6bca7a18d7e92e090be
…disabled 11a3272 test: run mempool_resurrect.py even with wallet disabled (Michael Dietz) Pull request description: Another functional test rewritten as proposed in bitcoin#20078 **Request for help:** `node.gettransaction(txid)` fails for transactions sent with `wallet.send_self_transfer`. Even though the `txid`s look correct, are added to the mempool correctly, and removed from the mempool when a block is mined - all as expected. However, `node.gettransaction(txid)` throws the error: ```sh Traceback (most recent call last): File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/test_framework.py", line 126, in main self.run_test() File "/Users/michaeldietz/Documents/bitcoin/test/functional/mempool_resurrect.py", line 43, in run_test assert_equal(len(list(filter(lambda txid: node.gettransaction(txid)["confirmations"] > 0, spends_ids))), len(spends_ids)) File "/Users/michaeldietz/Documents/bitcoin/test/functional/mempool_resurrect.py", line 43, in <lambda> assert_equal(len(list(filter(lambda txid: node.gettransaction(txid)["confirmations"] > 0, spends_ids))), len(spends_ids)) File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/coverage.py", line 47, in __call__ return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs) File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/authproxy.py", line 146, in __call__ raise JSONRPCException(response['error'], status) test_framework.authproxy.JSONRPCException: Invalid or non-wallet transaction id (-5) ``` Anyone know what's going wrong / can point me in the right direction if I'm making a mistake, or `MiniWallet` needs to be improved for this to work correctly? ACKs for top commit: MarcoFalke: ACK 11a3272 Tree-SHA512: 13d83a13ec23920db716e99b68670e61329d1cc73b12063d85bc1679ee6425a9951da4d2e392ca1f27760be7be049ccdc6f504e192ed5cd24ed0ba003b66fab3
…disabled 11a3272 test: run mempool_resurrect.py even with wallet disabled (Michael Dietz) Pull request description: Another functional test rewritten as proposed in bitcoin#20078 **Request for help:** `node.gettransaction(txid)` fails for transactions sent with `wallet.send_self_transfer`. Even though the `txid`s look correct, are added to the mempool correctly, and removed from the mempool when a block is mined - all as expected. However, `node.gettransaction(txid)` throws the error: ```sh Traceback (most recent call last): File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/test_framework.py", line 126, in main self.run_test() File "/Users/michaeldietz/Documents/bitcoin/test/functional/mempool_resurrect.py", line 43, in run_test assert_equal(len(list(filter(lambda txid: node.gettransaction(txid)["confirmations"] > 0, spends_ids))), len(spends_ids)) File "/Users/michaeldietz/Documents/bitcoin/test/functional/mempool_resurrect.py", line 43, in <lambda> assert_equal(len(list(filter(lambda txid: node.gettransaction(txid)["confirmations"] > 0, spends_ids))), len(spends_ids)) File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/coverage.py", line 47, in __call__ return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs) File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/authproxy.py", line 146, in __call__ raise JSONRPCException(response['error'], status) test_framework.authproxy.JSONRPCException: Invalid or non-wallet transaction id (-5) ``` Anyone know what's going wrong / can point me in the right direction if I'm making a mistake, or `MiniWallet` needs to be improved for this to work correctly? ACKs for top commit: MarcoFalke: ACK 11a3272 Tree-SHA512: 13d83a13ec23920db716e99b68670e61329d1cc73b12063d85bc1679ee6425a9951da4d2e392ca1f27760be7be049ccdc6f504e192ed5cd24ed0ba003b66fab3
…disabled 11a3272 test: run mempool_resurrect.py even with wallet disabled (Michael Dietz) Pull request description: Another functional test rewritten as proposed in bitcoin#20078 **Request for help:** `node.gettransaction(txid)` fails for transactions sent with `wallet.send_self_transfer`. Even though the `txid`s look correct, are added to the mempool correctly, and removed from the mempool when a block is mined - all as expected. However, `node.gettransaction(txid)` throws the error: ```sh Traceback (most recent call last): File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/test_framework.py", line 126, in main self.run_test() File "/Users/michaeldietz/Documents/bitcoin/test/functional/mempool_resurrect.py", line 43, in run_test assert_equal(len(list(filter(lambda txid: node.gettransaction(txid)["confirmations"] > 0, spends_ids))), len(spends_ids)) File "/Users/michaeldietz/Documents/bitcoin/test/functional/mempool_resurrect.py", line 43, in <lambda> assert_equal(len(list(filter(lambda txid: node.gettransaction(txid)["confirmations"] > 0, spends_ids))), len(spends_ids)) File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/coverage.py", line 47, in __call__ return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs) File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/authproxy.py", line 146, in __call__ raise JSONRPCException(response['error'], status) test_framework.authproxy.JSONRPCException: Invalid or non-wallet transaction id (-5) ``` Anyone know what's going wrong / can point me in the right direction if I'm making a mistake, or `MiniWallet` needs to be improved for this to work correctly? ACKs for top commit: MarcoFalke: ACK 11a3272 Tree-SHA512: 13d83a13ec23920db716e99b68670e61329d1cc73b12063d85bc1679ee6425a9951da4d2e392ca1f27760be7be049ccdc6f504e192ed5cd24ed0ba003b66fab3
It is possible to compile Bitcoin Core without the wallet. Though, many tests require the Bitcoin Core wallet to create and sign transactions. Ideally, every non-wallet test (for example consensus tests) are run even with the Bitcoin Core wallet disabled. This ensures that users running without the wallet can test their Bitcoin Core with the functional test suite.
Thus, tests that use the Bitcoin Core wallet should be rewritten to use the python MiniWallet.
This is an open-ended issue. You can find candidate tests via
git grep 'self.skip_if_no_wallet()'
. Exclude the tests that start withwallet_
, as those are tests that are meant to test the Bitcoin Core wallet.Refer to #19922 and #19800 as examples on how to approach this issue.
Edit: See also #20078 (comment)
Useful skills:
Want to work on this issue?
For guidance on contributing, please read CONTRIBUTING.md before opening your pull request.
The text was updated successfully, but these errors were encountered: