Add reseed_rng option to p_iter_fork #39025
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Documentation preview for this PR (built with commit 8518218; changes) is ready! 🎉 |
kwankyu
reviewed
Jan 31, 2025
kwankyu
reviewed
Jan 31, 2025
kwankyu
reviewed
Jan 31, 2025
kwankyu
reviewed
Jan 31, 2025
kwankyu
reviewed
Jan 31, 2025
kwankyu
reviewed
Jan 31, 2025
|
Implemented the comments as far as I understood them. I'm using list to follow the prior code. (Also, Python's iterators are untyped and thus easy to get wrong in corner cases for hobbyists like me. Lists behave as expected.) |
kwankyu
reviewed
Feb 10, 2025
vbraun
pushed a commit
to vbraun/sage
that referenced
this pull request
Feb 18, 2025
sagemathgh-39025: Add reseed_rng option to p_iter_fork This PR implements a convenience feature for the `@parallel` decorator, or more precisely, for `p_iter_fork`. By setting `reseed_rng=True` the random number generator is reset in each subprocess by running `set_random_seed(self.worker_seed)`. It is ensured that the worker seeds are deterministically derived and thus, the results are always reproducible. The new behaviour is useful for running probabilistic experiments and taking advantage of multiple cores. The PR adds one "doc test". It would make sense to have additional (randomized) tests to assert that future modifications don't break the reproducibility. But I failed to locate a comprehensive test suite for sage/parallel, in particular, no existing tests for reproducibility of parallel computations. The `sage/tests` folder seems to contain other non-specific tests? Coding style w.r.t. <https://doc.sagemath.org/html/en/reference/misc/sag e/misc/randstate.html> - The documentation said `with seed(worker_seed)` should be used. I did not use it, because the current implementation reseeds the whole subprocess once and for all. - The documentation says NTL does not reproduce. This is a very surprising and unfortunate state of affairs. I expect that `reseed_rng` inherits this caveat. I did not make it explicit in the doc string, because if the problem occurs, then also `reseed_rng=False` should have it, but there is no mention of it. ### 📝 Checklist - [x] The title is concise and informative. - [x] The description explains in detail what this PR is about. - [x] I have linked a relevant issue or discussion. - [x] I have created tests covering the changes. - [x] I have updated the documentation and checked the documentation preview. The doc tests passed for me, but preview via running `sage --docbuild tutorial html` gives me `ImportError: cannot import name count_all_local_good_types_normal_form' from 'sage.quadratic_forms.count_local_2'` so doc tests don't work although that has probably nothing to with this PR. URL: sagemath#39025 Reported by: mklss Reviewer(s): Kwankyu Lee
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR implements a convenience feature for the
@paralleldecorator, or more precisely, forp_iter_fork.By setting
reseed_rng=Truethe random number generator is reset in each subprocess by runningset_random_seed(self.worker_seed).It is ensured that the worker seeds are deterministically derived and thus, the results are always reproducible.
The new behaviour is useful for running probabilistic experiments and taking advantage of multiple cores.
The PR adds one "doc test". It would make sense to have additional (randomized) tests to assert that future modifications don't break the reproducibility. But I failed to locate a comprehensive test suite for sage/parallel, in particular, no existing tests for reproducibility of parallel computations. The
sage/testsfolder seems to contain other non-specific tests?Coding style w.r.t. https://doc.sagemath.org/html/en/reference/misc/sage/misc/randstate.html
with seed(worker_seed)should be used. I did not use it, because the current implementation reseeds the whole subprocess once and for all.reseed_rnginherits this caveat. I did not make it explicit in the doc string, because if the problem occurs, then alsoreseed_rng=Falseshould have it, but there is no mention of it.📝 Checklist
The doc tests passed for me, but preview via running
sage --docbuild tutorial htmlgives meImportError: cannot import name count_all_local_good_types_normal_form' from 'sage.quadratic_forms.count_local_2'so doc tests don't work although that has probably nothing to with this PR.