add regression_threshold_seconds argument to regression/test_runner.py
#17485
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks! |
Tmonster
approved these changes
May 14, 2025
krlmlr
added a commit
to duckdb/duckdb-r
that referenced
this pull request
May 18, 2025
DROP of missing entry should fail in binding (duckdb/duckdb#17474) add regression_threshold_seconds argument to `regression/test_runner.py` (duckdb/duckdb#17485)
krlmlr
added a commit
to duckdb/duckdb-r
that referenced
this pull request
May 19, 2025
DROP of missing entry should fail in binding (duckdb/duckdb#17474) add regression_threshold_seconds argument to `regression/test_runner.py` (duckdb/duckdb#17485)
krlmlr
added a commit
to duckdb/duckdb-r
that referenced
this pull request
May 19, 2025
DROP of missing entry should fail in binding (duckdb/duckdb#17474) add regression_threshold_seconds argument to `regression/test_runner.py` (duckdb/duckdb#17485)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Current value of the
regression_threshold_secondsis for very fast benchmarks (e.g. sf=0.1). Which more likely causes false positive regressions on large scale factor (sf=100) regression tests.When run large scale factor (sf=100) regression tests, we'd like to define a bigger
regression_threshold_secondsvalue.Run the
regression/test_runner.py --regression-threshold-seconds=2.0when with large scale factor. There is a default value of the regression threshold in seconds for very fast benchmarks, so run it without passingregression-threshold-secondsparameter to the test runner.