This repository was archived by the owner on May 5, 2025. It is now read-only.
feat: don't do flake detection on private repos not on the pro plan #747
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAll modified and coverable lines are covered by tests ✅
✅ All tests successful. No failed tests found. @@ Coverage Diff @@
## main #747 +/- ##
=======================================
Coverage 97.91% 97.91%
=======================================
Files 444 444
Lines 35403 35416 +13
=======================================
+ Hits 34664 34677 +13
Misses 739 739
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
✅ All tests successful. No failed tests found. Additional details and impacted files@@ Coverage Diff @@
## main #747 +/- ##
=======================================
Coverage 97.91% 97.91%
=======================================
Files 444 444
Lines 35403 35416 +13
=======================================
+ Hits 34664 34677 +13
Misses 739 739
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
✅ All tests successful. No failed tests found. @@ Coverage Diff @@
## main #747 +/- ##
=======================================
Coverage 97.91% 97.91%
=======================================
Files 444 444
Lines 35403 35416 +13
=======================================
+ Hits 34664 34677 +13
Misses 739 739
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Codecov ReportAll modified and coverable lines are covered by tests ✅ ✅ All tests successful. No failed tests found. @@ Coverage Diff @@
## main #747 +/- ##
=======================================
Coverage 97.91% 97.91%
=======================================
Files 444 444
Lines 35403 35416 +13
=======================================
+ Hits 34664 34677 +13
Misses 739 739
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Swatinem
approved these changes
Sep 30, 2024
services/tests/test_test_results.py
Outdated
Comment on lines
265
to
268
| if private: | ||
| repo.private = True | ||
| else: | ||
| repo.private = False |
There was a problem hiding this comment.
Suggested change
| if private: | |
| repo.private = True | |
| else: | |
| repo.private = False | |
| repo.private = private |
or maybe even put that into the RepositoryFactory invocation?
tasks/test_results_finisher.py
Outdated
|
|
||
| matching_flakes = list( | ||
| db_session.query(Flake) | ||
| .filter( # type:ignore |
There was a problem hiding this comment.
Suggested change
| .filter( # type:ignore | |
| .filter( |
tasks/test_results_finisher.py
Outdated
Comment on lines
412
to
413
| for flake in matching_flakes: | ||
| flaky_test_ids[flake.testid] = FlakeInfo(flake.fail_count, flake.count) |
There was a problem hiding this comment.
Suggested change
| for flake in matching_flakes: | |
| flaky_test_ids[flake.testid] = FlakeInfo(flake.fail_count, flake.count) | |
| flaky_test_ids = {flake.testid: FlakeInfo(flake.fail_count, flake.count) for flake in matching_flakes} |
ajay-sentry
reviewed
Oct 7, 2024
services/test_results.py
Outdated
| identifier=repoid, default=False | ||
| ) and read_yaml_field(commit_yaml, ("test_analytics", "flake_detection"), True) | ||
| def should_do_flaky_detection(repo: Repository, commit_yaml: UserYaml) -> bool: | ||
| should_config = read_yaml_field( |
There was a problem hiding this comment.
has_flakey_configured instead of should_config imo
ajay-sentry
reviewed
Oct 7, 2024
services/test_results.py
Outdated
| should_feature = FLAKY_TEST_DETECTION.check_value( | ||
| identifier=repo.repoid, default=True | ||
| ) | ||
| should_plan = not_private_and_free_or_team(repo) |
There was a problem hiding this comment.
has_valid_plan_or_repo instead of should_plan imo
ajay-sentry
reviewed
Oct 7, 2024
services/test_results.py
Outdated
| should_config = read_yaml_field( | ||
| commit_yaml, ("test_analytics", "flake_detection"), True | ||
| ) | ||
| should_feature = FLAKY_TEST_DETECTION.check_value( |
There was a problem hiding this comment.
feature_enabled instead of should_feature imo
- remove flaky_shadow_mode feature flag
- simplify should_{write,read}_flaky_detection functions to
general should_do_flake_detection that checks whether to do either
- add check to make sure repo is not private and on a free plan or
team plan
- the new logic is:
- if a user has explicitly set flake_detection to false in their
yaml we don't do flake detection
- else:
- if a user is part of the feature flag we do flake detection
- else if the repo is private then we require them to be on a
pro plan
- else we do flake detection
e1f9b74 to
07c7b2a
Compare
|
✅ All tests successful. No failed tests were found. 📣 Thoughts on this report? Let Codecov know! | Powered by Codecov |
6BFD
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
general should_do_flake_detection that checks whether to do either
team plan
yaml we don't do flake detection
pro plan