8000 Don't fail workflow for expected individual benchmarks failures by anissa111 · Pull Request #499 · NCAR/geocat-comp · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Don't fail workflow for expected individual benchmarks failures #499

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Oct 31, 2023
Merged

Don't fail workflow for expected individual benchmarks failures #499

merged 2 commits into from
Oct 31, 2023

Conversation

anissa111
Copy link
Member
@anissa111 anissa111 commented Oct 31, 2023 •
edited
Loading

PR Summary

Closes #498

Adding || true to the end of the asv-run command makes sure that we won't accidentally cancel the run when asv fails any individual benchmark.

Also, moves up the earliest tag to start benchmarking since commits before v2023.02.0 fail on import

See on my fork for tag v2022.10.0, which we know will fail from these logs

  • This run fails, as our benchmark CI did
  • This run, with the addition of || true, passes and continues past the benchmark running step even though the benchmark "failed"

PR Checklist

General

  • Make an issue if one doesn't already exist
  • Link the issue this PR resolves by adding closes #XXX to the PR description where XXX is the number of the issue.
  • Add a brief summary of changes to docs/release-notes.rst in a relevant section for the next unreleased release. Possible sections include: Documentation, New Features, Bug Fixes, Internal Changes, Breaking Changes/Deprecated
  • Add appropriate labels to this PR
  • Make your changes in a forked repository rather than directly in this repo
  • Passes precommit. To set up on your local, run pre-commit install from the top level of the repository. To manually run pre-commits, use pre-commit run --all-files and re-add any changed files before committing again and pushing.
  • If needed, squash and merge PR commits into a single commit to clean up commit history

@anissa111 anissa111 added bug Something isn't working testing Issue related to testing scalability and performance Related to scalability, performance, or benchmarking labels Oct 31, 2023
philipc2
philipc2 approved these changes Oct 31, 2023
View reviewed changes
Copy link
Member
@philipc2 philipc2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

@anissa111 anissa111 merged commit 4af9f7b into NCAR:main Oct 31, 2023
@anissa111 anissa111 deleted the issue498 branch October 31, 2023 01:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@philipc2 philipc2 philipc2 approved these changes

@kafitzgerald kafitzgerald Awaiting requested review from kafitzgerald

Assignees
No one assigned
Labels
bug Something isn't working scalability and performance Related to scalability, performance, or benchmarking testing Issue related to testing
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Benchmark CI fails if any individual benchmark fails
2 participants
@anissa111 @philipc2
0