8000 Get CI Tests Passing by mlline00 · Pull Request #9842 · conda/conda · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Get CI Tests Passing #9842

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 17, 2020
Merged

Get CI Tests Passing #9842

merged 4 commits into from
Apr 17, 2020

Conversation

mlline00
Copy link
Contributor

Fix failing CI tests. Changes are scoped to the test code and CI configurations. The conda codebase is not modified.

Utilize a single PR to demonstrate CI tests pass prior to merge.

Disable the following tests (can be restored if deemed necessary/desirable):

  • Linux: conda-build 3.10. Conda-build has a dedicated repo, CI testing should be done there.
  • Windows: Python 2.7. The install steps fail and 2.7 is EOL. Disabled instead of working through the noise.

Issues of note:

  • Unicode Paths on Windows (no surprise, right). Several tests in test_create.py fall on their face when handling unicode paths on Windows - even under Python 3. In all cases, failure is external to conda, with libarchive frequently appearing in the stack traces. Address failures using the same approach as Python 2.7; specifically, construct prefixes using a restricted set of Unicode characters in the ascii range. A future iteration can re-enable the Unicode path tests on Windows.
  • activate_deactivate_package. This was historically available on c3i_test2 channel. Not anymore. Modify tests to build and consume the package from within conda source tree.

(Slightly) more detailed information is available in the commit descriptions.

Summary of changes
- Disable conda-build 3.10 tests. It is an old version and it should handle its own CI.
- activate_deactivate_package
Use robocopy on windows platform
- Integration Tests:
Build and use local activate_deactivate_package in integration tests (requires installing conda-build)
- python 2.7 integration tests
Multiple versions of pip were getting pulled into the test container. Having multiple pips caused test_conda_pip_interop_conda_editable_package to see unexpected link/unlink actions.  Utilizing a single version of pip in the tests addressed the problem.
- AppVeyor/Windows platform
Incomplete Unicode support within conda_package_handler. Create temporary package cache directories using a very restricted set of "unicode" characters on Windows. Additionally, set Python 3 restricted characters to the same as Python 2. I was unable to make any characters outside the ASCII range pass. I traced this behavior to the libarchive handling of tar.bz2 archives within conda_package_handling.  I confirmed the pure Python handling of these archives worked fine; i.e. conda-package-handling.tarball.libarchive_enabled == False
Newer versions of the image use Python 3.8 and the version of ruamel spec in utils\requirements.txt is not compatible with Python 3.8
- Disable 2.7 tests. Too much to bite off at this point...and since 2.7 is deprecated it may not matter
- Update conda-build to version 3.17 and build local activate_deactivate_package
- Retain line endings on bat files in the activate_deactivate_package
- Simplify path adjustment logic in activate_deactivate_package batch files. (This may be unnecessary as controlling the line endings may have fixed the issue)
- Update condarc tests to write files without duplicate keys.
@mlline00 mlline00 requested a review from a team as a code owner April 14, 2020 17:32
@cla-bot cla-bot bot added the cla-signed [bot] added once the contributor has signed the CLA label Apr 14, 2020
It was redirecting out to a file name 1.23 anyway
@kalefranz
Copy link
Contributor

@mlline00 Thank you so much for this. Very sincerely thank you.

Courtesy of @mcg1969: https://www.youtube.com/watch?v=51uE0IaXL1U

@kalefranz
Copy link
Contributor

In theory I agree about conda-build being a downstream project and therefore conda-build tests should belong in the conda-build repo. For historical reasons and historical coupling between the two projects (i.e. conda-build treating the entire conda code base as "public"), we've needed to test conda against conda-build to make sure a minor refactor somewhere doesn't break conda-build entirely. Hopefully we're beyond that at this point with conda-build following better patterns to isolate its dependencies on conda. We may need to bring the conda-build tests back here someday, but now might be a good opportunity to give it a go without them.

@kalefranz
Copy link
Contributor

@mingwandroid Any objection to disabling the appveyor py2.7 tests here?

@mingwandroid
Copy link
Contributor

I would rather we kept python 2 ticking along where possible, but am fully on-board with not doing that too!

@mlline00
Copy link
Contributor Author

yw.

I'll take a look at the AppVeyor python 2 tests when I have some free cycles.

Copy link
Contributor
@kalefranz kalefranz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mingwandroid I think I'm going to merge this PR as-is, with removing the win py27 testing for now. Agree it would be nice to get it back in, and if @mlline00 has some extra time to work on it, that would very much appreciated. Having passing tests though means we can actually start clearing up some of the PR backlog. I'll be extra careful with PR reviews for anything that might affect encodings or other sticky win py27 topics.

Again @mlline00, thank you so SO SO much for this contribution. 🙇‍♂️

@kalefranz kalefranz merged commit 07a113d into conda:master Apr 17, 2020
@mlline00 mlline00 deleted the conda_ci_fixes branch April 17, 2020 23:31
@kalefranz kalefranz added this to the 4.8.4 milestone Apr 26, 2020
@github-actions
Copy link

Hi there, thank you for your contribution to Conda!

This pull request has been automatically locked since it has not had recent activity after it was closed.

Please open a new issue or pull request if needed.

@github-actions github-actions bot added the locked [bot] locked due to inactivity label Aug 21, 2021
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 21, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
cla-signed [bot] added once the contributor has signed the CLA locked [bot] locked due to inactivity
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0