DSNC-2173 : Move Windows CI to Azure #10176
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Enables python 2.7 & 3.7 unit tests for win-32 and win-64 on the vs2017-win2016 microsoft hosted agent. Integration tests are not enabled as this configuration identified two issues in the unit testing framework: 1 - python 2.7 win-64 tests do not work 2 - One test has an issue when the source tree is not on the C drive.
The Microsoft hosted image makes no guarantees about where the source is located. The current image checks it out onto the D drive.
Detected bug while migrating python 2.7 windows unit tests to Azure Pipelines. Previous CI platform ran windows unit tests in a 32-bit environment which does not exhibit this bug. Recall, python 2 has int and long types, while python 3 has only int. logic.py contains int specific logic which does properly handle longs. sys.maxsize is an int on win-32 platform, but a long on win-64. This change uses sys.maxint when running under python 2, and sys.maxisze otherwise.
Significant extension to pipeline setup - Improved logging and display in the UI - Create a junction on host to bring conda, source, and temp folder all onto the same path. There are some tests which use relative paths and require all three be on the same drive. A junction is used to allow spanning local disk volumes. - Current CI tests push python into the base conda environment on the build host. This fails miserably for python 2.7 32-bit. Using a named environment seems to work, but there are comments in the test suite warning against using anything other than the base python. - Test results are published directly in the Azure Pipelines UI - Code coverage reports are not posted since Azure Pipelines cannot aggregate multiple coverage reports from a single test run. - Set a 90 minute timeout as the longest running successful tests take 60-70 minutes
No need to fail tests because a script creates a Windows path with C:\ instead of c:\
Something was tripping a R6034 error: "An application has made an attempt to load the C runtime library incorrectly". Unable to sort out which packages were root cause. Only solution was to download and install a new miniconda 2.7 x86 on the host. Decided to disable the test instead.
…gher verbosity levels
Address persistent MemoryErrors on 32-bit platforms by splitting integration tests into two runs.
tests\data\conda_format_repo\win-64 subdir has multiple zlib versions in repodata.json. Since the index is a dict, python 2.7 tests might see either zlib version first. One versoin of zlib (1.2.11) has both a bzip and conda package, while the other (1.2.8) has only a bzip. The point of the test is to ensure conda packages are preferred over bzip, so the test always fails if zlib 1.2.8 happens to be the first key found while iterating the index. Python 3.7 preserves insertion order, so the tests is stable in that version.
…tack is updated, and reduce environment clutter
chenghlee
approved these changes
Aug 21, 2020
|
Hi there, thank you for your contribution to Conda! This pull request has been automatically locked since it has not had recent activity after it was closed. Please open a new issue or pull request if needed. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
So much thanks to @mlline00 for this work to translate our CI to azure pipelines.
I pruned the build matrix to:
The appveyor tests for py27 appear to have been for win32, so let me know if these are the correct targets or if we want more/less (py38 might need a little work to fix conflicts when creating the test environment).
The free tier for public azure projects allow for 10 concurrent jobs, with a 360 minute limit per job. This should help avoid timeouts we've seen on appveyor.