8000 Fix `conda search` for Python 3.9 by chenghlee · Pull Request #10542 · conda/conda · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Fix conda search for Python 3.9 #10542

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Mar 12, 2021
Merged

Fix conda search for Python 3.9 #10542

merged 1 commit into from
Mar 12, 2021

Conversation

chenghlee
Copy link
Contributor

Make _vendor.distro import absolute to avoid thread safety problems
that were seemingly introduced by change in Python 3.9.

Fixes #10490

@chenghlee
Fix conda search for Python 3.9
d15d6dd 
Make `_vendor.distro` import absolute to avoid thread safety problems
that were seemingly introduced by change in Python 3.9.

Fixes conda#10490
@chenghlee chenghlee requested a review from a team March 6, 2021 05:51
@anaconda-issue-bot anaconda-issue-bot added the cla-signed [bot] added once the contributor has signed the CLA label Mar 6, 2021
@dbast
Copy link
Member
dbast commented Mar 10, 2021

The Linux Python 3.9 ci container is available now https://hub.docker.com/r/continuumio/conda-ci-linux-64-python3.9

So the ci matrix can be extended :)

@chenghlee
Copy link
Contributor Author

Can you give more details on this please @chenghlee?

@mingwandroid: This relative import happens within a method (Context.os_distribution_name_version) that may not be invoked when conda starts and before Threads are spawned. When conda is running with multiple threads (e.g., when downloading and processing repodata.json to search for a package), more than one thread can (semi-)simultaneously invoke that method and its relative import.

In (C)Python < 3.9, such relative imports (presumably) have a locking mechanism that prevents the cannot import name 'id' from partially initialized module 'conda._vendor.distro' error reported in #10490 from happening. However, Python 3.9 changed so this locking mechanism no longer works.

I haven't yet been able to identify the Python changeset that changed relative import behavior, but I have been able to put together a simpler, fairly reproducible demonstration here: https://github.com/chenghlee/py3x-threads-import.

@mingwandroid
Copy link
Contributor

ISTM that such a lies level bug would be hitting projects all over the python ecosystem. Is that the case here?

@chenghlee
Copy link
Contributor Author

ISTM that such a lies level bug would be hitting projects all over the python ecosystem. Is that the case here?

bpo-41567 ("multiprocessing.Pool from concurrent threads failure on 3.9.0rc1") seems to be another example of this issue. That, in turn, may be affecting other projects; e.g., dask/distributed#4168.

Digging around a bit, the changeset that addressed bpo-35943 is what resulted in this behavior change in Python 3.9.

@chenghlee chenghlee mentioned this pull request Mar 11, 2021
Merged
@dbast
Copy link
Member
dbast commented Mar 12, 2021

+1 for the PR... we need to keep an eye on the topic itself independent of the PR.

@chenghlee chenghlee merged commit 4f7c7fe into conda:master Mar 12, 2021
@chenghlee chenghlee deleted the fix-search-for-py39 branch March 13, 2021 16:49
@github-actions
Copy link

Hi there, thank you for your contribution!

This pull request has been automatically locked because it has not had recent activity after being closed.

Please open a new issue or pull request if needed.

Thanks!

@github-actions github-actions bot added the locked [bot] locked due to inactivity label Mar 14, 2022
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 14, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@dbast dbast dbast approved these changes

Assignees
No one assigned
Labels
cla-signed [bot] added once the contributor has signed the CLA locked [bot] locked due to inactivity
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

conda search with Python 3.9 broken on Linux
4 participants
@chenghlee @dbast @mingwandroid @anaconda-issue-bot
0