8000 Fix deadlock at shutdown with python 3.9 by bdraco · Pull Request #52613 · home-assistant/core · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Fix deadlock at shutdown with python 3.9 #52613

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 7, 2021

Conversation

bdraco
Copy link
Member
@bdraco bdraco commented Jul 7, 2021

Proposed change

logged_shutdown happened and then the shutdown (which blocks or worse deadlocks) happened when we call close on the loop, but it wasn't noticeable until python 3.9 made a change where thread pool executor threads are no longer daemonic in python/cpython#19149 which means we deadlock at shutdown if a background process is still running if it won't die on its own.

Noticed while troubleshooting nmap in #52565 (comment)

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:
  • Link to documentation pull request:

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • The code has been formatted using Black (black --fast homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

  • No score or internal
  • 🥈 Silver
  • 🥇 Gold
  • 🏆 Platinum

To help with the load of incoming pull requests:

@probot-home-assistant probot-home-assistant bot added core small-pr PRs with less than 30 lines. labels Jul 7, 2021
@bdraco bdraco changed the title Fix double executor shutdown Fix deadlock at shutdown with python 3.9 Jul 7, 2021
@bdraco
Copy link
Member Author
bdraco commented Jul 7, 2021

Tested with both python 3.8 and python 3.9.

Verified roomba reconnect loop, samsung tv websocket, and nmap no longer block shutdown on 3.9

@@ -73,16 +73,6 @@ def new_event_loop(self) -> asyncio.AbstractEventLoop:
loop.set_default_executor = warn_use( # type: ignore
loop.set_default_executor, "sets default executor on the event loop"
)

# Shut down executor when we shut down loop
orig_close = loop.close
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

close already shuts down the executor. At least it appears to have done so since cpy3.5+

@pvizeli pvizeli added this to the 2021.7.0 milestone Jul 7, 2021
@frenck
Copy link
Member
frenck commented Jul 7, 2021

Tested locally with python 3.8.5 & 3.9.6, thanks @bdraco 👍

@frenck frenck merged commit 1ba5c1c into home-assistant:dev Jul 7, 2021
@github-actions github-actions bot locked and limited conversation to collaborators Jul 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants
0