8000 Remove `isatty` wrapper function by Kludex · Pull Request #7814 · celery/celery · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Remove isatty wrapper function #7814

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 17, 2022
Merged

Remove isatty wrapper function #7814

merged 1 commit into from
Oct 17, 2022

Conversation

Kludex
Copy link
Contributor
@Kludex Kludex commented Oct 16, 2022

This function was added 11 years ago to avoid an issue with Python 2.6.

References:

@Kludex Kludex changed the title Remove isatty wrapper function Remove isatty wrapper function Oct 16, 2022
@codecov
Copy link
codecov bot commented Oct 16, 2022

Codecov Report

Base: 89.59% // Head: 89.35% // Decreases project coverage by -0.23% ⚠️

Coverage data is based on head (3acee41) compared to base (651095e).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7814      +/-   ##
==========================================
- Coverage   89.59%   89.35%   -0.24%     
==========================================
  Files         128      128              
  Lines       15880    15917      +37     
  Branches     2117     2117              
==========================================
- Hits        14228    14223       -5     
- Misses       1423     1465      +42     
  Partials      229      229              
Flag Coverage Δ
unittests 89.33% <100.00%> (-0.24%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
celery/platforms.py 96.48% <ø> (-0.53%) ⬇️
celery/app/log.py 97.65% <100.00%> (-0.02%) ⬇️
celery/apps/worker.py 63.37% <100.00%> (ø)
celery/utils/term.py 96.26% <100.00%> (-0.04%) ⬇️
celery/utils/threads.py 93.64% <0.00%> (-6.36%) ⬇️
celery/utils/log.py 86.18% <0.00%> (-3.55%) ⬇️
celery/concurrency/gevent.py 97.50% <0.00%> (-2.50%) ⬇️
celery/backends/mongodb.py 96.36% <0.00%> (-2.40%) ⬇️
celery/backends/cassandra.py 95.14% <0.00%> (-1.89%) ⬇️
celery/backends/cosmosdbsql.py 89.79% <0.00%> (-1.88%) ⬇️
... and 8 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@auvipy auvipy added this to the 5.3 milestone Oct 17, 2022
@auvipy auvipy merged commit 8b84b46 into celery:master Oct 17, 2022
@Kludex Kludex deleted the remove-isatty branch October 17, 2022 04:38
@grembo
Copy link
grembo commented Jan 12, 2025

@Kludex I'm currently investigating a problem report from a user running FreeBSD 14.2 and Python 3.11, which sounds pretty much like the problem the original workaround was supposed to prevent:

>> Jan 10 16:55:32 paperless paperless-worker[2499]:   File "/usr/local/lib/python3.11/site-packages/celery/apps/worker.py", line 135, in on_start
>> Jan 10 16:55:32 paperless paperless-worker[2499]:     self.emit_banner()
>> Jan 10 16:55:32 paperless paperless-worker[2499]:   File "/usr/local/lib/python3.11/site-packages/celery/apps/worker.py", line 164, in emit_banner
>> Jan 10 16:55:32 paperless paperless-worker[2499]:     use_image = term.supports_images()
>> Jan 10 16:55:32 paperless paperless-worker[2499]:                 ^^^^^^^^^^^^^^^^^^^^^^
>> Jan 10 16:55:32 paperless paperless-worker[2499]:   File "/usr/local/lib/python3.11/site-packages/celery/utils/term.py", line 165, in supports_images
>> Jan 10 16:55:32 paperless paperless-worker[2499]:     return sys.stdin.isatty() and ITERM_PROFILE
>> Jan 10 16:55:32 paperless paperless-worker[2499]:            ^^^^^^^^^^^^^^^^
>> Jan 10 16:55:32 paperless paperless-worker[2499]: AttributeError: 'NoneType' object has no attribute ‘isatty’  

This happens when starting the specific service from a start script on boot when (I assume) stdin is not available.

I didn't dig into the issue at hand and also did not look into workarounds, so for now I'm just leaving this comment for others.

@grembo
Copy link
grembo commented Jan 12, 2025

Ok, so I can confirm that celery-worker does not start without stdin being available, which used to work until this change. Not sure if it's a big deal, I do have one user though whose setup does not work because of it. This can be worked around outside of celery, but maybe it would make sense to make that part of the code more resilient again (read: not requiring standard input to determine if the terminal supports images).

@Kludex
Copy link
Contributor Author
Kludex commented Jan 12, 2025

@auvipy can you revert this? Sorry.

@Kludex
Copy link
Contributor Author
Kludex commented Jan 12, 2025

The reason why this change was introduced initially was for Python 2.6, but it worked for other systems.

@grembo
Copy link
grembo commented Jan 12, 2025

To test the issue from a shell (close stdin, start worker):

(exec 0<&-; /usr/local/bin/celery --app myapp worker)

@auvipy
Copy link
Member
auvipy commented Jan 12, 2025

I can try to revert this

@Kludex
Copy link
Contributor Author
Kludex commented Jan 12, 2025

Thanks.

@grembo
Copy link
grembo commented Jan 12, 2025

It seems a bit strange to me, that this check

def supports_images():
    return sys.stdin.isatty() and ITERM_PROFILE

happens on stdin (maybe it's because the check should work if either stdout or stderr are redirected?).

So instead of reverting the whole patch, only this specific place could be wrapped in a try/except block and return false in case of attribute error (using the logic of the previous function):

def supports_images():
    try:
        return sys.stdin.isatty() and ITERM_PROFILE
    except AttributeError:
        pass

I'm not an expert on celery though, so just sharing my thoughts here.

Thank you for your quick reaction on my initial comment, much appreciated.

MehrazRumman added a commit to MehrazRumman/celery that referenced this pull request Jan 14, 2025
auvipy added a commit that referenced this pull request Feb 13, 2025
* PR #7814 reverted

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* format changed

* formatted

* Update celery/utils/term.py

* lint fix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* lint fix

* tesst added for support images

* tesst added for support images

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* spacing fixed

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Asif Saif Uddin <auvipy@gmail.com>
Co-authored-by: Omer Katz <omer.katz@kcg.tech>
@auvipy
Copy link
Member
auvipy commented Feb 13, 2025

We manually reverted this with suggested improvement and additional tests #9494 . you are most welcome to try this and provide feedback.

@Nusnus
Copy link
Member
Nusnus commented Feb 13, 2025

@auvipy Can you please summarize what happened in terms of why this change was introduced, and why it was reverted (+minor modification)?
I’ll use it in the release change log to describe the change eventually.

@Nusnus
Copy link
Member
Nusnus commented Feb 13, 2025

Just noticed this 😂
CleanShot 2025-02-13 at 11 53 17

Unfortunately we’re not there yet haha:
CleanShot 2025-02-13 at 11 53 41

@grembo
Copy link
grembo commented Feb 13, 2025

@Nusnus What about just scrolling back on this very issue, e.g., #7814 (comment)

In short: The original commit removed a workaround that was thought not to be necessary anymore.
It was reverted, as parts of the workaround are actually still needed (and the remaining part is actually not really a workaround, but just how python works).

You could write:

Fix issue that was introduced by accident when removing a workaround, which stopped celery from starting when executed without stdin available.

@Nusnus
Copy link
Member
Nusnus commented Feb 13, 2025

@grembo

@Nusnus What about just scrolling back on this very issue, e.g., #7814 (comment)

I’d be happy just to scroll up if it would efficiently cross-reference the description with the PR comments of the revert PR and explain concisely what the bottom line was.

Overall, when someone reviews the release change log, it is not expected that they will study all of the change items to understand what happened. Providing the community with a concise and clear description is more efficient and will avoid confusion.
Lastly, as our release is extremely late, I’d appreciate any help in completing the checklists for the release, which include clarifications on each change since v5.4.0.

@Nusnus
Copy link
Member
Nusnus commented Feb 13, 2025

@grembo

Fix issue that was introduced by accident when removing a workaround, which stopped celery from starting when executed without stdin available.

That’s great actually! Thanks!
Can we TL;DR what was the issue?

@grembo
Copy link
grembo commented Feb 13, 2025

@Nusnus
The issue was that the code was written in a way that required stdin to exist, even though this isn't always the case, e.g., when being started from a service manager.

It used stdin to determine if the terminal supports certain features, but did not take into account, that there sometimes is no stdin for the above mentioned reasons.

@grembo
Copy link
grembo commented Feb 13, 2025

The comment #7814 (comment) demonstrates this:

def supports_images():
    return sys.stdin.isatty() and ITERM_PROFILE

So as far as I am concerned, the release note line could be:

Fix an issue that was introduced by accident when removing a workaround for sys.stdout.isatty that was not required anymore. This stopped celery from starting when executed in certain environments, as sys.stdin.isatty was used to determine if the terminal supports images without taking into the account the situation in which celery is started without stdin available, as it, e.g., can happen when being started by a service manager on boot.

@grembo
Copy link
grembo commented Feb 13, 2025

@Nusnus Hopefully the latest version above cuts it (I don't know much about the tests that were added etc. though, as I only reported the original problem)

@Nusnus
Copy link
Member
Nusnus commented Feb 16, 2025

Hey @grembo - thanks a ton!
This is great++ !

Really appreciate it 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants
0