8000 reduce logging severity for set_to_none by ffuuugor · Pull Request #471 · pytorch/opacus · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

reduce logging severity for set_to_none #471

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

reduce logging severity for set_to_none #471

wants to merge 1 commit into from

Conversation

ffuuugor
Copy link
Contributor

We want to warn users of an unexpected behaviour with set_to_none flag. Normally, both nn.Module and Optimizer let clients choose whether they want to remove the .grad attribute altogether or just set it to None.
We, on the other hand, don't want to remove the attributes - it's more convenient to assume the .grad_sample attribute is always present. It's not an absolute requirement, but we did that historically and I don't see a case for changing it now.

However, default value for set_to_none is False, meaning most users are getting annoying logging notifications on every training step.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 16, 2022
@facebook-github-bot
Copy link
Contributor

@ffuuugor has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

alexandresablayrolles
alexandresablayrolles approved these changes Aug 17, 2022
View reviewed changes
Copy link
Contributor
@alexandresablayrolles alexandresablayrolles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much needed, thanks! I ended up manually adding set_to_none=False in most of my code...
QQ: is there a way to make this message appear only once?

@ffuuugor
Copy link
Contributor Author

QQ: is there a way to make this message appear only once?

I mean, we can always set a flag for this on a module/optimizer level - do you think it's worth it?

@karthikprasad karthikprasad deleted the ffuuugor_log branch August 23, 2022 21:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@karthikprasad karthikprasad karthikprasad approved these changes

@alexandresablayrolles alexandresablayrolles alexandresablayrolles approved these changes

@ashkan-software ashkan-software Awaiting requested review from ashkan-software

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@ffuuugor @facebook-github-bot @karthikprasad @alexandresablayrolles
0