8000 Backpropagation not working on KL divergence loss function due to data type mismatch · Issue #80158 · pytorch/pytorch · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Backpropagation not working on KL divergence loss function due to data type mismatch #80158

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
padmaksha18 opened this issue Jun 23, 2022 · 2 comments
Closed

Backpropagation not working on KL divergence loss function due to data type mismatch #80158

padmaksha18 opened this issue Jun 23, 2022 · 2 comments
Assignees
@lezcano
Labels
actionable module: loss Problem is related to loss function module: nn Related to torch.nn triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@padmaksha18
Copy link
padmaksha18 commented Jun 23, 2022 •
edited by pytorch-bot bot
Loading

🐛 Describe the bug

Backpropagation not working on KL divergence loss function due to data type mismatch

kl_loss = nn.KLDivLoss(reduction="batchmean")

input = torch.log_softmax(torch.randn(3, 5, requires_grad=True), dim=1)

target = torch.softmax(torch.rand(3, 5), dim=1).double()

output = kl_loss(input, target)

output.backward()

# RuntimeError: Found dtype Float but expected Double

Versions

Using Pytorch 1.2 on Google Colab

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are
@jbschlosser
Copy link
Contributor

This discrepancy between forward and backward has surfaced before - I think #69212 will fix this (cc @lezcano)

@jbschlosser jbschlosser added module: nn Related to torch.nn module: loss Problem is related to loss function triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module labels Jun 24, 2022
@lezcano
Copy link
Collaborator
lezcano commented Jun 24, 2022

I'll put up a PR fixing all this, making kl_div a composite, next week.

@lezcano lezcano mentioned this issue Jun 27, 2022
Closed
lezcano added a commit that referenced this issue Jun 27, 2022
@lezcano
Make kl_div a composite function.
43bc896 
Fixes #80158
Fixes #78867
Fixes #69230

Supersedes #79007
Supersedes #69212

[ghstack-poisoned]
lezcano added a commit that referenced this issue Jun 27, 2022
@lezcano
Update on "Make kl_div a composite function."
0b2edf9 
Benchmarks: #80334 (comment)

Fixes #80158
Fixes #78867
Fixes #69230

Supersedes
10000
 #79007
Supersedes #69212
Supersedes #19659

[ghstack-poisoned]
lezcano added a commit that referenced this issue Jun 27, 2022
@lezcano
Make kl_div a composite function.
2a6b935 
Fixes #80158
Fixes #78867
Fixes #69230

Supersedes #79007
Supersedes #69212

ghstack-source-id: 9d28b23
Pull Request resolved: #80334
lezcano added a commit that referenced this issue Jun 28, 2022
@lezcano
Update on "Make kl_div a composite function."
f8d2e87 
Benchmarks: #80334 (comment)

Fixes #80158
Fixes #78867
Fixes #69230

Supersedes #79007
Supersedes #69212
Supersedes #19659

[ghstack-poisoned]
lezcano added a commit that referenced this issue Jun 28, 2022
@lezcano
Make kl_div a composite function.
6c5ff4f 
Fixes #80158
Fixes #78867
Fixes #69230

Supersedes #79007
Supersedes #69212

ghstack-source-id: c25d0a0
Pull Request resolved: #80334
lezcano added a commit that referenced this issue Jun 30, 2022
@lezcano
Update base for Update on "Make kl_div a composite function."
7cb77ae 
Benchmarks: #80334 (comment)

Fixes #80158
Fixes #78867
Fixes #69230

Supersedes #79007
Supersedes #69212
Supersedes #19659

[ghstack-poisoned]
lezcano added a commit that referenced this issue Jun 30, 2022
@lezcano
Make kl_div a composite function.
048b491 
Fixes #80158
Fixes #78867
Fixes #69230

Supersedes #79007
Supersedes #69212

ghstack-source-id: 9b824cc
Pull Request resolved: #80334
lezcano added a commit that referenced this issue Jun 30, 2022
@lezcano
Update on "Make kl_div a composite function."
d435965 
Benchmarks: #80334 (comment)

Fixes #80158
Fixes #78867
Fixes #69230

Supersedes #79007
Supersedes #69212
Supersedes #19659

[ghstack-poisoned]
lezcano added a commit that referenced this issue Jul 4, 2022
@lezcano
Update base for Update on "Make kl_div a composite function."
ebb8eef 
Benchmarks: #80334 (comment)

Fixes #80158
Fixes #78867
Fixes #69230

Supersedes #79007
Supersedes #69212
Supersedes #19659

[ghstack-poisoned]
lezcano added a commit that referenced this issue Jul 4, 2022
@lezcano
Update on "Make kl_div a composite function."
6d315b8 
Benchmarks: #80334 (comment)

Fixes #80158
Fixes #78867
Fixes #69230

Supersedes #79007
Supersedes #69212
Supersedes #19659

[ghstack-poisoned]
lezcano added a commit that referenced this issue Jul 4, 2022
@lezcano
Make kl_div a composite function.
ef57f49 
Fixes #80158
Fixes #78867
Fixes #69230

Supersedes #79007
Supersedes #69212

ghstack-source-id: b02c11e
Pull Request resolved: #80334
facebook-github-bot pushed a commit that referenced this issue Jul 6, 2022
@lezcano @facebook-github-bot
Make kl_div a composite function. (#80334) (#80334)
a9619cf 
Summary:
Benchmarks: #80334 (comment)

Fixes #80158
Fixes #78867
Fixes #69230

Supersedes #79007
Supersedes #69212
Supersedes #19659

Pull Request resolved: #80334
Approved by: https://github.com/ezyang

Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/828c787ea98da39eb786925eedcb8527aae07153

Reviewed By: mehtanirav

Differential Revision: D37604775

Pulled By: mehtanirav

fbshipit-source-id: b188d47df5a3a820e5c15d9ce18b1a2c3f31f287
lezcano added a commit that referenced this issue Jul 12, 2022
@lezcano
Make kl_div a composite function.
d7edb33 
Fixes #80158
Fixes #78867
Fixes #69230

Supersedes #79007
Supersedes #69212

ghstack-source-id: 8dd9be6
Pull Request resolved: #80334
lezcano added a commit that referenced this issue Jul 12, 2022
@lezcano
Update base for Update on "Make kl_div a composite function."
16b4964 
Benchmarks: #80334 (comment)

Fixes #80158
Fixes #78867
Fixes #69230

Supersedes #79007
Supersedes #69212
Supersedes #19659

[ghstack-poisoned]
lezcano added a commit that referenced this issue Jul 12, 2022
@lezcano
Update on "Make kl_div a composite function."
cab416d 
Benchmarks: #80334 (comment)

Fixes #80158
Fixes #78867
Fixes #69230

Supersedes #79007
Supersedes #69212
Supersedes #19659

[ghstack-poisoned]
lezcano added a commit that referenced this issue Jul 13, 2022
@lezcano
Update base for Update on "Make kl_div a composite function."
b05bb94 
Benchmarks: #80334 (comment)

Fixes #80158
Fixes #78867
Fixes #69230

Supersedes #79007
Supersedes #69212
Supersedes #19659

[ghstack-poisoned]
lezcano added a commit that referenced this issue Jul 13, 2022
@lezcano
Update on "Make kl_div a composite function."
4d66b0c 
Benchmarks: #80334 (comment)

Fixes #80158
Fixes #78867
Fixes #69230

Supersedes #79007
Supersedes #69212
Supersedes #19659

[ghstack-poisoned]
lezcano added a commit that referenced this issue Jul 13, 2022
@lezcano
Make kl_div a composite function.
f00943c 
Fixes #80158
Fixes #78867
Fixes #69230

Supersedes #79007
Supersedes #69212

ghstack-source-id: 2e71ccd
Pull Request resolved: #80334
pytorchmergebot pushed a commit that referenced this issue Jul 13, 2022
@lezcano @pytorchmergebot
Make kl_div a composite function. (#80334)
b5b9db9 
Benchmarks: #80334 (comment)

Fixes #80158
Fixes #78867
Fixes #69230

Supersedes #79007
Supersedes #69212
Supersedes #19659
Pull Request resolved: #80334
Approved by: https://github.com/ezyang
facebook-github-bot pushed a commit that referenced this issue Jul 14, 2022
@lezcano @facebook-github-bot
Make kl_div a composite function. (#80334) (#80334)
491f6e5 
Summary:
Benchmarks: #80334 (comment)

Fixes #80158
Fixes #78867
Fixes #69230

Supersedes #79007
Supersedes #69212
Supersedes #19659

Pull Request resolved: #80334
Approved by: https://github.com/ezyang

Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/b5b9db9f844f4f100651c6afa57124fa5851edec

Reviewed By: DanilBaibak

Differential Revision: D37847477

Pulled By: DanilBaibak

fbshipit-source-id: a04919bbd2b746c30c654b971efcf76ef27ac5a6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lezcano lezcano

Labels
actionable module: loss Problem is related to loss function module: nn Related to torch.nn triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@lezcano @albanD @padmaksha18 @jbschlosser
0