8000 Floating Point Exception on PyTorch nightlies · Issue #17029 · pytorch/pytorch · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Floating Point Exception on PyTorch nightlies #17029

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
syed-ahmed opened this issue Feb 12, 2019 · 6 comments
Closed

Floating Point Exception on PyTorch nightlies #17029

syed-ahmed opened this issue Feb 12, 2019 · 6 comments

Comments

@syed-ahmed
Copy link
Collaborator

🐛 Bug

We are seeing a floating point exception in the nightly containers but not in v1.0.1 container. I believe there is a mkl-dnn patch that needs to be on master?

@soumith
@ngimel
Copy link
Collaborator
ngimel commented Feb 13, 2019

This is fixed in the latest mkl-dnn https://github.com/intel/mkl-dnn/blob/b88908909240bdab10f1cb7c256caa264fc15067/src/cpu/xbyak/xbyak_util.h#L216, but not in the mkl-dnn hash pytorch is using https://github.com/intel/mkl-dnn/blob/830a10059a018cd2634d94195140cf2d8790a75a/src/cpu/xbyak/xbyak_util.h#L186.
mkl-dnn comes as ideep submodule, and ideep is not updated to point to the latest mkl-dnn, so it's a bit of a mess.

@ngimel
Copy link
Collaborator
ngimel commented Feb 13, 2019

#16183 is related

@soumith
Copy link
Member
soumith commented Feb 14, 2019

@mingfeima can you help get this patch into a ideep/mkl-dnn release that pytorch uses. For v1.0.1, I manually patched the source before building binaries.

@mingfeima
Copy link
Collaborator

@soumith will do.

@Jianhui-Li
Copy link

@soumith We has done the internal validation and submit a patch for this. #17107

@soumith
Copy link
Member
soumith commented Feb 15, 2019

Thank you.
Fixed via #17107

@soumith soumith closed this as completed Feb 15, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@soumith @syed-ahmed @ngimel @mingfeima @Jianhui-Li
0