GitHub · Where software is built

8000 GitHub · Where software is built

More Web Proxy on the site http://driver.im/

Labels Milestones New issue

Using Triton with PyTorch 2.0 for AMP training results in tensors containing inf values.

#35

· DrRyanHuang opened

on Feb 17, 2024

Always getting NaNs in long training

#33

· danbochman opened

on Dec 5, 2023

AMD ROCM versions

#32

· bennmann opened

on Aug 6, 2023

Convergence guarantees for Lion

#30

· xnchxy opened

on Jul 20, 2023

Do you have the actual weights trained from the paper?

#29

· tr1-ai opened

on Jun 25, 2023

Same amount of VRAM is taken as in AdamW

#22

· VCasecnikovs opened

on Apr 3, 2023

add an 8-bit version with bitsandbytes

#17

· lucidrains opened

on Mar 8, 2023

Learning rate scaling for distributed training?

#8

· RahulBhalley opened

on Feb 18, 2023

0