-
FlashAttention never constructs a full attention matrix for the calculations
-
ALIBI adds a linear basis function to the attention calculation, which enables LLMs to extrapolate to longer context lengths than it was trained on
-
bf16 has the same size as fp16, but the same range as fp32 (Google Brain) https://pytorch.org/docs/stable/generated/torch.Tensor.bfloat16.html
-
Notifications
You must be signed in to change notification settings - Fork 0
License
umitkacar/LLMs-tricks
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
No description, website, or topics provided.
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published