Quantized Attention that achieves speedups of 2.1-3.1x and 2.7-5.1x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various models.
cuda triton attention quantization video-generation mlsys inference-acceleration efficient-attention llm
-
Updated
Feb 15, 2025 - Cuda