Quantized Attention achieves speedup of 2-3x and 3-5x compared to FlashAttention and xformers, without lossing end-to-end metrics across language, image, and video models.
Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model
SpargeAttention: A training-free sparse attention that can accelerate any model inference.
[NeurIPS 2024] AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising
⚡️ A fast and flexible PyTorch inference server that runs locally, on any cloud or AI HW.
This is the official repo of "QuickLLaMA: Query-aware Inference Acceleration for Large Language Models"
DepthStream Accelerator: A TensorRT-optimized monocular depth estimation tool with ROS2 integration for C++. It offers high-speed, accurate depth perception, perfect for real-time applications in robotics, autonomous vehicles, and interactive 3D environments.
The official repo for “Multi-Cue Adaptive Visual Token Pruning for Large Vision-Language Models”.
Code for paper "TLEE: Temporal-wise and Layer-wise Early Exiting Network for Efficient Video Recognition on Edge Devices"
Convert and run scikit-learn MLPs on Rockchip NPU.
Code for paper "Deep Reinforcement Learning based Multi-task Automated Channel Pruning for DNNs"
Code for paper "Joint Adaptive Resolution Selection and Conditional Early Exiting for Efficient Video Recognition on Edge Devices"
Modified inference engine for quantized convolution using product quantization
Add a description, image, and links to the inference-acceleration topic page so that developers can more easily learn about it.
To associate your repository with the inference-acceleration topic, visit your repo's landing page and select "manage topics."