Backbone
🍀 Pytorch implementation of various Attention Mechanisms, MLP, Re-parameter, Convolution, which is helpful to further understand papers.⭐⭐⭐
Two simple and effective designs of vision transformer, which is on par with the Swin transformer
QuadTree Attention for Vision Transformers (ICLR2022)
[ICLR2022] official implementation of UniFormer
[NeurIPS 2021] [T-PAMI] DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification
[ICLR 2022] "As-ViT: Auto-scaling Vision Transformers without Training" by Wuyang Chen, Wei Huang, Xianzhi Du, Xiaodan Song, Zhangyang Wang, Denny Zhou
Implementation of Memorizing Transformers (ICLR 2022), attention net augmented with indexing and retrieval of memories using approximate nearest neighbors, in Pytorch
Implementation of Gated State Spaces, from the paper "Long Range Language Modeling via Gated State Spaces", in Pytorch
[ICLR 2023 Spotlight] Vision Transformer Adapter for Dense Predictions
EVA Series: Visual Representation Fantasies from BAAI