SHI-Labs/Neighborhood-Attention-Transformer

This is stars and forks stats for /SHI-Labs/Neighborhood-Attention-Transformer repository. As of 07 May, 2024 this repository has 900 stars and 80 forks.

Neighborhood Attention Transformers Powerful hierarchical vision transformers based on sliding window attention. Neighborhood Attention (NA, local attention) was introduced in our original paper, NAT, and runs efficiently with our extension to PyTorch, NATTEN. We recently introduced a new model, DiNAT, which extends NA by dilating neighborhoods (DiNA, sparse global attention, a.k.a. dilated local attention). Combinations of NA/DiNA are capable of preserving locality, maintaining translational equivariance, expanding...