HazyResearch/flash-attention

This is stars and forks stats for /HazyResearch/flash-attention repository. As of 07 Dec, 2023 this repository has 7243 stars and 599 forks.

FlashAttention This repository provides the official implementation of FlashAttention and FlashAttention-2 from the following papers. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, Christopher Ré Paper: https://arxiv.org/abs/2205.14135 IEEE Spectrum article about our submission to the MLPerf 2.0 benchmark using FlashAttention. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning Tri Dao Paper:...