AlibabaResearch/flash-llm

This is stars and forks stats for /AlibabaResearch/flash-llm repository. As of 06 May, 2024 this repository has 73 stars and 9 forks.

Flash-LLM Flash-LLM is a large language model (LLM) inference acceleration library for unstructured model pruning. Flash-LLM mainly contains efficient GPU code based on Tensor-Core-accelerated unstructured sparse matrix multiplication calculations, which can effectively accelerate the performance of common matrix calculations in LLM. With Flash-LLM, the pruned LLM models can be deployed onto GPUs with less memory consumption and can be executed more efficiently. Currently, the code has been evaluated...