Vahe1994/SpQR

This is stars and forks stats for /Vahe1994/SpQR repository. As of 12 May, 2024 this repository has 451 stars and 33 forks.

SpQR model compression Note: This repository contains quantization algorithm and the model evaluation code for SpQR method for LLM compression; The efficient inference code will be added soon. It accompanies the research paper "SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression" . Installation Packages To run SpQR with falcon make sure that you have torch>=2.0.0 with CUDA support. Install packages from requirements.txt: pip install -r requirements.txt Note: the results...