qwopqwop200/GPTQ-for-LLaMa

This is stars and forks stats for /qwopqwop200/GPTQ-for-LLaMa repository. As of 28 Apr, 2024 this repository has 2651 stars and 429 forks.

GPTQ-for-LLaMA I am currently focusing on AutoGPTQ and recommend using AutoGPTQ instead of GPTQ for Llama. 4 bits quantization of LLaMA using GPTQ GPTQ is SOTA one-shot weight quantization method It can be used universally, but it is not the fastest and only supports linux. Triton only supports Linux, so if you are a Windows user, please use WSL2. News or Update AutoGPTQ-triton, a packaged version of GPTQ with triton, has been integrated into AutoGPTQ. Result LLaMA-7B(click me) LLaMA-7B Bits group-size memory(MiB) Wikitext2 checkpoint...