EleutherAI/gpt-neox

This is stars and forks stats for /EleutherAI/gpt-neox repository. As of 03 May, 2024 this repository has 6078 stars and 872 forks.

GPT-NeoX This repository records EleutherAI's library for training large-scale language models on GPUs. Our current framework is based on NVIDIA's Megatron Language Model and has been augmented with techniques from DeepSpeed as well as some novel optimizations. We aim to make this repo a centralized and accessible place to gather techniques for training large-scale autoregressive language models, and accelerate research into large-scale training. This library is in widespread use in academic, industry,...