hpcaitech/EnergonAI

This is stars and forks stats for /hpcaitech/EnergonAI repository. As of 03 May, 2024 this repository has 624 stars and 92 forks.

Energon-AI A service framework for large-scale model inference, Energon-AI has the following characteristics: Parallelism for Large-scale Models: With tensor parallel operations, pipeline parallel wrapper, distributed checkpoint loading, and customized CUDA kernel, EnergonAI can enable efficient parallel inference for larges-scale models. Pre-built large models: There are pre-built implementation for popular models, such as OPT. It supports the cache technique for the generation task and distributed...