opendatahub-io/caikit-tgis-serving

This is stars and forks stats for /opendatahub-io/caikit-tgis-serving repository. As of 07 May, 2024 this repository has 0 stars and 24 forks.

Caikit-TGIS-Serving Caikit-TGIS-Serving is a combined image that allows data scientists to perform Large Learning Model (LLM) inference. The Caikit-TGIS-Serving stack consists of these components: Text Generation Inference Server (TGIS): The serving backend that loads the models and provides the inference engine. Caikit: A wrapper layer that handles the lifecycle of the TGIS process, provides the inference endpoints, and has modules to handle different model types. Caikit-nlp: The Caikit module that...