promptfoo/promptfoo

This is stars and forks stats for /promptfoo/promptfoo repository. As of 08 May, 2024 this repository has 1127 stars and 39 forks.

promptfoo: test your prompts promptfoo is a tool for testing and evaluating LLM output quality. With promptfoo, you can: Systematically test prompts & models against predefined test cases Evaluate quality and catch regressions by comparing LLM outputs side-by-side Speed up evaluations with caching and concurrency Score outputs automatically by defining test cases Use as a CLI, library, or in CI/CD Use OpenAI, Anthropic, Azure, Google, open-source models like Llama, or integrate custom API providers...