Promptfoo

Promptfoo is a CLI and library for evaluating and red-teaming LLM applications.

• Github: promptfoo/promptfoo
  • 15.8k stars
  • MIT license

• Declarative YAML config for prompt testing
• Assertion types: string matching → LLM-as-judge rubrics
• Red-teaming / security vulnerability scanning
• Model comparison across providers (OpenAI, Anthropic, Gemini, Llama, etc.)
• CI/CD integration for automated checks