promptfoo

by promptfoo

9.9Kstars

870forks

33watchers

Updated 3 months ago

About

Promptfoo is a developer-friendly tool for testing and securing large language model (LLM) applications through automated evaluations and red teaming.

Test your prompts, agents, and RAGs. AI Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration.

Primary Use Case

This tool is primarily used by developers and security professionals to evaluate the performance and security of AI applications powered by LLMs. It enables users to conduct vulnerability scans and compare different models to ensure the reliability and safety of AI systems.

Key Features

Automated evaluations for prompts and models
Red teaming and vulnerability scanning for LLM applications
Side-by-side comparison of various AI models
CI/CD integration for automated checks
Local execution ensuring privacy of prompts
Support for multiple LLM APIs and programming languages
Generation of security vulnerability reports

Installation

Install and initialize project using: npx promptfoo@latest init

Usage

>_ npx promptfoo eval

Runs your first evaluation of prompts and models.

Security Frameworks

Reconnaissance

Initial Access

Execution

Persistence

Impact

Usage Insights

Can be chained with Metasploit for automated exploitation
Useful for continuous security monitoring in CI/CD pipelines
Encourage collaboration between developers and security teams for effective red teaming
Integrate with existing security tools to enhance vulnerability scanning capabilities
Leverage the tool's reporting features to improve security awareness across the organization