ps-fuzz, the Prompt Security Fuzzer, is an open-source security testing tool built by Prompt Security specifically for GenAI applications. Its core function is to assess the security of your application's system prompt by simulating various LLM-based attacks and reporting which ones succeed in breaking through your prompt's defenses. The tool then provides a security evaluation that guides developers in strengthening their system prompts iteratively.
The fuzzer supports 15 different attack types that cover the major categories of prompt-based vulnerabilities. These include AIM Jailbreak which probes ethical compliance through roleplay scenarios, DAN (Do Anything Now) jailbreaks that test resilience against unrestricted persona adoption, Amnesia attacks that attempt to make the LLM forget system instructions, Typoglycemia attacks that exploit text processing by omitting characters, and System Prompt Stealer attempts to extract internal configuration. Each attack type is dynamically tailored to your application's specific context.
What sets ps-fuzz apart from static prompt injection datasets is its dynamic testing approach. Rather than firing the same generic payloads at every application, the fuzzer reads your system prompt, understands its context and domain, and adapts its attack generation accordingly. This produces more realistic and meaningful test results because the attacks are contextually relevant to what your application actually does.
The tool supports 16 different LLM providers including OpenAI, Anthropic, Azure OpenAI, Google PaLM, Cohere, and many others. This LLM-agnostic design means teams can test against whichever provider they are deploying to production, ensuring the security evaluation reflects real-world behavior. Configuration is handled through environment variables for API keys, with a .env file option for convenience.
ps-fuzz offers both interactive and CLI modes. The interactive Playground mode is particularly valuable because it lets developers iterate on their system prompt in real time, running the fuzzer after each modification to see if hardening attempts actually improve security. The CLI mode supports automation and CI/CD integration with multi-threaded testing for faster execution. Custom benchmarks can be loaded from CSV files for organization-specific attack scenarios.
Results are reported in three categories: Broken (attacks that the LLM succumbed to), Resilient (attacks the LLM resisted), and Errors (inconclusive results). This clear categorization makes it straightforward to identify which attack vectors your system prompt is vulnerable to and prioritize hardening efforts. The tool includes example system prompts of varying security strengths for benchmarking purposes.
Installation is simple via pip as a Python package. The extensible architecture allows anyone to contribute new attack types by following a straightforward pattern: create a new Python file in the attacks directory, implement a test class, and register it. The project actively encourages community contributions of novel attack techniques, maintaining a growing library of increasingly sophisticated test scenarios.