What ps-fuzz Does
ps-fuzz, the Prompt Security Fuzzer, is an open-source security testing tool built by Prompt Security specifically for GenAI applications. Its core function is to assess the security of your application's system prompt by simulating various LLM-based attacks and reporting which ones succeed in breaking through your prompt's defenses. The tool then provides a security evaluation that guides developers in strengthening their system prompts iteratively.
Attack Types and Dynamic Fuzzing
The fuzzer supports 16 different attack types that cover the major categories of prompt-based vulnerabilities. These include AIM Jailbreak which probes ethical compliance through roleplay scenarios, DAN (Do Anything Now) jailbreaks that test resilience against unrestricted persona adoption, Amnesia attacks that attempt to make the LLM forget system instructions, Typoglycemia attacks that exploit text processing by omitting characters, and System Prompt Stealer attempts to extract internal configuration. Each attack type is dynamically tailored to your application's specific context.
What sets ps-fuzz apart from static prompt injection datasets is its dynamic testing approach. Rather than firing the same generic payloads at every application, the fuzzer reads your system prompt, understands its context and domain, and adapts its attack generation accordingly. This produces more realistic and meaningful test results because the attacks are contextually relevant to what your application actually does.
Provider Support and Modes
The tool supports 16 different LLM providers including OpenAI, Anthropic, Azure OpenAI, Google PaLM, Cohere, and many others. This LLM-agnostic design means teams can test against whichever provider they are deploying to production, ensuring the security evaluation reflects real-world behavior. Configuration is handled through environment variables for API keys, with a .env file option for convenience.
ps-fuzz offers both interactive and CLI modes. The interactive Playground mode is particularly valuable because it lets developers iterate on their system prompt in real time, running the fuzzer after each modification to see if hardening attempts actually improve security. The CLI mode supports automation and CI/CD integration with multi-threaded testing for faster execution. Custom benchmarks can be loaded from CSV files for organization-specific attack scenarios.
Reporting and Installation
Results are reported in three categories: Broken (attacks that the LLM succumbed to), Resilient (attacks the LLM resisted), and Errors (inconclusive results). This clear categorization makes it straightforward to identify which attack vectors your system prompt is vulnerable to and prioritize hardening efforts. The tool includes example system prompts of varying security strengths for benchmarking purposes.
Installation is simple via pip as a Python package. The extensible architecture allows anyone to contribute new attack types by following a straightforward pattern: create a new Python file in the attacks directory, implement a test class, and register it. The project actively encourages community contributions of novel attack techniques, maintaining a growing library of increasingly sophisticated test scenarios.
Limitations and Complementary Use
There are important limitations to understand. As with all prompt fuzzing tools, ps-fuzz tests a finite set of known attack patterns against the essentially infinite attack surface of natural language. A clean fuzzing report does not guarantee prompt security; it means the prompt resisted the specific attacks tested. The tool consumes LLM tokens during testing, which can add up during extensive fuzzing campaigns with multiple attack types and iterations.
ps-fuzz works well as a complement to runtime guardrail solutions like NeMo Guardrails or LLM Guard. While those tools provide real-time protection during inference, ps-fuzz operates at development time to strengthen the system prompt itself. Used together, they create a layered defense where the prompt is hardened against known attacks and runtime guardrails catch novel or zero-day attack patterns.
The Bottom Line
For the current state of LLM security tooling, ps-fuzz provides genuine value at zero cost. The fact that it dynamically adapts attacks rather than relying purely on static payloads puts it ahead of many alternatives, though it is worth noting that more comprehensive commercial platforms like Promptfoo offer deeper customization and broader vulnerability coverage. As a free first step in prompt security testing, ps-fuzz is hard to beat.
