aicoolies logo
ps-fuzz logo

ps-fuzz

Prompt fuzzing tool for LLM security testing

Share
open-sourceOpen Source
Visit Website →

ps-fuzz by Prompt Security is a security testing tool with 680+ GitHub stars that fuzzes system prompts against dynamic LLM-based attack scenarios including jailbreaks, prompt injection, and data extraction attempts. It helps developers harden their GenAI applications by simulating adversarial attacks in a controlled environment, turning LLM security into a testable and reproducible quality gate.

We have a review for this tool

A detailed review by the aicoolies team — click to read

ps-fuzz automates the security testing of GenAI applications by subjecting system prompts to a comprehensive suite of attack simulations. The tool generates dynamic adversarial inputs covering jailbreak attempts, prompt injection techniques, data extraction attacks, and system prompt leakage scenarios. Each test produces a detailed report showing which attacks succeeded and which defenses held, giving developers concrete guidance on where to strengthen their prompt engineering.

The testing approach treats prompt security like traditional fuzzing: systematically exploring edge cases and attack surfaces that manual testing would miss. Teams can integrate ps-fuzz into CI/CD pipelines to run prompt security regression tests whenever system prompts are modified, catching regressions before they reach production. This transforms LLM security from an ad-hoc concern into a structured, automated quality gate.

With 680+ GitHub stars and project activity into 2026, ps-fuzz addresses the growing need for practical LLM security testing tools. As organizations deploy more AI-powered features, the attack surface for prompt injection and related vulnerabilities expands rapidly. The tool is developed by Prompt Security, a company focused on GenAI security, and serves both security researchers and application developers building with LLMs.

Pricing

Free and open-source

Platforms

Python CLI, CI/CD pipelines

Categories

Tags

Use Cases

Alternatives

PromptLayer logo

PromptLayer

Prompt registry, observability, and evaluation workflows for LLM applications.

PromptLayer is a prompt management, observability, and evaluation platform for LLM applications. Teams use its Prompt Registry, visual editor, request logs, Tables, evaluations, Tool Registry, and Skill Collections to version prompts, replay requests, compare variants, run datasets, and ship prompt changes without redeploying code. Pricing starts with Free $0 for 5 users and 2.5K requests/month, Pro $49/month, Team $500/month, and Enterprise custom.

freemium
Guardrails AI logo

Guardrails AI

Validate and structure LLM outputs with composable Guards

Guardrails AI is an open-source Python and JavaScript framework for validating and structuring LLM outputs using composable Guards built from a Hub of pre-built validators. It handles structured data extraction with Pydantic models, content safety checks including toxicity, PII detection, competitor mentions, and bias filtering, plus automatic re-prompting when validation fails. The Guardrails Hub offers dozens of validators from regex matching to hallucination detection via LLM judges.

free

NeMo Guardrails

Programmable safety rails for LLM applications

NeMo Guardrails is NVIDIA's open-source toolkit for adding programmable safety rails to LLM applications. It supports five guardrail types — input, dialog, retrieval, execution, and output rails — covering content safety, jailbreak detection, topic control, PII masking, hallucination detection, and fact-checking. The toolkit uses Colang, a domain-specific language for defining conversational constraints, and integrates with OpenAI, Azure, Anthropic, HuggingFace, and LangChain/LangGraph.

free

Related Tools

Agent Governance Toolkit logo

Agent Governance Toolkit

Microsoft’s public-preview runtime governance toolkit for policy, identity, sandboxing, audit, and MCP security around AI agents.

Agent Governance Toolkit is Microsoft’s MIT-licensed public-preview toolkit for governing AI agent runtimes. It adds policy enforcement, zero-trust identity, execution sandboxing, audit, reliability, and MCP security-gateway patterns around tool calls and autonomous actions, helping platform teams move beyond prompt-only guardrails while preserving architecture review requirements.

open-sourceOpen SourceTelemetry
Baz logo

Baz

Telemetry-aware AI code reviewer that checks how pull requests may affect real services.

Baz is an AI code-review platform focused on production-aware pull requests. Instead of only reading the diff, Baz connects code changes to application telemetry so reviewers can understand what endpoints, services, and runtime behavior may be affected. That makes it a useful complement to existing AI PR bots when the question is not just whether a change looks correct, but whether it could break a live system.

freemiumTelemetry
rampart

Rampart

Microsoft’s pytest-native red teaming framework for turning AI agent safety findings into CI tests.

RAMPART is an open-source Microsoft framework for safety and security testing of agentic AI applications. It brings red-team findings into a pytest-native workflow so teams can turn prompt injection, unsafe tool use, and behavioral boundary failures into repeatable regression tests. The strongest aicoolies angle is developer workflow: RAMPART makes agent safety part of CI/CD instead of a one-off security review.

open-sourceOpen Source
Statewright logo

Statewright

State-machine guardrails for controlling which tools AI coding agents can use at each phase.

Statewright is a guardrail layer for AI coding agents that uses explicit state machines to control what an agent can do at each stage of a workflow. Instead of relying only on prompt instructions, teams can model phases such as plan, implement, test, and review, then constrain tool access for clients like Claude Code, Codex, Cursor, opencode, and related MCP workflows.

open-sourceOpen Source
Magika logo

Magika

AI-powered file-type detection at Google scale

Open-source AI-powered file-type detection tool from Google that uses a custom deep-learning model under a few megabytes to identify more than 200 binary and textual content types in milliseconds, even on a single CPU. Magika ships as a CLI, Python package, JavaScript/TypeScript library, and an ONNX model, achieves around 99% accuracy on its test set, and is already used at Google scale across Gmail, Drive, and Safe Browsing as well as by VirusTotal and abuse.ch.

freeOpen Source
Trent AI logo

Trent AI

Agentic AI security posture management

Trent AI is a specialized security platform for agentic AI applications providing AI Security Posture Management that compounds with every development cycle. Scans, judges, mitigates, and evaluates AI agent security detecting threats traditional tools miss including prompt injection attacks, tool misuse, unintended autonomous actions, data exfiltration through agent chains, and privilege escalation. Offers continuous assessment with remediation plan execution through Claude Code.

paid

Used in Stacks

Comparisons