Threat model and layered red-team design
This stack divides automated red teaming by attack depth. garak supplies broad probes, PyRIT orchestrates adaptive multi-turn attacks, DeepTeam organizes risk-driven conversational scenarios, Agentic Security connects security testing to developer workflows, and iFixAi extends scanning toward continuously changing LLM and agent behavior. The layers are complementary: a wide probe finds obvious exposure, while adaptive attacks test whether the system can be steered around controls across several turns or tools.
Start with a threat model tied to the actual application, data, tools, and deployment boundary. Map risks such as prompt injection, jailbreaks, sensitive-data disclosure, unsafe tool use, excessive agency, denial of service, and insecure output handling to owned controls and test cases. A generic scanner result is not evidence that every application path was exercised. The pipeline must name what is in scope, what is mocked, and which consequences require a release block.
Run broad probes with garak
Use garak as the fast coverage layer against a stable test endpoint. Select probes relevant to the model and application, fix seeds where supported, capture configuration and target versions, and separate scanner errors from model findings. Rate limits, content filters, and provider outages should be recorded explicitly because they can reduce coverage without producing a clear failure. Store raw evidence securely and publish only a minimized report suitable for engineering review.
Treat probe results as candidates that need triage. Reproduce severe findings, group duplicates by root cause, and label whether the weakness belongs to the model, system prompt, retrieval layer, tool boundary, or output handler. A large count of low-quality hits should not outweigh one reproducible path to sensitive data or an unauthorized action. Baseline known findings so CI highlights new exposure without normalizing accepted risk indefinitely.
Exercise adaptive attacks with PyRIT and DeepTeam
PyRIT is the multi-turn orchestration layer for attacks that adapt to prior responses, transform prompts, and pursue an objective across a conversation. Build scenarios around real assets and policies, use isolated test credentials, and cap turns, spend, and concurrency. Adaptive attacks can generate harmful or sensitive content, so logs require restricted access and retention rules. Never point the harness at production actions unless the environment and approvals are explicitly designed for security testing.
DeepTeam adds risk-prioritized conversational tests and repeatable attack cases that can be aligned to the application's threat model. Use it to maintain scenario suites by risk family, expected refusal or containment behavior, and evidence requirements. Review scenario quality after model or policy changes because an old attack may stop being meaningful even when it still returns a score. Human security review remains necessary for novel chains and business-specific abuse.
Automate the release gate
Agentic Security should run a small, deterministic subset on pull requests and a broader suite on schedules or release candidates. iFixAi can extend continuous scanning across agent and safety behaviors as prompts, tools, and models change. Keep the CI job isolated from production data, restrict network access, use least-privilege test accounts, and ensure every potentially mutating tool has a sandbox or mock. Security tests must not become a path for the test system itself to cause harm.
The gate should block on reproducible high-severity paths, missing coverage for a changed trust boundary, untriaged new findings, or disabled controls. Flaky or judge-based signals can open a review requirement instead of an automatic hard fail until calibrated. Every exception needs a risk owner, rationale, expiry, and compensating control. Upload sanitized summaries to CI while keeping detailed attack transcripts in a restricted evidence store.
Triage, remediation, and operating cost
Route each confirmed finding to the control that can actually reduce risk: input isolation, retrieval filtering, policy checks, tool allowlists, argument validation, approval gates, output encoding, secret management, or runtime containment. Prompt edits alone rarely close tool or data-boundary vulnerabilities. After remediation, add the exact exploit path to a regression suite and rerun adjacent scenarios to ensure the fix did not merely block one string while leaving the underlying capability exposed.
The budget varies with model calls, attack volume, analyst triage, isolated infrastructure, and evidence retention, even when the core tools are open source. Begin with the highest-impact application paths and a small suite that engineers trust. A static internal summarizer may need a narrower program than an autonomous agent with credentials. The full stack is justified when releases frequently change prompts, models, tools, or permissions and security evidence must move at the same pace.