aicoolies logo

AI Agent Red-Teaming and Evaluation Stack

$0/mo

Stress-test LLM applications against OWASP threats with security scanning, evaluation frameworks, and content safety models.

PurpleLlamaContent Safety Models
Free and open-source (custom Meta license)
Agentic RadarAgent Security Scanner
Free and open-source
DeepEvalLLM Unit Testing
Open-source Apache-2.0 framework; Confident AI offers Free and Starter entry points plus Business/Enterprise paths for hosted evals, observability, red teaming, and governance.
PromptfooRed-Teaming & Eval
Free open-source core; enterprise/security platform offerings under OpenAI-era Promptfoo positioning
TruLensExperiment Tracking
Free and open-source (MIT)
LangfuseProduction Observability
Hobby free / Core from $29/mo / Pro from $199/mo

What This Stack Does

PurpleLlama provides Meta's Llama Guard models for content safety classification and CodeShield for insecure code detection. Agentic Radar scans agent workflows for MCP vulnerabilities. DeepEval offers pytest-native LLM testing with 50+ metrics. Promptfoo adds CLI-first evaluation with red-teaming attack generation. TruLens provides experiment tracking with RAG Triad metrics. Langfuse delivers observability for production monitoring.

The Bottom Line

Deploy PurpleLlama for content safety, Agentic Radar for agent architecture auditing, DeepEval and Promptfoo for pre-deployment testing, TruLens for experiment tracking, and Langfuse for production observability. The stack covers the complete lifecycle from development testing to production monitoring.

Stack Overview

PurpleLlamaContent Safety Models
Pricing
Free and open-source (custom Meta license)
Open Source
Yes
Agentic RadarAgent Security Scanner
Pricing
Free and open-source
Open Source
Yes
DeepEvalLLM Unit Testing
Pricing
Open-source Apache-2.0 framework; Confident AI offers Free and Starter entry points plus Business/Enterprise paths for hosted evals, observability, red teaming, and governance.
Open Source
Yes
PromptfooRed-Teaming & Eval
Pricing
Free open-source core; enterprise/security platform offerings under OpenAI-era Promptfoo positioning
Open Source
Yes
TruLensExperiment Tracking
Pricing
Free and open-source (MIT)
Open Source
Yes
LangfuseProduction Observability
Pricing
Hobby free / Core from $29/mo / Pro from $199/mo
Open Source
Yes