What Giskard is built to test
Giskard is best understood as an AI-agent evaluation and red-teaming system, not as a general application-observability dashboard. Its current documentation separates two products: Giskard Open Source, a free Python library aimed at developers and researchers, and Giskard Hub, an enterprise platform for collaborative and continuous testing. The common workflow is to describe or connect an agent, generate security or quality scenarios, run them against the system, judge the responses, and convert failures into repeatable tests. This orientation makes Giskard relevant before deployment and during controlled regression testing, especially when an ordinary unit-test suite cannot express conversational, adversarial, or domain-specific failure modes.
The security emphasis is concrete. Giskard documents attacks covering prompt injection, harmful or unauthorized advice, and adaptive multi-turn jailbreak behavior, mapped to categories including OWASP guidance for generative AI. Its Hub material describes 55 specialized probes across 11 vulnerability categories, security grades, severity-ranked findings, and conversation logs showing how an attack unfolded. Those are vendor-stated capabilities rather than an independent guarantee that every vulnerability will be found, so a buyer should treat the scan as one layer in a broader assurance program. Giskard is most compelling when the goal is to create evidence, remediation work, and regression suites—not to outsource the final security decision to one automated score.
Open-source workflow and the v3 transition
The open-source route is genuinely useful for code-first teams. Giskard’s current scan workflow wraps an agent as an asynchronous function, generates tailored scenarios from a plain-language description, and runs single-turn or multi-turn conversations with an LLM acting as generator and judge. Teams choose the model provider, can write custom checks, save generated scenarios as suites, and reuse the same suite in CI or against another model. The repository is active, Apache-2.0 licensed, and showed 5,507 GitHub stars in the live July 13, 2026 API snapshot. Local execution also gives engineering teams direct control over the application adapter, test definitions, and the data they intentionally pass into the evaluation flow.
Versioning deserves special attention. Giskard announced v3 as a rewrite for dynamic, multi-turn agent testing, while older v2 workflows used giskard.Model, giskard.scan(model, dataset), and RAGET-style dataset generation. The July 2026 documentation presents a redesigned v3 scan, yet the latest stable GitHub release visible in the same live check was v2.19.2, published July 6, 2026. That combination is not a reason to reject Giskard, but it is a procurement and implementation risk: teams should pin dependencies, confirm whether a required scanner or RAG feature is stable in their chosen branch, and budget for migration work. Copying examples between legacy, stable, and v3 documentation without checking the version can produce an avoidable integration failure.
What Giskard Hub adds for enterprise teams
Hub turns local tests into a managed quality program. Giskard’s official comparison lists centralized projects, shared datasets, multi-user collaboration, role-based access, scheduled evaluation runs, historical comparison dashboards, configurable alerts, and audit trails. The Hub SDK can define agents, generate or import test cases, launch evaluations and scans, schedule recurring runs, create tasks from failures, and preserve annotations or comments. This matters when evaluation is owned by more than one ML engineer: security, product, compliance, and domain experts need a common record of the scenario, the response, the judgment, the severity, and the decision to fix or accept the risk.
The commercial tier also widens the testing envelope. Giskard says Hub supports continuous red teaming, tool and function-calling tests, advanced agent-specific scans, team workflows, versioning, and integrations with CI/CD processes. Its pricing page additionally advertises SSO, role-based controls, data residency and isolation, a zero-training policy for customer data, dedicated support, and hybrid deployment choices spanning SaaS, private cloud, and on-premises environments. These claims should be validated in the contract and architecture review because deployment language alone does not answer retention, subprocessors, encryption, regional hosting, or incident-response requirements. Buyers should request the relevant security package and map each control to their own threat model before treating a compliance label as sufficient.
Pricing and the real cost of adoption
Giskard publishes a simple packaging split but no public enterprise price. Open Source is listed as Free and includes local deployment, basic vulnerability scanning, basic RAG evaluation, community support, and best-effort maintenance. Enterprise is sold through a demo and quote, with advanced red teaming, broader evaluation, collaboration, automation, governance, deployment, and support features. That makes the free tier easy to assess at the software-license level, but it prevents a reliable public comparison of Hub by seat, project, scan, token, or environment. A serious evaluation should ask for a written quote tied to expected agent count, scan frequency, user roles, data volume, deployment mode, support level, and any professional-services package.
License cost is only one part of total cost. The open-source scanner uses an LLM to generate adversarial scenarios and another LLM judgment step to evaluate responses, and the documentation supports providers such as OpenAI, Anthropic, Gemini, LiteLLM-compatible services, and local options. Those calls can create variable model charges and can expose the agent description and generated responses to the selected provider, although Giskard states the evaluator receives only information passed into the test flow. Teams also need to account for scenario review, false-positive triage, remediation, suite maintenance, CI runtime, and version migration. Giskard saves the most effort when failures become durable tests; running broad scans without ownership for the findings simply creates another queue.
Security, privacy, and governance considerations
The open-source package offers the strongest data-control story when it runs inside a team’s own environment with carefully selected model providers. Giskard lets the operator choose the generator and judge, including providers routed through LiteLLM conventions, so a team can align evaluation traffic with an approved model endpoint. That flexibility does not make the workflow automatically private: prompts, agent descriptions, retrieved content, tool outputs, and responses can still contain sensitive information. Buyers should minimize test data, use synthetic or redacted cases where possible, restrict credentials available to the tested agent, and verify that the evaluation model’s retention terms match internal policy.
Hub is designed for more formal governance, but its controls need evidence at procurement time. Official materials advertise SSO, RBAC, 2FA, audit trails, SOC 2, privacy and healthcare compliance support, data isolation, and private deployment options. The useful buyer question is not whether those labels appear on a feature page; it is whether the selected deployment satisfies the organization’s exact data path and review obligations. Ask who can export conversations, how generated test cases are retained, whether model-provider keys are customer-managed, how audit logs are preserved, what telemetry leaves a private deployment, and how access is revoked. Giskard’s security focus is valuable, yet the testing platform itself becomes sensitive infrastructure once it stores adversarial prompts and agent failures.
Best fit, limitations, and alternatives
Giskard is a strong fit for teams shipping customer-facing RAG systems or tool-using agents where a security or quality failure has meaningful business impact. It suits organizations that want to begin with Python-based local tests and later centralize results for domain experts, security reviewers, and compliance stakeholders. The decisive strengths are adversarial scenario generation, multi-turn testing, reusable checks, and a commercial path to scheduled scans and governed collaboration. It is less attractive for a solo prototype that needs only a few deterministic assertions, or for a team whose primary need is production tracing, token analytics, or prompt experimentation rather than controlled security testing.
Alternatives divide by job. Promptfoo is a natural option for configuration-driven prompt regression and CI red teaming; DeepEval focuses on developer-centric LLM unit tests and metrics; RAGAS concentrates on RAG evaluation; Garak and PyRIT emphasize adversarial security testing; observability platforms add tracing and production analytics that Giskard does not position as its core. Giskard earns the recommendation when red teaming and quality evaluation must converge into a reviewable program, especially for multi-turn or tool-using agents. The final decision should follow a version-pinned proof of concept using the team’s own threat scenarios, plus a Hub quote and security review, rather than a feature checklist alone.
