Inspect AI is an open-source evaluation framework created by the UK AI Security Institute. It structures evaluations around tasks, datasets, solvers, tools, model interfaces, and scorers so researchers and engineering teams can run repeatable tests instead of relying on ad hoc prompt checks.
The framework supports prompt-engineering workflows, tool-using agents, multi-turn dialogue, model-graded evaluation, and reusable evaluation tasks. These capabilities make it relevant to safety research, red-team programs, governance evidence, and CI gates that compare behavior across model, prompt, or agent changes.
Inspect AI does not turn an evaluation score into a production-safety guarantee. Teams still need to choose representative datasets, define appropriate scorers, review model-judge limitations, manage provider credentials and costs, and connect evaluation findings to broader monitoring, security, and release decisions.
