ModelScan is an open-source project from Protect AI that scans machine learning model files for unsafe code. It is the first model scanning tool to support multiple serialization formats including H5, Pickle, and SavedModel, covering PyTorch, TensorFlow, Keras, Sklearn, and XGBoost frameworks. The tool reads model files byte by byte looking for code signatures that indicate malicious content, without actually loading or executing the model. This approach makes scanning incredibly fast, completing in seconds for most models. ModelScan can be integrated into CI/CD pipelines as a pre-deployment gate and used as a pre-commit hook. Protect AI also offers Guardian as an enterprise-grade model scanning product with broader format support and Hugging Face integration.
LLM Guard operates as a runtime security layer that inspects every prompt sent to an LLM and every response generated, blocking attacks in real time. It protects against prompt injection, jailbreak attempts, toxic content generation, PII leakage, and unauthorized data exfiltration. The tool runs as a middleware between your application and the LLM API, adding security without requiring changes to the underlying model. LLM Guard supports multiple detection strategies including regex patterns, ML classifiers, and heuristic checks. It is designed for production deployment with low latency overhead and can be configured with custom policies to match organizational security requirements.
Garak is an LLM vulnerability scanner that performs automated red-teaming by probing language models with adversarial inputs to discover weaknesses. Named after a Star Trek character known for subterfuge, garak tests for prompt injection susceptibility, data leakage, hallucination patterns, toxicity generation, and other failure modes catalogued in frameworks like OWASP LLM Top 10 and MITRE ATLAS. The tool generates comprehensive reports showing which attacks succeeded and which defenses held, providing a quantitative security assessment of model behavior. Garak supports testing against multiple LLM providers and can be extended with custom probes and detectors.
These three tools address fundamentally different threat vectors in the AI security stack. ModelScan protects the model supply chain by ensuring that model files downloaded from repositories like Hugging Face do not contain embedded malware or trojans. LLM Guard protects production applications by filtering malicious inputs and unsafe outputs in real time. Garak identifies vulnerabilities before deployment through systematic testing. A comprehensive AI security program uses all three in sequence: scan models before loading them, test them before deployment, and guard them in production.
Model serialization attacks are the specific threat ModelScan addresses. When a model is saved to disk using formats like Pickle, arbitrary Python code can be embedded alongside the model weights. Loading such a model automatically executes the malicious code, potentially stealing cloud credentials, exfiltrating training data, or poisoning model outputs. This is a modern version of the Trojan Horse that exploits the trust developers place in pre-trained models. ModelScan detects these attacks without executing the model, making it safe to run on untrusted model files. The tool has become essential for organizations that download models from public hubs.