ModelScan was created by Protect AI to address one of the most overlooked security risks in machine learning: model serialization attacks. When ML models are saved to disk and shared between systems, the serialization process can embed arbitrary executable code alongside the model weights. An attacker who compromises a model file can inject code that executes automatically when the model is loaded, potentially stealing cloud credentials, poisoning outputs, or opening backdoors into production systems.
The tool works by reading model files one byte at a time, scanning for code signatures known to be unsafe without ever loading or executing the model itself. This design choice is critical because the attack vector it protects against fires at load time. If the scanner loaded models to inspect them, it would trigger the very attacks it aims to detect. The byte-level approach makes scans extremely fast, typically completing in seconds proportional to file size.
ModelScan currently supports multiple serialization formats including Pickle and its variants like cloudpickle, dill, and joblib, as well as H5 (HDF5), TensorFlow SavedModel, and formats used by PyTorch, Keras, Sklearn, and XGBoost. The format list continues to expand with each release. It categorizes detected issues by severity level from CRITICAL for potential remote code execution risks down to HIGH for significant but less immediately dangerous vulnerabilities.
Installation is straightforward as a Python package supporting Python 3.9 through 3.12. Teams can add it to their requirements.txt or pyproject.toml for consistent scanning across the organization. The CLI interface handles both individual model files and entire directories, with exit codes designed for CI/CD integration. A Python API is also available for embedding scanning into custom workflows and automated pipelines.
Configuration is flexible through a TOML settings file that can customize reporting format, output locations, and scanning rules. JSON reporting output makes it easy to integrate scan results into security dashboards and alerting systems. The tool can be run ad-hoc during model evaluation or integrated into automated pipelines that gate model deployment on successful security scans.
When ModelScan detects an issue, it reports the specific unsafe operators found and their location within the model file. For example, it might flag ReadFile and WriteFile operations in a TensorFlow model that could allow an attacker to read AWS credentials and exfiltrate them. The reports provide enough detail for security teams to assess whether the flagged operations are intentional or malicious.
Protect AI positions ModelScan as the open-source foundation with Guardian as the enterprise-grade upgrade. Guardian adds cutting-edge scanning capabilities, broader model format support with automatic format detection, proactive security policies for Hugging Face models, seamless CI/CD pipeline integration, and comprehensive audit trails. This dual offering lets teams start with ModelScan for free and upgrade when they need enterprise governance.