NeMo Guardrails is an open-source Python toolkit from NVIDIA with nearly 5,900 GitHub stars that adds programmable safety and control layers to LLM-based conversational systems. Rather than relying solely on model alignment, it provides explicit guardrails that intercept both user inputs and model outputs, applying configurable checks before anything reaches the end user. The toolkit defines five types of rails: input rails that can reject or alter user messages, dialog rails that control conversational flow using Colang (a domain-specific language for state machine-like dialogue definitions), retrieval rails for filtering RAG chunks, execution rails for validating tool calls, and output rails for screening final responses.
The pre-built guardrails cover the most critical LLM vulnerabilities: content safety moderation using NVIDIA's Nemotron Safety Guard models or third-party services like ActiveFence and Cisco AI Defense, jailbreak and prompt injection detection with NemoGuard JailbreakDetect, topic control to keep conversations within defined boundaries, PII detection and masking via GLiNER integration, fact-checking against knowledge bases, and hallucination detection through AlignScore. The IORails engine supports parallel execution of multiple guardrails to minimize latency. A recent integration with Fiddler Guardrails adds low-latency hosted models for additional safety checks. LangGraph integration enables applying guardrails to multi-agent workflows.
NeMo Guardrails works with major LLM providers including OpenAI, Azure OpenAI, Anthropic, Google, and HuggingFace models, plus NVIDIA NIM for local inference. The server exposes an OpenAI-compatible API with a v1/models endpoint, and a GuardrailsMiddleware enables integration with LangChain agents. For production deployments, NVIDIA offers NeMo Guardrails as a microservice container image designed for Kubernetes with Helm charts. The toolkit includes evaluation tools for measuring accuracy of content moderation, fact-checking, and jailbreak detection. Python 3.10 through 3.13 is supported, with installation via pip.
