LiteLLM is an open-source library and proxy server that provides a single, unified interface to call over 100 large language model providers using the OpenAI request format. It solves the integration complexity developers face when their applications need to work with multiple LLM providers, each with their own API format, authentication, and response structure. LiteLLM standardizes all of this behind one consistent interface, allowing developers to switch between providers with a single configuration change.
The LiteLLM Proxy Server acts as a production-grade API gateway with centralized authentication, multi-tenant cost tracking per project and user, and real-time monitoring of all API calls. It supports automatic retries for failed requests, load balancing across multiple provider endpoints, and virtual API keys for secure access control. The Python SDK provides a simple completion function that works identically across OpenAI, Anthropic, Vertex AI, Bedrock, Azure, HuggingFace, Ollama, and dozens more providers, standardizing all responses to the OpenAI format. An admin dashboard UI offers visual monitoring and management of the entire gateway.
LiteLLM is designed for engineering teams and platform builders who need to manage LLM usage across multiple providers and projects at scale. It is commonly used as the LLM gateway layer in enterprise AI platforms, enabling centralized cost control, provider failover, and usage analytics. The library integrates with observability tools like Langfuse and supports deployment on AWS Marketplace for enterprise environments. LiteLLM competes with OpenRouter as a multi-provider gateway, differentiating itself as a self-hosted, open-source solution that gives teams full control over their LLM routing, logging, and cost management infrastructure.
Security note (October 2026): On October 21, 2026, LiteLLM's release pipeline was compromised and a malicious litellm 1.107.0 was published to PyPI for about two hours (11:39–13:23 UTC), with the Docker Hub and GHCR tags v1.107.0, latest, main-latest and main-stable pointing to compromised images until 14:20 UTC. The artifacts were built from an attacker-controlled repository, not the official source tag, and installed a credential stealer and remote-code loader that ran at every Python start. LiteLLM pulled the release; anyone who installed or ran the affected artifacts should treat that environment as compromised and rotate reachable credentials.
On October 27, 2026, LiteLLM disclosed a separate critical vulnerability, GHSA-w4c7-34wj-hqm8 (CVSS 10.0), that affects only proxies running with an Enterprise license. An unauthenticated request to any LLM route could make the proxy read a local file, such as the process environment holding LITELLM_MASTER_KEY and provider API keys, and send it to an attacker-chosen URL, giving full admin access to the proxy and, in some configurations, code execution. Open-source deployments without a license are not affected. The fixed versions are 1.103.10, 1.104.8, 1.105.6 and 1.106.5, so Enterprise-licensed proxies should upgrade to one of these patched releases or newer; the 1.106.2 pin suggested in LiteLLM's October 21 incident post is still vulnerable for Enterprise deployments. If an immediate upgrade is not possible, the advisory recommends rejecting request bodies that contain callback-related JSON keys at the edge and restricting the proxy's outbound traffic to known provider endpoints. LiteLLM had shipped the fix to enterprise customers early through a temporary private registry and says it will make a private registry for the most severe vulnerabilities permanent, while still releasing all security fixes publicly. Pin exact, patched versions and image digests when you deploy LiteLLM.



