Skip to content
aicoolies logo
LiteLLM logo

LiteLLM

Unified API proxy for 100+ LLMs

Drop-in OpenAI-compatible proxy supporting 100+ LLM providers with load balancing, spend tracking, rate limiting, and fallback routing. Acts as a unified gateway for all your AI model calls, letting teams switch between providers, enforce budgets, and add reliability layers without changing application code. Essential infrastructure for multi-model AI architectures.

About LiteLLM

LiteLLM is an open-source library and proxy server that provides a single, unified interface to call over 100 large language model providers using the OpenAI request format. It solves the integration complexity developers face when their applications need to work with multiple LLM providers, each with their own API format, authentication, and response structure. LiteLLM standardizes all of this behind one consistent interface, allowing developers to switch between providers with a single configuration change.

The LiteLLM Proxy Server acts as a production-grade API gateway with centralized authentication, multi-tenant cost tracking per project and user, and real-time monitoring of all API calls. It supports automatic retries for failed requests, load balancing across multiple provider endpoints, and virtual API keys for secure access control. The Python SDK provides a simple completion function that works identically across OpenAI, Anthropic, Vertex AI, Bedrock, Azure, HuggingFace, Ollama, and dozens more providers, standardizing all responses to the OpenAI format. An admin dashboard UI offers visual monitoring and management of the entire gateway.

LiteLLM is designed for engineering teams and platform builders who need to manage LLM usage across multiple providers and projects at scale. It is commonly used as the LLM gateway layer in enterprise AI platforms, enabling centralized cost control, provider failover, and usage analytics. The library integrates with observability tools like Langfuse and supports deployment on AWS Marketplace for enterprise environments. LiteLLM competes with OpenRouter as a multi-provider gateway, differentiating itself as a self-hosted, open-source solution that gives teams full control over their LLM routing, logging, and cost management infrastructure.

Security note (October 2026): On October 21, 2026, LiteLLM's release pipeline was compromised and a malicious litellm 1.107.0 was published to PyPI for about two hours (11:39–13:23 UTC), with the Docker Hub and GHCR tags v1.107.0, latest, main-latest and main-stable pointing to compromised images until 14:20 UTC. The artifacts were built from an attacker-controlled repository, not the official source tag, and installed a credential stealer and remote-code loader that ran at every Python start. LiteLLM pulled the release; anyone who installed or ran the affected artifacts should treat that environment as compromised and rotate reachable credentials.

On October 27, 2026, LiteLLM disclosed a separate critical vulnerability, GHSA-w4c7-34wj-hqm8 (CVSS 10.0), that affects only proxies running with an Enterprise license. An unauthenticated request to any LLM route could make the proxy read a local file, such as the process environment holding LITELLM_MASTER_KEY and provider API keys, and send it to an attacker-chosen URL, giving full admin access to the proxy and, in some configurations, code execution. Open-source deployments without a license are not affected. The fixed versions are 1.103.10, 1.104.8, 1.105.6 and 1.106.5, so Enterprise-licensed proxies should upgrade to one of these patched releases or newer; the 1.106.2 pin suggested in LiteLLM's October 21 incident post is still vulnerable for Enterprise deployments. If an immediate upgrade is not possible, the advisory recommends rejecting request bodies that contain callback-related JSON keys at the edge and restricting the proxy's outbound traffic to known provider endpoints. LiteLLM had shipped the fix to enterprise customers early through a temporary private registry and says it will make a private registry for the most severe vulnerabilities permanent, while still releasing all security fixes publicly. Pin exact, patched versions and image digests when you deploy LiteLLM.

Pricing & Platform Specs

Pricing Summary

Free & open-source universal AI gateway and proxy (MIT License) unifying 100+ LLM providers under the standard OpenAI API format. Open-source core is 100% free ($0/mo) for self-hosting with unlimited requests and models via pip install litellm or Docker container. LiteLLM Enterprise tier ($500-$1,000+/mo or custom annual contract) adds advanced governance, SAML/OIDC SSO, SCIM provisioning, RBAC, AWS KMS key rotation, audit logging, advanced guardrails, and 24/7 dedicated support with 99.99% uptime SLA.

full pricing breakdown →

Supported Platforms

Python, Docker

Explore categories, tags & use cases

CLI token usage tracker for AI coding agents

Tokscale is a CLI tool that tracks token usage and costs across AI coding agents including Claude Code, Codex, OpenCode, Gemini CLI, Cursor, and more. Built with a native Rust core for high-performance processing, it provides detailed breakdowns of input, output, cache, and reasoning tokens with real-time pricing calculations via LiteLLM data. Features include interactive 2D/3D contribution graphs, web visualization dashboards, global leaderboards, and JSON export for cost analysis.

freeOpen Source

Unified LLM API gateway and proxy hub

New API is an open-source multi-tenant AI gateway that aggregates and distributes LLM API requests across providers like OpenAI, Claude, and Gemini through a unified proxy interface. It cross-converts requests into OpenAI-compatible, Claude-compatible, or Gemini-compatible formats, with built-in channel management, quota control, token-based authentication, and billing capabilities. Deploy via Docker with SQLite or MySQL for centralized model management.

Open Source

Side-by-Side Comparisons

Helicone logo
Helicone
vs
LiteLLM logo
LiteLLM

Helicone vs LiteLLM — LLM Observability Layer or Routing Gateway?

Teams researching LLM infrastructure often land on “Helicone vs LiteLLM” expecting a straight head-to-head, the way you would compare two code editors or two vector databases. That expectation is the wrong starting point. Helicone and LiteLLM solve adjacent but distinct problems in a production LLM stack, and understanding which layer each one occupies matters more than picking a “winner.” This comparison breaks down what each tool actually does, how they are priced and deployed, and — because it materially affects the decision — what a March 2026 ownership change means for one of them going forward.

HeliconeLiteLLM
One API logo
One API
vs
LiteLLM logo
LiteLLM

One API vs LiteLLM — Self-Hosted LLM Gateways for Multi-Provider Management

One API and LiteLLM are both open-source LLM API gateways that provide unified OpenAI-compatible endpoints for managing multiple model providers. One API offers a web-based management dashboard popular in the Chinese ecosystem, while LiteLLM provides a Python-first proxy with broader Western adoption. This comparison helps teams choose the right gateway for their multi-provider LLM infrastructure.

One APILiteLLM
RouteLLM logo
RouteLLM
vs
LiteLLM logo
LiteLLM

RouteLLM vs LiteLLM — Intelligent Model Router vs Universal LLM Gateway

RouteLLM and LiteLLM both sit between applications and LLM providers but serve different primary functions. RouteLLM uses trained classifier models to intelligently route each request to the most cost-effective model that can handle its complexity, reducing costs by up to 85%. LiteLLM provides a unified API gateway that normalizes access to 100+ LLM providers with load balancing, fallbacks, rate limiting, and spend tracking.

RouteLLMLiteLLM

Community experience

Sources & verification

Sources checked
Content verified

Verification dates are editorial checks. Routine CMS saves and automatic updatedAt timestamps do not advance them.

optional / review badge

Optional image for a README or docs page. The snippet links to this review. No link is required.

LiteLLM reviewed on aicoolies
[![LiteLLM reviewed on aicoolies](https://aicoolies.com/api/badge/tool/litellm.svg)](https://aicoolies.com/reviews/litellm-review)

FAQ

What is LiteLLM?

Drop-in OpenAI-compatible proxy supporting 100+ LLM providers with load balancing, spend tracking, rate limiting, and fallback routing. Acts as a unified gateway for all your AI model calls, letting teams switch between providers, enforce budgets, and add reliability layers without changing application code. Essential infrastructure for multi-model AI architectures.

Is LiteLLM free?

Yes — LiteLLM is open source and free to use. Free & open-source universal AI gateway and proxy (MIT License) unifying 100+ LLM providers under the standard OpenAI API format. Open-source core is 100% free ($0/mo) for self-hosting with unlimited requests and models via pip install litellm or Docker container. LiteLLM Enterprise tier ($500-$1,000+/mo or custom annual contract) adds advanced governance, SAML/OIDC SSO, SCIM provisioning, RBAC, AWS KMS key rotation, audit logging, advanced guardrails, and 24/7 dedicated support with 99.99% uptime SLA.

Is LiteLLM open source?

Yes — LiteLLM is open source.

Is LiteLLM still maintained?

Yes — LiteLLM is active. Its listing was last verified on October 28, 2026.

What are the best LiteLLM alternatives?

The first editor-selected LiteLLM alternatives are Tokscale, New API.

How does LiteLLM score in our review?

The published editorial review lists LiteLLM at 82/100 overall across speed, privacy, and developer experience. Check the review's evidence status and test metadata for its verification level.