Litellm

Litellm is an open-source LLM gateway and OpenAI-compatible proxy designed to unify access to over 100 language models through a single, consistent API. Its primary purpose is to simplify authentication, routing, and cost control for teams working with multiple LLM providers, without requiring application code changes. By exposing an OpenAI-format interface, Litellm lets developers plug in different models or providers behind the scenes while keeping the same client libraries and request patterns.

Key capabilities include centralized authentication management, configurable load balancing across providers and models, and detailed spend tracking and usage analytics. Litellm supports provider failover and routing rules, allowing organizations to optimize for cost, latency, or reliability while abstracting away vendor-specific differences. It offers rate limiting, request logging, and role-based access controls, enabling teams to enforce governance and security policies around LLM usage. The gateway can be deployed in your own infrastructure, giving you more control over data flow and compliance requirements.