LiteLLM alternatives: 7 open-source and managed options

LiteLLM, from BerriAI, describes itself on its GitHub README as an “Open Source AI Gateway for 100+ LLMs”. It comes in two forms: a Python SDK you call from your code, and a proxy server (the LiteLLM AI Gateway) that a whole team or company shares.

According to the proxy documentation, the gateway gives you:

• One OpenAI-compatible endpoint for 100+ providers, so existing OpenAI clients work by changing the base URL.
• Virtual keys for each user, team or project, each carrying its allowed models, budget and rate limits.
• Load balancing, fallbacks and retries that move traffic between deployments when a provider has a problem.
• Spend tracking by team and project, and logging to tools such as Langfuse, Datadog, OpenTelemetry and Prometheus.
• Caching in memory, Redis, S3 or disk, plus semantic caches on Redis or Qdrant (caching docs).

Licence. The LICENSE file puts everything under the MIT licence except the enterprise/ directory, which is governed by its own licence file. The enterprise page lists the features that need a paid licence key, including audit logs, role-based access control, tag-based budgets and model-specific budgets per key, and says “SSO is free for up to 5 users” before an enterprise licence is needed. Enterprise pricing is quoted on request.

These reasons come from LiteLLM’s own documentation, not from complaints:

• You have to run it. The production deployment guide calls for a PostgreSQL database for keys, teams and spend logs, Redis once you run more than one instance, two or more proxy replicas behind a load balancer and a migrations job on every upgrade. Teams that do not want to own that stack look at hosted gateways.
• Some governance features are licensed. If you need SSO for more than 5 users, audit logs or tag-based budgets, you either buy LiteLLM Enterprise or compare it with gateways that include those features elsewhere.
• Your platform is already chosen. Teams that run Kong for APIs, or Envoy on Kubernetes, may prefer an AI gateway built on the same proxy they already operate.
• Different runtime or embedding needs. LiteLLM’s SDK is Python. A team writing services in Go, for example, may prefer a gateway it can also embed as a Go library.

If running LiteLLM is the only problem, you do not have to switch products: Perimattic ManageStacks offers managed LiteLLM hosting on a dedicated instance in AWS, Azure or GCP, with PostgreSQL, Redis and the admin UI pre-configured, and upgrades and backups handled.

Portkey: the closest like-for-like swap

Portkey’s gateway is MIT-licensed, OpenAI-compatible and covers fallbacks, retries and weighted load balancing, with guardrails built in. The catch: on its pricing page semantic caching and granular budgets and rate limits sit on the Enterprise plan, so check which features your use needs before assuming the free self-hosted build covers them.

Bifrost: self-hosted, with budgets and semantic caching in the open-source build

Maxim’s Bifrost is Apache 2.0 and its README lists virtual keys, hierarchical budgets, rate limiting and semantic caching. It ships as an HTTP gateway via npx or Docker and as a Go SDK. Its Enterprise plan adds SSO, audit logs, RBAC, cluster mode and adaptive load balancing. Its README also publishes latency figures; they are the vendor’s own benchmarks, so test in your setup.

Agent Router (formerly Envoy AI Gateway): for Kubernetes platform teams

The project’s README says Envoy AI Gateway is now Agent Router, with the same code, maintainers and Apache 2.0 licence. It runs on Kubernetes with Envoy Gateway or standalone, and puts credentials, routing, quotas and failover in one place. It suits teams that already operate Envoy; its README does not describe response caching.

Kong AI Gateway: if Kong already fronts your APIs

Kong Gateway itself is Apache 2.0, but the AI plugins that match LiteLLM’s router, semantic cache and token-based rate limits are marked “only available as part of our AI Gateway Enterprise offering” in Kong’s plugin docs. Plan for a commercial licence if you need them.

OpenRouter, Vercel AI Gateway, Cloudflare AI Gateway: if you do not want to run anything

All three are hosted, and all three let you buy model access through them with credits, so you can start without your own provider accounts. Each says tokens are billed at the provider’s price, with the fees shown in the table. Your prompts then flow through the vendor’s infrastructure, so read each one’s data-handling settings. See the OpenRouter alternatives page for a closer look at hosted options.

• It already works. If your proxy is stable, the cost of migrating keys, budgets, routing rules and dashboards is rarely worth a marginal feature.
• You want the widest open-source provider coverage. LiteLLM’s README claims 100+ providers behind one MIT-licensed codebase.
• You need budgets, caching and fallbacks without a licence. Per-key budgets, rate limits, Redis and semantic caching and fallbacks are all in the open-source proxy.
• Your team works in Python. The same project gives you a Python SDK for direct calls and the proxy for shared use.
• The only pain is operations. Managed hosting, such as LiteLLM on ManageStacks, removes the server work without a migration.

Perimattic LLM Gateway is left out of the comparison above, which covers third-party tools. It is built for teams that want LiteLLM-style control with less to operate. Its features:

• One OpenAI-compatible API in front of OpenAI, Anthropic, Google, Mistral, DeepSeek and open models.
• Routing rules by cost, latency or task, with automatic fallback when a provider errors or rate-limits.
• Exact-match and semantic caching of responses.
• Budgets and rate limits per team, API key or customer.
• Cost tracking and attribution by team, feature and customer, with alerts.
• Request logging with PII redaction.
• Runs in your own cloud (VPC) or managed.

Get started: Sign up and our team will get in touch, or read the LLM gateway overview. If you want LiteLLM running today, managed LiteLLM on ManageStacks is available now.