We publish a picker at llmapi.ai/litellm-alternatives that narrows 20 gateways, proxies and control planes down to the three that fit your deployment. This post is the working behind it: which alternatives qualified, what we scored, what the layer costs, and where the comparison stops being reliable. It is the sibling of our OpenRouter write-up, and the field is deliberately different.
Why the LiteLLM field is different
Someone leaving OpenRouter is leaving a hosted marketplace: the replacements are other places to buy tokens. Someone leaving LiteLLM is leaving a piece of infrastructure they run themselves — a Python proxy, a config file, a Postgres, a Redis, virtual keys, budgets and a deployment that has to scale with traffic. So the field here is gateways and control planes, not model catalogues.
Four walls come up again and again on migration calls.
- Upkeep. Upgrades, config drift between environments, and a database nobody wanted to own.
- Concurrency. A Python proxy that is invisible at 10 requests per second is measurable at 1,000.
- Governance. Budgets exist; RBAC, SSO, audit trails and guardrails an auditor accepts are a product.
- Modality. Speech, video and OCR arrive, and a chat-completions proxy gets a second vendor bolted next to it.
LiteLLM stays in the comparison table as the baseline row and is excluded from the ranking, the same way OpenRouter is on the other page.
Who made the list
To qualify, a platform had to do at least one of LiteLLM's three jobs for real traffic: normalise multiple providers behind one endpoint, control the request path (retries, failover, caching, budgets), or account for what was spent. We grouped the 20 into four families.
| Family | Entries | What they replace |
|---|---|---|
| Self-hosted and open-source gateways | Portkey Gateway OSS, Kong AI Gateway, Apache APISIX, Envoy AI Gateway, Bifrost, Helicone, Langfuse | The proxy, still inside your perimeter |
| Managed gateways with the same job | Portkey Cloud, Cloudflare AI Gateway, Vercel AI Gateway, Requesty, Unify, TrueFoundry, Databricks Mosaic AI Gateway | The proxy, run by somebody else |
| Hyperscaler platforms | AWS Bedrock, Azure AI Foundry, Google Vertex AI | The proxy and the procurement problem at once |
| Aggregators teams land on instead | LLM API, OpenRouter, Eden AI | The proxy and the provider accounts behind it |
The twelve criteria
Every entry was read against the same twelve capabilities, from the vendor's own documentation, licence and pricing pages, plus hands-on use where we had an account.
- Self-hostable — can the component in the request path run on your own infrastructure, and under which licence.
- Managed option — is there a hosted edition with somebody else on call.
- Kubernetes-native — first-class charts, operators or CRDs rather than a container you wrap yourself.
- Edge or serverless — can it run without a long-lived process.
- BYOK — can it route through provider keys you already hold.
- Unified billing — one balance and one invoice across providers.
- Failover — automatic retry onto another provider when an upstream degrades.
- Caching and spend caps — prompt or semantic caching, budgets, guardrails.
- Budgets and virtual keys — per-team limits and attribution.
- Observability — traces, logs and cost reporting built in or exportable.
- Enterprise readiness — SLA, RBAC, SSO, DPA, compliance reports.
- Migration effort — how close it is to a base-URL-and-key swap.
A capability is marked covered only when the vendor documents it as a shipped feature. Roadmaps, betas behind a sales call and community plugins do not count, which is why a few well-known names have emptier rows than their landing pages suggest.
How the picker scores
There is no single best LiteLLM alternative, so we did not publish a ranking. Each entry carries tags on ten axes — who runs it, migration driver, deployment target, traffic scale, model ecosystem, keys and billing, routing behaviour, privacy regime, observability and migration effort — and the picker adds a weight every time one of your answers matches a tag.
Two refinements matter. First, the reason text leads with your rarest matches: if only three of the twenty can run on Kubernetes under a permissive licence, that match tells you more than a match on something fifteen of them do, so it is named first. Second, the two “probably not for you” cards are chosen the same way — by the heaviest, rarest thing the alternative missed — rather than by lowest total score, because a useful rejection explains itself.
What the gateway layer actually costs
“Free and open source” is true of the licence and not of the bill. A production LiteLLM deployment is typically two gateway replicas, a small managed Postgres and a cache.
| Component | Typical monthly cost | Notes |
|---|---|---|
| Gateway compute, 2 replicas | $60 – $120 | 1 vCPU / 2 GB each on a managed container platform |
| Managed Postgres | $50 – $90 | Keys, budgets, spend logs |
| Redis or equivalent cache | $15 – $40 | Routing state and response caching |
| Logs and metrics retention | $0 – $60 | Depends on whether it lands in an existing stack |
| Infrastructure total | ~$125 – $310 | We use $180 as the mid-point in the calculator |
| Engineering time | not costed here | Upgrades, incidents, config; usually the larger number |
The shape of the answer: below roughly $1,000 a month of token spend, a flat-fee managed control plane usually beats the servers. Between $1,000 and about $20,000, flat fees and self-hosting land close together and the deciding factor is who is on call. Above that, percentage fees start to dominate everything else — 5% of $100,000 is $5,000 a month for a layer that two pods were doing for $180.
Licences in the request path
If the reason you self-host is legal rather than financial, the licence of the component in the hot path is the fact that matters.
| Alternative | Licence of the in-path component | What is held back |
|---|---|---|
| LiteLLM | MIT | Enterprise features (SSO, audit, support) behind a commercial licence |
| Portkey Gateway | Apache 2.0 | Logs, governance, guardrails live in the paid control plane |
| Apache APISIX | Apache 2.0 | Nothing — but no LLM console either |
| Envoy AI Gateway | Apache 2.0 | Nothing; it is a CNCF project, and the UI is your own |
| Bifrost | Open source core | Enterprise tier for governance and support |
| Kong AI Gateway | Apache 2.0 core | Most AI plugins require Kong Enterprise |
| Helicone | Apache 2.0 | Hosted convenience and higher retention are paid |
| Langfuse | MIT core | Some enterprise features are commercial |
| TrueFoundry, Databricks, hyperscalers, aggregators | Commercial | Not self-hostable as source |
Deployment shape, side by side
| Alternative | Kubernetes | VM / container | Edge or serverless | Fully hosted |
|---|---|---|---|---|
| LiteLLM | Helm chart | Yes | No | Managed tier |
| Portkey | Yes | Yes | Workers-compatible | Yes |
| Kong AI Gateway | Operator | Yes | No | Konnect |
| Apache APISIX | Ingress controller | Yes | No | No |
| Envoy AI Gateway | Gateway API CRDs | No | No | No |
| Bifrost | Yes | Yes | No | Yes |
| Helicone | Yes | Yes | Yes | Yes |
| Cloudflare AI Gateway | n/a | n/a | Native | Yes |
| Vercel AI Gateway | n/a | n/a | Native | Yes |
| TrueFoundry | Your cluster | Yes | No | Yes |
| LLM API / OpenRouter / Eden AI | n/a | n/a | n/a | Yes |
Observability: what you get for free
A surprising number of LiteLLM installs exist because somebody wanted a dashboard. This is the column those teams should read first.
| Alternative | Traces | Cost attribution | Export |
|---|---|---|---|
| LiteLLM | Request logs in Postgres | Per virtual key | OpenTelemetry, Langfuse, Datadog |
| Portkey | Full traces and replays | Per key, per team | OpenTelemetry |
| Helicone | Full traces, sessions, evals | Per user and per key | OpenTelemetry, webhooks |
| Langfuse | Full traces, evals, prompt versions | Per trace and per user | SDKs and API |
| Cloudflare AI Gateway | Request analytics and logs | Per gateway | Logpush |
| Envoy AI Gateway / APISIX | Prometheus metrics | Token counters only | Your own stack |
| Hyperscalers | Platform logging | On the cloud bill | Native cloud tooling |
| LLM API | Request logs and usage reporting | Per key and per project | API |
What we could not measure
- Latency overhead. Gateway overhead depends on hardware, deployment topology, payload size and concurrency. Published numbers from vendors are measured on their own benches and are not comparable. Benchmark the two finalists on your own cluster with your own prompts; it is an afternoon of work and it settles the argument.
- Support quality. Read from plan documents, not tested. A response-time commitment in a PDF is not evidence.
- Real upgrade pain. How badly a minor version breaks your config is not something a documentation page reports.
- Negotiated pricing. Every commercial row here is list price. Enterprise deals move.
Four situations, four answers
| If this is you | Start with | Runner-up | Why |
|---|---|---|---|
| Small team, the proxy is the only infrastructure you run | Portkey | Cloudflare AI Gateway | Same job, nothing to operate, and you keep your own keys |
| Python overhead is showing at peak | Bifrost | Envoy AI Gateway | A Go or Envoy data plane instead of a Python one |
| The platform team owns ingress already | Kong AI Gateway | Apache APISIX | AI traffic under the policy and telemetry you run today |
| Speech, video or OCR just landed on the roadmap | LLM API | Eden AI | One managed key across modalities a chat proxy has no endpoint for |
Sources and when we checked them
Every figure on the landing page and in this post comes from a vendor's own documentation, licence file, pricing page or status page, read on 22 September 2026. Infrastructure costs are list prices from common managed platforms on the same date. Where a vendor does not publish a fact, the comparison shows a dash instead of an estimate. We re-read the pricing pages quarterly, and the date at the top of the landing page is the last time we did.
Where we are biased, stated plainly
LLM API is our product. It sits in the table on the same criteria as everything else, wins the planner where it genuinely fits — managed, multi-modal, unified billing, white label — and loses it where it does not. If you answered that you must keep the gateway inside your own perimeter, the planner will point you at Portkey, Bifrost, Envoy or Kong and not at us, because that is the correct answer. A comparison that always ends with the author's product is an advertisement, and nobody links to advertisements.
Ready to use the picker? It is at llmapi.ai/litellm-alternatives, ten questions, no email.
