Self-hosted AI gateway comparison: Ragmux, LiteLLM, Portkey and more

“Self-hosted AI gateway” narrows a crowded field fast. Some of the best-known names under the broader “AI gateway” label — OpenRouter, Cloudflare AI Gateway — are hosted services you call as an API, not software you run yourself, so they answer a different question than “what do I self-host.” This page compares the ones you actually can self-host against Ragmux, on the facts that are verifiable from each project’s own repository, docs or license file — not on opinion, and not on a benchmark nobody published a method for.

The self-hosted vs. hosted line

Two of the names people search alongside “self-hosted AI gateway” don’t actually belong in a self-hosting comparison: OpenRouter and Cloudflare AI Gateway are both proprietary services that run entirely on the vendor’s infrastructure. There is no container to pull, no binary to run and no source to audit — you send requests to their API and trust their operational and data-handling practices, the same way you would trust any SaaS. That’s not a criticism; a hosted router removes an operational burden entirely, which is exactly the point for a team that doesn’t want to run anything. It just means the comparison on this page treats them differently from the gateways below, which you deploy and operate yourself.

Ragmux, LiteLLM, Portkey’s AI Gateway, Helicone and Langfuse are all software you can run on your own infrastructure — a Docker container, a Kubernetes cluster, a bare VM. What differs between them is how much of what each one offers is actually in that self-hosted package, versus behind a hosted control plane you’d still be trusting a vendor with.

Compare every self-hosted AI gateway at a glance

“Built-in retrieval” means the gateway itself embeds, searches and injects your documents into the request — not “can be wired up to a vector database with enough glue code.” None of the alternatives below do that today; it’s the specific gap Ragmux fills, and also the specific reason none of them is a drop-in replacement for what Ragmux does.

LiteLLM

LiteLLM is the closest thing to a direct alternative: a self-hosted, multi-provider proxy with its own database, its own admin dashboard and a provider catalog that dwarfs Ragmux’s — its README claims support for “100+ LLM providers”. It also does several things Ragmux genuinely does not do today, and does not merely do differently: routing strategies beyond weighted and prioritized route targets, content guardrails, and spend budgets denominated in actual currency rather than tokens. On provider breadth and dollar-denominated budgets specifically, LiteLLM is simply the more capable project right now — that’s not a close call, and it’s worth saying plainly rather than only implying it through a feature table. What LiteLLM does not do is retrieval. The full breakdown, including why a team already running LiteLLM might still put Ragmux in front of it, is in Ragmux vs LiteLLM.

Portkey AI Gateway

Portkey open-sources its gateway core — MIT-licensed and self-hostable with a single npx @portkey-ai/gateway command — while its role-based access control, SSO and compliance certifications sit behind a hosted plan, now marketed under the name PRISMA AIRS AI Gateway following its acquisition by Palo Alto Networks. Portkey’s gateway has no retrieval feature of its own. The full breakdown, including what specifically moves behind the paid tier, is in Ragmux vs Portkey.

OpenRouter

OpenRouter is not self-hostable at all: a hosted router in front of a wide model catalog, with no infrastructure of your own to run. It belongs in this comparison only because “self-hosted AI gateway” and “OpenRouter alternative” overlap heavily in what people actually search for — see Ragmux vs OpenRouter for when a hosted router is genuinely the right answer instead of a self-hosted one.

Cloudflare AI Gateway

Cloudflare AI Gateway is a proprietary layer bundled into Cloudflare’s platform, built around edge caching, analytics and rate limiting in front of the providers you already call, rather than on hosting your own routing or retrieval logic. It runs entirely on Cloudflare’s network, available on all plans per its own docs, with no self-hosted deployment of it at all. See Ragmux vs Cloudflare AI Gateway for the full comparison — including a disclosure this site should make, since ragmux.com itself runs on Cloudflare Pages.

Helicone

Helicone is open source and genuinely self-hostable, but it isn’t primarily a gateway in the sense Ragmux, LiteLLM and Portkey are, and it ships as two separate repositories under two separate licenses: the observability platform (Helicone/helicone, Apache-2.0) and a newer, standalone AI Gateway router (Helicone/ai-gateway) that its own LICENSE file and GitHub’s license detection both mark GPL-3.0 — a copyleft license, not the permissive Apache-2.0 the platform repository carries. Helicone was acquired by Mintlify in March 2026 and now runs in maintenance mode. Neither Helicone repository competes with Ragmux for the routing-plus-retrieval job; the platform can ingest OpenTelemetry-shaped trace data through its own /v1/trace/log endpoint, which is how it can sit downstream of Ragmux (or any gateway on this page) tracing what happened to a request after it was forwarded, though it isn’t a drop-in OTLP receiver — see Ragmux vs Helicone for what that actually takes, and what each license obligates.

Langfuse

Langfuse is also open source and self-hostable, but it is observability and tracing only, instrumenting calls an application already makes rather than routing them itself. Its core is MIT-licensed outside its ee/ directory, and its own README states the team has been part of ClickHouse since January 2026. Langfuse’s docs describe it as able to operate as an OpenTelemetry backend on its /api/public/otel endpoint, with third-party libraries such as OpenLLMetry and OpenLIT extending that reach to languages beyond its native Python and JS/TS SDKs — which is how it, too, can sit downstream of Ragmux without competing for the routing job. See Ragmux vs Langfuse for the full picture.

Database and cache requirements, compared

“Self-hosted” still means operating something, and what that something is varies a lot across this list. Ragmux needs exactly one stateful service, unconditionally: PostgreSQL with the pgvector extension, bundled inside the default container so a single-host install operates nothing separate at all. LiteLLM needs no database for a credentials-free, budget-free proxy, but PostgreSQL becomes required the moment virtual keys, spend tracking or budgets are wanted, and its own docs add Redis once more than one instance is running. Portkey’s open-source gateway core is a stateless router with no database of its own to operate; the budgets, analytics and governance layer sits behind Portkey’s hosted product instead, which is infrastructure you don’t operate because someone else does. Helicone’s self-hosted Docker stack is the heaviest of the group by component count — its own docs list PostgreSQL, ClickHouse and MinIO S3 as the three datastores the all-in-one deployment runs. Langfuse’s is heavier still: Postgres, ClickHouse, Redis (or Valkey) and S3-compatible blob storage, plus separate web and worker containers, per its own self-hosting docs — reflecting that it’s built to ingest and query a high volume of trace data, a different workload than routing a chat request.

None of that makes one architecture wrong; a team already running ClickHouse for other analytics may find Helicone or Langfuse’s requirements trivial, and a team that wants exactly one dependency to think about will read Ragmux’s single-database design as the point. It’s a real axis to evaluate on regardless of which side of it you land.

What AGPL, MIT, Apache-2.0 and “proprietary” actually mean here

The license column in the table above isn’t decoration — it decides what you’re allowed to do with the software, and the differences are large enough to matter before you commit infrastructure to one of these. Ragmux’s AGPL-3.0-or-later carries a network-use clause: modify Ragmux and run the modification as a service your users interact with over a network, and you’re obligated to offer those users the corresponding source of your modification, not just of the unmodified upstream. MIT (LiteLLM’s core, Portkey’s gateway core, Langfuse’s core outside ee/) and Apache-2.0 (Helicone’s Helicone/helicone platform repository) carry no such obligation — you can fork, modify and run any of them inside a closed-source product without releasing anything back. Helicone’s separate AI Gateway router (Helicone/ai-gateway) is the exception in this table: its own LICENSE file and GitHub’s license detection both mark it GPL-3.0, a copyleft license that does obligate releasing source for a distributed modified version, just not for running one purely as an internal network service the way AGPL’s network-use clause does — see Ragmux vs Helicone for what that distinction means in practice. “Proprietary (SaaS)” — OpenRouter, Cloudflare AI Gateway — means there is no source to fork or audit at all; you are trusting the vendor’s operational practices the way you would any other cloud dependency, full stop.

That makes AGPL the one license on this page a team building a closed-source, modified gateway-as-a-product needs to actually read before adopting Ragmux for that purpose — running Ragmux unmodified, or modified for internal use with no external users, carries no such obligation. It’s the trade-off for the rest of what AGPL protects: nobody can take Ragmux’s own source, modify it and re-sell it as a closed competing service without releasing their changes back.

Where Ragmux is not the answer

• PostgreSQL with pgvector is mandatory, in every deployment shape — there is no database-free mode and no alternative database engine.
• Ragmux is pre-1.0. The current release is 0.5.0; SSO/OIDC login, dollar-denominated budgets and several other things a mature deployment may expect are listed on the roadmap as not scheduled, not shipped.
• AGPL-3.0-or-later carries a network-use clause: running a modified Ragmux as a service your users interact with obligates you to offer them your modification’s source. LiteLLM’s core, Portkey’s gateway core and Helicone’s platform repository are all permissively licensed (MIT or Apache-2.0), which is the simpler choice inside a closed-source product; Helicone’s separate AI Gateway router is GPL-3.0, which is simpler than AGPL for a service you never distribute but still requires releasing source for a distributed modified version.
• Reference docs are in English; guides are mostly in English too, with a few in Turkish, and Ragmux’s install base is small next to LiteLLM’s — a large open-source gateway community brings more third-party tutorials and faster answers to unusual questions.

Choosing between them

As a starting point rather than a rule: reach for Ragmux when retrieval over your own documents needs to be part of every gateway request, not a system you bolt on separately, and a self-hosted Postgres database is not a burden. Reach for LiteLLM when provider breadth, advanced load balancing or dollar-denominated budgets are the actual requirement. Reach for Portkey or Cloudflare AI Gateway when you’d rather not operate a control plane at all and a hosted layer on top of (or instead of) a thin self-hosted core is an acceptable trade. Reach for OpenRouter when you don’t want to run any infrastructure, full stop. And add Helicone or Langfuse downstream of whichever gateway you pick when tracing and evaluation, not routing, is the gap you’re closing next.

A few concrete shapes this tends to take: a support or internal-tools team building a chatbot over a handbook or a knowledge base is the Ragmux case almost by definition — the whole point is that a document search step has to run on every request, and a second system for that would be the thing being avoided. A platform team standardizing how dozens of internal services call a wide, changing set of model providers, with per-team dollar budgets enforced centrally, is closer to LiteLLM’s shape than Ragmux’s — Ragmux has no equivalent to a currency-denominated ceiling today, however precisely it tracks where tokens went. A product team that wants a managed control plane and would rather not run infrastructure at all — no Postgres, no containers, nothing to patch — is the case none of the self-hosted options here actually serve, Ragmux included; that’s OpenRouter, Portkey’s hosted product or Cloudflare AI Gateway’s territory instead. And a team that already has a gateway in production and just wants to see what it’s actually doing — latency by model, cost by user, where an agent went wrong mid-trace — is reaching for Helicone or Langfuse regardless of which gateway sits in front of it, because neither one requires replacing that gateway to adopt.