Ragmux vs LiteLLM: which self-hosted AI gateway fits?

Ragmux and LiteLLM both put a single, self-hosted API in front of the model providers you actually call, and neither expects you to hand your traffic to a vendor’s cloud to get that. Past that, they are built for different jobs. LiteLLM is a broad multi-provider proxy — one interface in front of the widest provider catalog in the open-source gateway space, with load balancing, spend tracking and guardrails layered on top. Ragmux is a narrower, single-binary gateway that pairs routing with retrieval-augmented generation (RAG) as a first-class part of the request path, not a separate system you wire in yourself. Picking between them is less “which is better” than “which job you’re actually doing.”

What each one is

LiteLLM’s own README describes it as an open-source AI gateway that provides “a single, unified interface to call 100+ LLM providers — OpenAI, Anthropic, Gemini, Bedrock, Azure, and more — using the OpenAI format.” It ships as a Python SDK and, separately, as the LiteLLM Proxy: a standalone service the README describes as a “production-ready gateway — virtual keys, spend tracking, guardrails, load balancing, and an admin dashboard out of the box.”

Ragmux is a single Go binary that exposes /v1/chat/completions, /v1/models, /v1/embeddings, /v1/retrieve and /v1/rerank, with chat, models and embeddings in the OpenAI-compatible schema, translating Anthropic and Gemini requests — tool calls included — to and from that schema. A project can be attached to a document store, and when it is, every request against that project gets its query embedded, searched and the matched passages injected into the system prompt before the request reaches a provider. Everything the gateway needs — users, connections, projects, documents, their vector embeddings and the request log — lives in one PostgreSQL database via the pgvector extension. See What is an AI gateway? for the fuller version of that story.

Feature comparison

A few of those rows need the source behind them spelled out. The license row and the Redis row both come from LiteLLM’s own repository and docs, not from guesswork: the enterprise/ directory of the LiteLLM repository carries its own license requiring a paid subscription for production use outside development and testing, and the proxy deployment docs list Redis as handling rate limiting, router state and caching across instances, required once more than one instance is running. Ragmux carries no equivalent requirement — see Deployment and scaling below. The database row is conditional on purpose: the same deployment docs list what PostgreSQL is there for — keys, teams, users, spend logs and config — and mark it required for the proxy’s auth and tracking features specifically, while separately describing a mode with no database at all that authenticates every call with a single shared master key and keeps no history. Ragmux has no such mode — every deployment, evaluation included, needs PostgreSQL with pgvector from the first request.

Retrieval: the part LiteLLM leaves to you

Nothing in LiteLLM’s README or its cost-tracking and deployment docs describes a retrieval or document-grounding feature — it is, by design, a routing and spend-management layer, and pairing it with your own documents means standing up a vector database and a retrieval pipeline yourself, then calling that pipeline before you call LiteLLM.

Ragmux treats that pipeline as part of the gateway instead of a separate system in front of it. Documents (PDF, DOCX, HTML, Markdown, plain text) are parsed, chunked with section-aware splitting and contextual embeddings, and indexed with pgvector; a search fuses vector similarity with either PostgreSQL full-text ranking or ParadeDB BM25 by reciprocal rank fusion, optionally reranked by an LLM or a dedicated reranking API. None of that needs a second system to operate — it lives in the same PostgreSQL database as everything else the gateway does. The Private document Q&A guide walks through setting one up end to end.

Deployment and scaling

The default Ragmux image bundles PostgreSQL 17 with pgvector inside the same container as the gateway — one container, one volume, no Redis and no separate vector database. Scaling out to several replicas needs nothing extra either: every replica keeps its rate limits, budgets, sessions, audit log and document-ingestion queue in that same PostgreSQL database, with ingestion claimed under a lease and the retention job electing a leader through an advisory lock, so a plain round-robin proxy in front is enough — no session affinity, no shared cache tier.

LiteLLM’s own deployment docs describe PostgreSQL as required specifically for authentication, spend tracking and key management — a deployment that only proxies requests under a shared master key, with none of those features, needs no database. Once virtual keys or budgets are in play, though, PostgreSQL is there to stay, and the same docs add Redis to the requirements list once more than one instance is running, to keep rate limits, router state and caching consistent across replicas. That is a reasonable trade for a system built to load-balance across many provider deployments — the state Redis holds is exactly what makes fast, cross-instance load balancing possible — but it is a second stateful service Ragmux’s architecture never asks for: every Ragmux deployment needs PostgreSQL unconditionally, and needs nothing else even at several replicas.

Spend control: tokens vs dollars

This is the sharpest operational difference between the two, and it is worth being exact about it rather than picking a side. LiteLLM’s max_budget setting caps spend directly in currency: its per-key and per-user budget docs show a budget defined in USD with an optional budget_duration reset window, and its per-tag budget docs describe the same mechanism scoped to a request tag instead. Both describe the same enforcement: once spend crosses the configured dollar ceiling, the next request is refused with an error naming the budget that was exceeded, not just logged for later.

Ragmux’s project and API key budgets (budget_daily_tokens, budget_monthly_tokens) are counted in prompt-and-completion tokens, never currency — a request is refused for exceeding a token count, and only that. The dashboard does show an estimated cost per request and per project, built from an editable price table, but that estimate is informational: it is never what a budget check compares against. If a provider’s price changes and nobody edits the price table, the cost figure drifts while the token budget keeps enforcing exactly what it always did. If a hard dollar ceiling is the requirement, that is LiteLLM’s max_budget, not Ragmux today. If tracking where the tokens went — by project, broken down with CSV export — is the requirement, see Tracking LLM spend per project.

Provider coverage — and a specific overlap

LiteLLM’s provider count is the more meaningful difference for most teams evaluating either tool: more than 100 providers under one interface against Ragmux’s six provider types (openai, anthropic, gemini, deepseek, ollama, and custom_openai for everything else that speaks the OpenAI Chat Completions format). Coverage isn’t only a count, though — what a native adapter does with a request that isn’t plain text matters too. Ragmux’s own provider documentation details, per adapter, how Anthropic and Gemini tool calls are translated to and from the OpenAI schema (including a full JSON Schema sanitiser for Gemini, which rejects keywords OpenAI-style tool definitions routinely carry), how streamed tool-call fragments are reassembled where a provider sends them whole, and which image formats each provider’s adapter will accept inline versus fetch from a URL itself. LiteLLM’s README documents its provider count and its OpenAI-format promise at the same level of confidence, but nothing in the pages this comparison draws from goes into translation specifics at that depth — which may simply mean the detail lives elsewhere in LiteLLM’s docs, not that the translation is shallower.

That last provider type, custom_openai, is worth noting specifically: Ragmux’s own provider documentation lists it as covering “vLLM, LM Studio, LiteLLM, text-generation-inference’s OpenAI route, Ollama’s /v1 shim and any other server speaking the Chat Completions format” — which means a team that already runs LiteLLM for its provider breadth can put Ragmux in front of that same LiteLLM deployment as one more upstream, and get Ragmux’s retrieval and per-project token budgets without giving up LiteLLM’s routing underneath.

The admin surface: what’s in the box

Both projects ship an admin dashboard rather than leaving operators to the API alone, but what’s verifiably inside that dashboard differs. Ragmux’s dashboard ships nine tabs in its single embedded UI, served from the same binary with no separate build step: Overview (traffic, budget-exhaustion forecasts, a CSV export), Models, RAG stores (upload, ingestion progress, a retrieval tester), Projects (usage rings, key rotation), API keys (per-key limits and budgets), a Playground for live /v1/chat/completions calls against a real project key, Prices (the editable price table, open to the editor role and up), and, for admins only, Users and an Audit log covering every create, update, delete, login and key rotation with actor, target and IP recorded. Access to those screens follows one of three fixed roles — admin, editor, viewer — enforced on every request rather than cached on a session.

LiteLLM’s README confirms an admin dashboard ships with the proxy too, alongside virtual keys and spend tracking, but the pages this comparison draws from don’t enumerate its tabs or its role model with the same specificity Ragmux’s own docs give for the above — which is a gap in what’s citable here, not a claim that LiteLLM’s dashboard does less. Readers evaluating that surface specifically should check LiteLLM’s own admin UI documentation rather than infer it from this comparison.

Where Ragmux is not the answer

• PostgreSQL with pgvector is mandatory. There is no SQLite or MySQL option and no way to run the gateway without a database at all — every deployment, from a single container to a multi-replica cluster, needs that one dependency operated somewhere.
• Ragmux is pre-1.0. The current release is 0.5.0, and its own roadmap lists SSO/OIDC login, cost-denominated (dollar) budgets and an import tool for pre-PostgreSQL databases as not scheduled — features a team migrating off a more mature gateway may already depend on.
• AGPL-3.0-or-later is a copyleft license with a network-use clause. Running a modified version of Ragmux as a service your users interact with over a network obligates you to offer them the corresponding source of your modification. LiteLLM’s core is MIT — a permissive license with no such obligation outside the enterprise/ directory — which makes it the simpler choice to embed inside a closed-source SaaS product.
• Reference docs are in English; guides are mostly in English too, with a few in Turkish. LiteLLM’s larger install base also means a larger body of third-party tutorials, Stack Overflow answers and integration guides in more languages.