Ragmux

Self-hosted AI gateway

Every model behind one API. Every answer grounded in your documents.

Ragmux is a single Go binary that sits between your applications and OpenAI, Anthropic, Gemini, DeepSeek, Ollama or your own vLLM — speaking one OpenAI-compatible API, and injecting retrieval from your own PDFs into every request it forwards.

Get startedRead the docs

One container/PostgreSQL 17 + pgvector/AGPL-3.0-or-later

~/ragmuxbash
$ docker run -d --name ragmux \
  -p 127.0.0.1:8765:8765 \
  -e SECRET_KEY="$(openssl rand -hex 32)" \
  -v ragmux-data:/data \
  ragmux/ragmux:latest
✔ragmux/ragmux:lateststarted:8765
✔postgres 17 + pgvectorembedded/data
Startup log — docker logs ragmux
→ http://localhost:8765/admin/
Routes to
  • OpenAI
  • Anthropic
  • Gemini
  • DeepSeek
  • Ollama
  • vLLM · LM Studio · LiteLLM
  • Static Go binary, CGO_ENABLED=0, distroless base
  • Two containers — no Redis, no separate vector database
  • Provider keys AES-256-GCM encrypted at rest

How it works

One request in. The right model, with the right context, out.

A project key is the whole configuration surface. It decides which provider answers, which system prompt applies, which documents are searched and what the caller is allowed to spend.

Your app

POST /v1/chat/completions
Authorization: Bearer sk-proj-…
{"model": "default",
 "stream": true,
 "messages": [ … ]}

The official OpenAI SDKs work unchanged. The model field is echoed back — the project decides the real one.

Ragmux:8765

  1. 01Resolve the projectKey → model connection, system prompt, rate limits and token budget.
  2. 02Retrieve, if the project has a storeEmbed the query, hybrid search the chunks, inject the top-k passages into the system prompt.
  3. 03Adapt, forward, normaliseTranslate to the provider's schema — including Anthropic tool calls — and stream the answer back as OpenAI SSE.

Upstream

  • OpenAI
  • Anthropic
  • Gemini
  • DeepSeek
  • Ollama /api/chat
  • custom_openai

Credentials stay server-side and are never returned by the API.

PostgreSQL + pgvector

  • users & roles
  • connections
  • projects
  • documents (bytea)
  • chunks
  • HNSW vector indexes
  • request logs

Drop-in

Change two lines. Keep your SDK.

The gateway answers /v1/chat/completions and /v1/models in the OpenAI schema, JSON and SSE alike. Nothing else in your codebase moves.

  • Anthropic and Gemini requests and streams are translated for you — tool calls included for Anthropic.
  • Every response carries x-ragmux-rag-hits — how many passages were injected.
  • Over a limit you get an OpenAI-style 429 with Retry-After and x-ratelimit-* headers.
app/client.pydiff
from openai import OpenAI
client = OpenAI(
-    base_url="https://api.openai.com/v1",
-    api_key="sk-…",
+    base_url="https://gateway.internal/v1",
+    api_key="sk-proj-…",
)
resp = client.chat.completions.create(
    model="default", stream=True,
    messages=[{"role": "user",
             "content": "How many vacation days do I get?"}],
)
# the handbook PDF is searched and injected server-side

Retrieval

Your documents belong in the request — not in a second system.

Upload a file to a RAG store and an ingestion worker takes it from pending to ready. The vectors land in the same database as everything else.

  1. 01ParsePDF, DOCX, HTML, Markdown and plain text, straight into the database as bytea.
  2. 02ChunkSection-aware splitting, with contextual chunks so a passage still makes sense out of its page.
  3. 03EmbedThrough the embedding connection you chose for the store; indexed with HNSW in pgvector.
  4. 04SearchVector similarity and Postgres full-text, fused with reciprocal rank fusion.
  5. 05InjectTop-k passages prepended to the system prompt, counted in the response headers.

Reranking, your choice

Your own chat model, Cohere Rerank or Voyage Rerank re-orders the candidates before they reach the prompt. Any failure falls back to the fused order rather than failing the request.

A distance threshold

max_distance drops weak matches instead of padding the prompt with noise.

BM25 or tsvector

Postgres full-text ranking by default; point a store at ParadeDB pg_search for real BM25 without moving your documents out of Postgres. Switching either way needs no reprocessing.

Control plane

Keys, roles, limits — and a log of everything that touched them.

What a shared provider key can't give you: who spent what, on which project, under which prompt.

Projects and keys

One sk-proj-… key per project, mapped to a model connection, a system prompt and an optional RAG store. Provider credentials are AES-256-GCM encrypted at rest.

Roles and membership

admin, editor, viewer, plus per-project membership. Non-admins only see their own projects, and a foreign project id answers 404 rather than 403.

Rate limits and budgets

Requests and tokens per minute, plus daily and monthly token budgets in UTC windows — counted in the database, so replicas share one ceiling.

Login protection

Per-username and per-IP attempt limits with a lockout window, stored in Postgres. The client IP is the TCP peer unless you opt into proxy headers.

Audit log

Every login and every management action is recorded, readable at /admin/api/audit and in its own dashboard tab.

Metrics and retention

Tokens, latency, status, streaming and RAG use per request; summaries and daily series on top, with a retention window you set and an hourly janitor that enforces it.

Dashboard & API

The UI is just another client.

The dashboard at /admin/ ships inside the binary and manages models, RAG stores, documents, projects, metrics, users and the audit log — with a playground to try a project key before you wire it up. Everything it does is a REST call you can make yourself.

No build step

Vanilla JS, embedded

Same auth

Bearer session token

The surface
  • POST/v1/chat/completionsthe proxy — JSON or SSE
  • GET/v1/modelswhat this key may call
  • POST/admin/api/modelsadd a provider connection
  • POST/admin/api/rag-storescreate a store
  • POST/admin/api/rag-stores/{id}/documentsupload a file
  • POST/admin/api/projectsthe key is shown once
  • GET/admin/api/auditwho changed what
  • GET/admin/api/systemversions, database size, backup
  • GET/healthzfor your orchestrator

Quick start

From nothing to a grounded answer.

  1. 01Start the container and create the first administrator

    $ docker run -d --name ragmux \
      -p 127.0.0.1:8765:8765 \
      -e SECRET_KEY="$(openssl rand -hex 32)" \
      -v ragmux-data:/data \
      ragmux/ragmux:latest

  2. 02Add a model connection and a RAG store

    $ curl -s localhost:8765/admin/api/models -H "$AUTH" \
      -d '{"name":"claude","provider_type":"anthropic", …}'
    $ curl -s localhost:8765/admin/api/rag-stores/1/documents \
      -H "$AUTH" -F [email protected]

  3. 03Create the project and call it like OpenAI

    $ curl -N localhost:8765/v1/chat/completions \
      -H "Authorization: Bearer sk-proj-…" \
      -d '{"model":"default","stream":true,"messages":[…]}'
    x-ragmux-rag-hits: 4

Documentation

Written for the person on call.

Browse the repository

Configuration

Every environment variable and default, the -healthcheck and -version flags, reverse-proxy notes.

API reference

Authentication, the role required per endpoint, request and response examples, the error envelope.

Retrieval

Formats, contextual chunking, hybrid search and its backends, reranking, the context format.

Users, roles and limits

The roles matrix, project membership, login protection, the audit log, budgets and retention.

Providers

Provider types and endpoints, the Anthropic, Gemini and Ollama translation details, passthrough and model echo.

Backup and restore

What to back up, the scripts, scheduled backups, point-in-time recovery and the restore runbook.

Observability

The /metrics endpoint and why it is authenticated, the metric table, cardinality rules, readiness, tracing spans.

Scaling

Running several replicas on one database: what is already shared, leased ingestion, rolling restarts.

Not shipped yetOCR for scanned PDFs · SSO and OIDC login · budgets denominated in money · an import tool for 0.1 SQLite databases

Learn

The concepts behind the flags.

Browse Learn

What is an AI gateway?

The concept behind Ragmux, and what a gateway adds on top of calling a model provider directly.

Ragmux vs LiteLLM

Two self-hosted gateways built for different jobs — provider breadth against built-in retrieval — compared feature by feature.

Self-hosted AI gateway comparison

LiteLLM, Portkey, OpenRouter, Cloudflare AI Gateway and more, lined up on license, database and retrieval.

Private document Q&A

Ground a chatbot in your own PDFs without handing them to a third-party retrieval service.

Your keys, your documents, your database.

Ragmux is free software under the AGPL-3.0-or-later. Run it on a laptop, run it in your cluster — nothing phones home.

Get startedGitHub
docker run -d -p 127.0.0.1:8765:8765 -v ragmux-data:/data ragmux/ragmux:latest

Arrow keys to move, Enter to open.