# Getting started

> Ragmux is a single-binary AI gateway written in Go.

Source: https://ragmux.com/docs/getting-started/
Updated: 2026-10-11
Project: https://github.com/ragmux/ragmux (AGPL-3.0-or-later)

---

Ragmux is a single-binary AI gateway written in Go. It sits between your applications
and LLM providers, exposes one **OpenAI-compatible API**, and can augment every request
with **retrieval (RAG)** from documents you upload. All state — users, model
connections, projects, documents, chunks, vectors and metrics — lives in one
**PostgreSQL** database with the [pgvector](https://github.com/pgvector/pgvector)
extension. One container, no Redis, no separate vector database.

```
client  ──►  POST /v1/chat/completions (Bearer sk-proj-…)
             │
             ├─ project → model connection (provider credentials stay server-side)
             ├─ project → RAG store (optional): embed query → top-k chunks → inject context
             └─ adapter: OpenAI · Anthropic · Gemini · DeepSeek · Ollama · custom OpenAI (vLLM…)
                          JSON and SSE streaming, normalised to the OpenAI schema
             │
             └─ PostgreSQL + pgvector: users, connections (AES-256-GCM keys), projects,
                documents (bytea), chunks, HNSW vector indexes, request logs
```

## Features

- **Providers:** `openai`, `azure_openai`, `anthropic`, `gemini`, `deepseek`, `ollama`
  (native `/api/chat`) and `custom_openai` for vLLM, LM Studio, LiteLLM or any
  OpenAI-compatible server, plus `cohere_rerank` and `voyage_rerank` for reranking. Ready-made
  presets cover Mistral, xAI and Groq; see [Providers](/docs/providers/).
- **OpenAI-compatible proxy:** `/v1/chat/completions` (JSON + SSE streaming) and
  `/v1/models`; works with the official SDKs by changing `base_url` and `api_key`.
  Anthropic and Gemini requests and streams are translated, tool calls included, and remote
  images are fetched for the providers that cannot follow a URL themselves.
- **More `/v1` endpoints:** `POST /v1/embeddings`, `POST /v1/retrieve` (search a RAG store
  without calling a model) and `POST /v1/rerank` (Cohere-shaped), each behind its own key
  scope. See the [API reference](/docs/api/#client-api-v1).
- **Routes, retry and fallback:** a project can define named routes whose targets are
  ordered by priority and weight; the request's `model` then selects a route, and an unknown
  name is `404 model_not_found`. A chat call that fails before the first byte is retried or
  handed to the next target, and a connection that keeps failing is taken out of rotation by
  a circuit breaker. See [Routing](/docs/providers/#routing) and
  [Retry and fallback](/docs/providers/#retry-and-fallback).
- **RAG stores:** PDF / DOCX / HTML / TXT / Markdown, section-aware chunking with contextual
  embeddings, hybrid search fusing pgvector with either Postgres full-text ranking or
  ParadeDB BM25, reranking by your own model or the Cohere / Voyage APIs, distance threshold.
- **Projects:** one `sk-proj-…` key per project mapped to a model connection, a system prompt
  and an optional RAG store; provider credentials are encrypted at rest and never exposed.
- **API keys:** per-user `sk-user-…` keys for `/v1` scoped to the projects you grant them and
  `sk-mgmt-…` keys for the management API, both with scopes (`/v1` keys: `chat`, `models`,
  `embeddings`, `retrieve`, `rerank`), expiry, revocation and optional limits under the
  project's.
- **Users and roles:** `admin`, `editor`, `viewer`, project membership, login rate limiting
  with lockout, audit log of every management action, and `ragmux reset-password` when the
  last administrator is locked out.
- **Cost:** prompt-caching passthrough, cache-aware token accounting and an editable price
  table that estimates what each request cost.
- **Rate limits and budgets:** per-project requests / tokens per minute and daily / monthly
  token budgets shared across replicas, OpenAI-style `429` and `x-ratelimit-*` headers.
- **Observability:** per-request logs (tokens, latency, status, streaming, RAG use, cost),
  summaries, daily series, configurable retention, an authenticated Prometheus `/metrics`
  endpoint and OpenTelemetry traces over OTLP (HTTP/JSON, or gRPC on request) with GenAI
  span names and one child span per upstream attempt.
- **Scale:** run several replicas against one PostgreSQL — ingestion is claimed under a
  lease, the retention pass elects a leader, and streaming needs no session affinity.
- **Dashboard:** embedded UI at `/admin/` for models, RAG stores, documents, projects and
  their routes, API keys, metrics, users, audit log and a playground; everything is also a
  REST API.

## Quick start

Nothing to clone — one container with the gateway and its own PostgreSQL 17 + pgvector
server inside:

```bash
docker run -d --name ragmux \
  -p 127.0.0.1:8765:8765 \
  -e SECRET_KEY="$(openssl rand -hex 32)" \
  -v ragmux-data:/data \
  ragmux/ragmux:latest
```

Open <http://localhost:8765/admin/>. On a fresh database the dashboard asks you to
**create the first administrator** (username and a password of at least 12 characters);
that form only works while no user exists. For unattended installs pass
`-e ADMIN_USER=admin -e ADMIN_PASSWORD=...` instead and the account is created on first
start. `ADMIN_PASSWORD` follows the same rule (at least 12 characters, at most 72 bytes);
on an empty database a shorter one stops the start with an error (see
[Configuration](/docs/configuration/#environment-variables)).

Two things worth knowing about that command:

- **`SECRET_KEY` encrypts the provider credentials.** Nothing is required to start, but
  without it the gateway generates `/data/ragmux/secret.key` inside the volume, which you
  then have to back up together with the database. Keep the key with your backups.
- **The port is published on loopback only.** Put a TLS-terminating reverse proxy in
  front for network access and set `TRUST_PROXY_HEADERS` / `SECURE_COOKIES`; see
  [Configuration](/docs/configuration/#behind-a-reverse-proxy). Change the mapping to
  `-p 8765:8765` only deliberately.

Everything persistent is under `/data`: `/data/pg` is the PostgreSQL cluster (`PGDATA`)
and `/data/ragmux` holds the `secret.key` fallback (`DATA_DIR`). That volume plus
`SECRET_KEY` is the whole state. The embedded PostgreSQL has no TCP listener at all, so
`8765` is the only port.

Running models locally? Provider URLs on private networks (Ollama on the Docker host, a
vLLM service on the same Docker network) are refused by default as an SSRF guard. Allow
them by hostname before you add the connection — add
`-e PRIVATE_UPSTREAM_ALLOWLIST=host.docker.internal,ollama` to the command above, or put
the same variable in `.env` under Compose (or `ALLOW_PRIVATE_UPSTREAMS=true` on a trusted
network — see [Configuration](/docs/configuration/#private-upstreams)).

### Which image

Three variants are published from every release:

| Tag | What it is | Choose it when |
|---|---|---|
| `ragmux/ragmux:latest` | **All-in-one:** the gateway plus its own PostgreSQL 17 + pgvector in one container, supervised by a single entrypoint (`Dockerfile.aio`). | You want one container and no database to operate. The default, and the right answer for most installations. |
| `ragmux/ragmux:latest-app` | **Gateway only**, on a distroless base, with no database inside; you point it at your own PostgreSQL with `DATABASE_URL` (`Dockerfile`). | You already run PostgreSQL (managed service, your own cluster, a separate container), or you want **several gateway replicas** against one database — the all-in-one image cannot be scaled that way. |
| `ragmux/ragmux:latest-paradedb` | **All-in-one on ParadeDB:** same gateway and same data layout, but its PostgreSQL carries `pg_search` (BM25) as well as `pgvector` (`Dockerfile.aio.paradedb`). | You want the lexical half of hybrid retrieval answered by **BM25** instead of PostgreSQL full-text ranking. Swapping from the default all-in-one image needs no reprocessing. |

The `-paradedb` image takes exactly the same command and the same volume layout as the
default one. Docker Hub is the primary registry; the full tag matrix, the GHCR mirror and
the platforms are listed under
[Published images](/docs/configuration/#published-images).

**`latest` is for trying it out; pin a version in production.** The commands here use
`latest` so they keep working as you read them, but a deployment you care about should
name the release it was tested against (`ragmux/ragmux:<version>`), or better the
manifest digest that `cosign verify` reports (`ragmux/ragmux@sha256:…`), which no later
tag move can change. Images are signed from v0.3.1 onward; see
[Verifying the container image](/docs/security/#verifying-the-container-image).

### Docker Compose

Compose is the second way in, and the better one once you want more than the single
container gives you: **your own PostgreSQL**, the **split layout** (the gateway next to a
`pgvector/pgvector:pg17` service you can back up and upgrade on its own schedule), the
**ParadeDB** variant, a **multi-replica** setup behind a load balancer, or the scheduled
**backup** profile and the `scripts/` helpers. It works from a clone of the repository:

```bash
git clone https://github.com/ragmux/ragmux.git
cd ragmux
cp .env.example .env
echo "SECRET_KEY=$(openssl rand -hex 32)" >> .env   # keep it with your backups
docker compose up -d
```

That is the same all-in-one image and the same `/data` volume as the `docker run` above,
with `.env` as the one place configuration lives. The other files layer on top:

| File | What it runs |
|---|---|
| [`docker-compose.yml`](https://github.com/ragmux/ragmux/blob/main/docker-compose.yml) | the default all-in-one container, plus a `backup` profile |
| [`docker-compose.split.yml`](https://github.com/ragmux/ragmux/blob/main/docker-compose.split.yml) | the gateway (`-app`) next to its own `pgvector/pgvector:pg17`; needs `SECRET_KEY`, `POSTGRES_PASSWORD` and `RAGMUX_DB_PASSWORD` in `.env` |
| [`docker-compose.paradedb.yml`](https://github.com/ragmux/ragmux/blob/main/docker-compose.paradedb.yml) | the split layout with ParadeDB in place of pgvector, for BM25 lexical retrieval |
| [`docker-compose.scale.yml`](https://github.com/ragmux/ragmux/blob/main/docker-compose.scale.yml) | an overlay on the split file: several gateway replicas behind nginx — see [Scaling](/docs/scaling/) |
| [`docker-compose.otel.yml`](https://github.com/ragmux/ragmux/blob/main/docker-compose.otel.yml) | an overlay adding an OpenTelemetry Collector — see [Observability](/docs/observability/) |

For a PostgreSQL you already operate, set `DATABASE_URL` and the default container skips
its embedded server, or run the gateway-only `-app` image with `DATABASE_URL` and
`SECRET_KEY`. The two layouts, the `/data` volume and how to move between them are
described in [Deployment layouts](/docs/configuration/#deployment-layouts).

**Where state lives.** With the default layout everything is in the `ragmux-data`
volume (`/data/pg`: the database, `/data/ragmux`: the `secret.key` fallback) plus
`SECRET_KEY`. With the split layout the gateway container is stateless and the state is
the `pgdata` volume plus `SECRET_KEY`. Either way, back up with `scripts/backup.sh`
(`pg_dump -Fc`, works with both layouts) or the scheduled `backup` Compose profile — see
[Backup and restore](/docs/backup-restore/).

## Using the gateway

**1. Log in** and keep the session token:

```bash
TOKEN=$(curl -s localhost:8765/admin/api/login -H 'Content-Type: application/json' \
  -d '{"username":"admin","password":"YOUR_PASSWORD","bearer":true}' | jq -r .token)
AUTH="Authorization: Bearer $TOKEN"
```

**2. Add model connections** (a chat model and, for RAG, an embedding model):

```bash
curl -s localhost:8765/admin/api/models -H "$AUTH" -H 'Content-Type: application/json' -d '{
  "name": "claude", "provider_type": "anthropic", "api_key": "sk-ant-...", "model_name": "claude-sonnet-4-5"}'
curl -s localhost:8765/admin/api/models -H "$AUTH" -H 'Content-Type: application/json' -d '{
  "name": "openai-embed", "provider_type": "openai", "api_key": "sk-...", "model_name": "text-embedding-3-small"}'
```

**3. Create a RAG store and upload documents** (optional):

```bash
curl -s localhost:8765/admin/api/rag-stores -H "$AUTH" -H 'Content-Type: application/json' -d '{
  "name": "handbook", "embedding_connection_id": 2}'
curl -s localhost:8765/admin/api/rag-stores/1/documents -H "$AUTH" -F file=@handbook.pdf
curl -s localhost:8765/admin/api/rag-stores/1/documents -H "$AUTH"    # status: pending → processing → ready
```

**4. Create a project** and get its `sk-proj-…` key (shown only once):

```bash
curl -s localhost:8765/admin/api/projects -H "$AUTH" -H 'Content-Type: application/json' -d '{
  "name": "support-bot", "model_connection_id": 1, "rag_store_id": 1,
  "system_prompt": "You are the company support assistant."}'
```

**5. Call it like OpenAI.** The `model` field is echoed back but the project decides the
real model (a project with [routes](/docs/providers/#routing) uses it to pick one);
retrieved passages are injected into the system prompt and counted in the
`x-ragmux-rag-hits` header.

```bash
curl -N localhost:8765/v1/chat/completions \
  -H "Authorization: Bearer sk-proj-..." -H 'Content-Type: application/json' \
  -d '{"model":"default","stream":true,"messages":[{"role":"user","content":"How many vacation days do I get?"}]}'
```

```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8765/v1", api_key="sk-proj-...")
resp = client.chat.completions.create(model="default", stream=True,
    messages=[{"role": "user", "content": "How many vacation days do I get?"}])
for chunk in resp:
    # Guard the array: an upstream may send a frame of its own with no
    # choices (a content filter result, a proxy keep-alive), and asking for
    # usage adds a final chunk that carries only the token counts.
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="")
```

## Dashboard

`/admin/` serves a single-page dashboard (vanilla JavaScript, no build step, embedded in
the binary) over the same REST API. On a fresh database it shows the first-run setup
form; afterwards a sign-in screen that reports remaining attempts and lockouts. Screens:

- **Overview:** 24h / 7d / 30d window with deltas against the previous period, requests
  per day, traffic by project, budget watch with exhaustion forecasts, recent requests
  and a CSV export.
- **Models:** connection cards with provider capabilities, last test result and private
  upstream flag; the form can test a connection before saving it, start from a preset and
  takes an Azure OpenAI endpoint, deployment and API version.
- **RAG stores:** quota meters, drag-and-drop upload, ingestion progress, per-document
  reprocess and a retrieval tester (hybrid / vector, rerank, per-stage latencies); the rerank
  connection list follows the chosen backend.
- **Projects:** usage rings for the minute, day and month windows with reset countdowns,
  budget forecast banner, members, key rotation, a route editor (kind, up to eight targets
  with model override, weight and priority) and a project CSV export.
- **API keys:** mint and revoke your own `sk-user-…` and `sk-mgmt-…` keys with a scope
  checklist, grants and limits.
- **Playground:** real `/v1/chat/completions` calls with a project key, streaming or JSON,
  optional extra system prompt, RAG sources as citations and the response headers.
- **Users** and **Audit log** (admins): role and failed-login tiles, filters, NDJSON export.

What a user sees follows their role: viewers get read-only screens, editors the forms,
admins the user and audit screens. The page ships its own fonts (Bricolage Grotesque,
IBM Plex Sans, JetBrains Mono, all SIL OFL, `web/fonts/`) and runs under a strict
`Content-Security-Policy`, so it makes no third-party requests.

## Roadmap

Not in this release: an import tool for 0.1 (pre-PostgreSQL) databases, OCR for scanned
PDFs, SSO / OIDC login, cost-denominated budgets, and a cache breakpoint on the gateway's
own system prompt and RAG context block.

The direction after this release is in [ROADMAP.md](https://github.com/ragmux/ragmux/blob/main/ROADMAP.md); to help, see
[CONTRIBUTING.md](https://github.com/ragmux/ragmux/blob/main/CONTRIBUTING.md).

## License

Ragmux is free software licensed under the **GNU Affero General Public License v3.0 or
later** (AGPL-3.0-or-later); see [LICENSE](https://github.com/ragmux/ragmux/blob/main/LICENSE). The AGPL's network-service clause
applies: if you run a modified version as a service that users interact with over a
network, you must offer those users the corresponding source of your modified version.