What do I put in the model field when a client connects to Ragmux?

The name of one of the project's routes of the matching kind. A project with chat routes resolves model to an exact route name, the default connection's model name, or an empty value; anything else is a 404 with code model_not_found. For chat, a project with no routes ignores model for selection and the connection decides. Embeddings and rerank differ: with no route of that kind, only the RAG store's connection model name or an empty value is accepted.

Why does a client's model list show route names instead of provider models?

GET /v1/models describes what the project can reach. With chat routes it lists each route's name plus the default connection's model name, so a client that fills its model picker from that endpoint offers the routes.

Do these clients need the same key as the OpenAI SDK?

Yes. Every client here sends an API key as a Bearer token, and a project's sk-proj- key works for chat, embeddings, retrieve and rerank without extra scopes.

Can LangChain's OpenAIEmbeddings use Ragmux?

Yes, through /v1/embeddings. LangChain tokenizes input by default, so pass check_embedding_ctx_length=False; Ragmux refuses arrays of token ids and accepts text only.

How do I get RAG passages into a client that builds its own prompt?

Call POST /v1/retrieve with a query. It runs the project's store search, applies its rerank and distance settings, and returns the passages with scores and metadata, with no chat model involved.

Connecting Vercel AI SDK, n8n, Open WebUI and LibreChat to Ragmux

Every client on this page speaks the OpenAI wire format, so none of them needs a Ragmux plugin. The connection is the same three values each time: a base URL, an API key, and a model string. The first two are mechanical. The third is where most first connections go wrong, so it comes first.

What the model field means

How Ragmux treats model depends on whether the project has routes. A project without chat routes ignores model for selection: the project’s connection decides which model answers, and the value you send is echoed back. A project with chat routes treats it as a selector. POST /v1/chat/completions resolves it to an exact route name, to the default connection’s model name, or to an empty value; anything else is a 404 with code model_not_found, and the message does not list the valid names.

So for a chat client below, the model field is a chat route’s name. Embeddings and rerank work the same way with an embeddings or rerank route, with one difference: when the project has no route of that kind, the call goes to the RAG store’s connection, which answers to its own model name and to an empty value. Anything else is a 404. Routes are what give a name its retries and fallback targets; the API reference has the details.

That also explains what a client’s model picker shows. GET /v1/models lists each chat route’s name for a project that has them, so a client that fills its picker from that endpoint offers routes, not provider model ids.

Vercel AI SDK

The AI SDK’s documentation connects any OpenAI-compatible server through @ai-sdk/openai-compatible: createOpenAICompatible takes a name, an apiKey and a baseURL, then provider('model-id') gives a chat model and provider.embeddingModel('model-id') an embedding model.

import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import { generateText } from 'ai';

const ragmux = createOpenAICompatible({
  name: 'ragmux',
  baseURL: 'http://localhost:8765/v1',
  apiKey: process.env.RAGMUX_API_KEY,
});

const { text } = await generateText({
  model: ragmux('<chat-route-name>'),
  prompt: 'How many vacation days do I get?',
});

RAGMUX_API_KEY holds the project’s sk-proj-… key. For embeddings, call ragmux.embeddingModel('<embeddings-route-name>'); the gateway answers POST /v1/embeddings with OpenAI’s response shape.

n8n

In n8n, the OpenAI Chat Model node takes an OpenAI credential. Set the API Key to the project key and leave Organization ID empty. To aim the credential at Ragmux, set its Base URL to your gateway address with the /v1 suffix; if n8n runs in a container, that address must be one the container can reach, which localhost usually is not. Then choose the route name in the node’s Model field. n8n’s documentation says its model dropdown is loaded from OpenAI; whether it lists Ragmux’s routes was not verified.

One caveat on sourcing: n8n’s own credential page lists only the API Key and Organization ID fields. The Base URL field, and typing the route name as an expression when the dropdown does not list it, are described by third-party gateway guides rather than by n8n’s documentation. Confirm both in your n8n version before building a workflow on them.

Open WebUI

Open WebUI’s documentation adds a server under Settings → Admin → Connections → Manage OpenAI API Connections → Add Connection. Enter the gateway’s base URL (http://localhost:8765/v1) and the project key. When Open WebUI runs in Docker and Ragmux on the host, its guide says to use host.docker.internal in place of localhost.

The Verify Connection button calls the provider’s /models endpoint, and the guide is explicit that saving does not test a connection, so verify before relying on it. If verification fails, add the route name by hand under Model IDs with the + button; chat can still work without a /models answer. Then pick the route in the chat model selector.

LibreChat

LibreChat reads custom endpoints from librechat.yaml. Its documentation says an endpoint is kept only when name, baseURL, apiKey and models are all present, and models needs fetch: true or a non-empty default list.

endpoints:
  custom:
    - name: 'Ragmux'
      apiKey: '${RAGMUX_API_KEY}'
      baseURL: 'http://localhost:8765/v1'
      models:
        default: ['<chat-route-name>']
        fetch: false
      titleConvo: true
      titleModel: '<chat-route-name>'
      modelDisplayLabel: 'Ragmux'

The ${RAGMUX_API_KEY} form reads the key from LibreChat’s .env; the docs warn that a missing variable still shows the endpoint but fails each message with Missing API Key for <endpoint>. models.default is the route name. LibreChat’s documentation does not say how fetch: true obtains its models, so this page uses the explicit default list.

LangChain and LlamaIndex: embeddings and retrieval

Chat works through the same base URL and key as above. Two newer endpoints matter for RAG code.

Embeddings. POST /v1/embeddings accepts text input, up to 2048 strings per call; arrays of token ids and the dimensions parameter are refused with 400. LangChain’s OpenAIEmbeddings tokenizes input by default, and its reference says to set check_embedding_ctx_length=False for non-OpenAI endpoints so raw text is sent instead:

from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(
    base_url="http://localhost:8765/v1",
    api_key="sk-proj-...",
    model="<embeddings-route-name>",
    check_embedding_ctx_length=False,
)

Retrieval. When the framework owns the prompt, POST /v1/retrieve returns passages without calling a chat model:

curl -s localhost:8765/v1/retrieve \
  -H "Authorization: Bearer sk-proj-..." -H 'Content-Type: application/json' \
  -d '{"query": "how many vacation days do I get?", "top_k": 5}'

The response lists hits, each with content, score, distance and metadata. The store’s search mode, backend, rerank and distance settings apply and the request cannot override them. A project with no RAG store answers 400 with code no_rag_store. The RAG docs cover how a store is configured, and private document Q&A shows the whole setup.

If your pipeline gathers candidates elsewhere, POST /v1/rerank reorders them in Cohere’s request and response shape, behind the rerank scope.

After it’s connected

From the gateway’s side these four clients are indistinguishable from any OpenAI SDK: limits, budgets and the request log key off the project, so tracking what each client spent needs one project per client or per key. The general case is in Ragmux as an OpenAI-compatible API gateway. Routes and the /v1/embeddings, /v1/retrieve and /v1/rerank endpoints are Ragmux 0.5 features; on an earlier release, the model field is always ignored for selection and these endpoints do not exist.

Full reference: /docs/api/.