Track LLM spend per project with Ragmux

If every application shares one provider API key, “how much did this feature cost us last month” is a question you can only answer by asking the provider’s own billing dashboard, which knows nothing about which of your applications made which call. Ragmux’s answer is to make the project — not the API key you happen to be holding — the unit spend is attributed to, budgeted against and exported by.

A worked example: chargeback across three customers

Take a SaaS product that embeds an LLM feature and serves three customers from the same Ragmux deployment, each on a different plan. Each customer gets its own project — call them acme-support, initech-support, globex-support — created with its own sk-proj-… key, wired into that customer’s own instance of the application. Nothing about that setup requires the application code to know it’s being billed; it just calls /v1/chat/completions with the key it was given, the way it would against any OpenAI endpoint.

The decisions worth making up front are the budgets themselves. A customer on the entry plan gets a budget_monthly_tokens sized to what that plan is priced to cover, so a runaway integration on their side caps out at the plan’s own ceiling rather than becoming an unbilled cost the SaaS absorbs; a customer on an unlimited enterprise plan gets 0 (unlimited) instead, deliberately, with the forecast in GET /admin/api/projects/{id}/usage watched rather than a ceiling enforced. At the end of a billing period, GET /admin/api/projects/{id}/metrics.csv?window=720h run once per customer project turns into three CSV files — one per customer, cost column included — that map directly onto three line items on three invoices, with no manual attribution step where costs could be split incorrectly between customers.

Projects are the attribution boundary, not an afterthought

A Ragmux project bundles a model connection, an optional RAG store, a system prompt and its own sk-proj-… key into one thing an application calls. Every field that matters for cost tracking hangs off that same project: its rate limits (rate_limit_rpm, rate_limit_tpm), its token budgets (budget_daily_tokens, budget_monthly_tokens), and every row request_logs writes for a call made with its key. Nothing about that attribution depends on trusting the calling application to tag its own requests correctly — the project a request belongs to is fixed by which key authenticated it, so “support-bot’s spend” and “internal-tools’ spend” can never bleed into each other by accident.

That is also what makes per-team or per-customer cost centers straightforward: one project per cost center, one budget per project, one CSV export per project. A SaaS serving several customers from the same gateway can give each customer’s integration its own project and read GET /admin/api/projects/{id}/metrics for exactly what that customer’s traffic cost — the same shape a finance team would want for an internal team budget.

Tokens are enforced. Dollars are an estimate.

It’s worth being precise about what “budget” means here, because it’s easy to assume it means money and be wrong in a way that matters. budget_daily_tokens and budget_monthly_tokens count prompt-plus-completion tokens, aligned to UTC day and calendar-month boundaries, and a request is refused once the counted window is exceeded — that refusal is real and enforced on every call. The dashboard’s cost figure next to it — cost_micros and cost_usd on every request-log row — comes from a separate, editable price table matched against the model name, and it is explicitly informational: no request is ever refused for a dollar figure, only for a token count.

This is a deliberate trade-off, not an oversight: token counts are what the gateway actually knows for certain from every provider’s usage block, while a dollar figure depends on a price table someone has to keep current. If a hard, enforced dollar ceiling is the actual requirement rather than visibility into spend, that is a real gap — see Ragmux vs LiteLLM for how another self-hosted gateway handles currency-denominated budgets directly.

Attributing spend within a project

A project’s own spend can still be split further, by the user-owned key that made each call. Every request log carries api_key_id and user_id — both null for calls made with the project’s own default key, populated for a call made with a teammate’s or a service’s own sk-user-… key. The metrics summary’s recent list and the CSV export both carry those two columns, so “which integration inside this project is actually expensive” is answerable without splitting the project itself into several smaller ones.

The CSV export (GET /admin/api/projects/{id}/metrics.csv or the instance-wide GET /admin/api/metrics/requests.csv) is built for exactly this: one row per request, oldest first, with cost_usd written to six decimal places so a cheap request doesn’t round away to nothing, alongside the model name, token counts, latency and whether the request used retrieval. New columns are only ever appended at the end, so a spreadsheet or billing pipeline that reads columns by position keeps working across upgrades.

When several integrations share one project

Not every cost center maps cleanly to one project. An internal platform team might run a single internal-tools project that several services call — a Slack bot, a nightly report generator, an ad-hoc CLI a few engineers use directly — where splitting each into its own project would mean each one also needing its own model connection and system prompt duplicated three times over for no real benefit. For that shape, a project-wide budget still caps the total, but it can’t tell a runaway nightly job from the Slack bot everyone actually depends on.

That’s what a key sub-limit is for: mint a separate sk-user-… key per integration (internal-tools/slack-bot, internal-tools/nightly-report), each with its own rate_limit_rpm or budget_daily_tokens, checked as a second ceiling under the project’s own. The token-based limits — daily and monthly budgets, tokens-per-minute — are evaluated in the order monthly, then daily, then tokens-per-minute, project before key, with the first one actually crossed producing the 429. The requests-per-minute limit works differently: both the project’s and the key’s minute slots are reserved atomically, project first, so a burst can’t slip through between the two checks. Either way, the response headers report whichever tier has less room left, so a client always sees the constraint it’s actually about to hit next, not just the project’s. It’s the difference between “this project is over budget” and “the nightly report job is over budget, and the Slack bot still has headroom” — the second is the one worth being able to say.

Watching spend before the invoice arrives

GET /admin/api/projects/{id}/usage is the live counterpart to the historical CSV: it reports the current minute, day and month windows against their configured limits, and a forecast object that extends the last 60 minutes of consumption linearly to project when a budget will actually run out. That forecast is what turns a budget from a hard stop you discover after the fact into a warning a project owner can see coming — the dashboard’s project page surfaces the same numbers as a progress ring with a reset countdown, for teams who would rather check a page than poll an endpoint.

Where this stops being the right tool

A few shapes of “tracking spend” fall outside what’s described above, worth naming rather than discovering mid-integration:

• A hard ceiling denominated in currency, not tokens. This is worth repeating plainly because it’s the single most common expectation this feature doesn’t meet: nothing in Ragmux refuses a request for crossing a dollar figure, only a token count. A team that needs “stop at $500” enforced, not just visible, needs to build that check outside Ragmux today — reading cost_usd from the metrics endpoint or the requests CSV (the usage report carries token counters and forecasts, not cost) and revoking or pausing a key programmatically once it’s crossed a threshold you define, since the gateway itself won’t do it for you.
• Billing cycles that don’t align to the UTC calendar month. budget_monthly_tokens resets at the start of the UTC calendar month, not on a customer’s own contract anniversary or fiscal period; a plan that bills from the 15th to the 15th won’t have its budget reset line up with its invoice, only the CSV export’s own window parameter will, since that one you control directly.
• A CSV pull covering more than 50,000 requests in one window. Both metrics CSV endpoints cap a single response at 50,000 rows; a project pushing enough volume to exceed that inside one export window needs to slice the export into narrower time windows and stitch the files together, rather than expecting one call to cover an arbitrarily large period.
• A cost figure precise enough to skip reconciling against the provider’s own invoice. cost_micros/cost_usd is explicitly an estimate from an editable price table, not what the provider actually billed — treat it as the number that tells you where spend is concentrated, and reconcile the total against the provider’s own dashboard before it goes on an invoice of your own.