Skip to main content

Built-in AI gateway

Understand, manage and optimise your LLM traffic.

Every model call your agents make goes through Distri’s gateway. It is traced end to end, attributed to the agent, the end user, the API key and the use case, held to your limits, and routed to whichever provider the agent uses. Nothing extra to deploy: it is part of every workspace.

The Usage page grouped by end user: requests, tokens and cost for each person, cost per active end user, and a limit per row
Traces: a list of runs with their cost, one run opened as a tree of agent, model and tool spans with timings, and a span's input and output

End-to-end observability

Follow every call from the agent to the model and back.

Every agent run, bot and workflow run, and direct model call is a trace: a tree of agent, model and tool spans with timings, tokens and cost. Filter by end user, use case, tag or agent, or show only the failures.

Usage attribution

Every call, split by who and what made it.

Group usage by any dimension, over today, 7 days, 30 days or this month. Costs use each provider’s list price; cached input and what caching saved are shown on their own. Tag a call with any key and value and the same tag groups this page, filters traces and can carry a limit.

  • Use case
  • Agent
  • End user
  • Source
  • Model
  • API key
One end user's page: cost in the last 30 days, cost by use case and by agent, and a Set a limit button
One end user: what they cost, by use case and by agent.

End users

Name the person once. Every surface counts them.

Your server names the person, so they cannot spoof it: per call with a header, or on a short-lived token you give to their browser. People messaging a bot on Telegram, WhatsApp or Discord are identified automatically. Every agent is an A2A-compatible endpoint, so calls from your backend or from another agent are counted the same way.

# From your server: name the person on each call
curl https://api.distri.dev/v1/agents/support_assistant \
-H "Authorization: Bearer $DISTRI_API_KEY" \
-H "X-Distri-End-User: usr_8812" \
-d '{ "jsonrpc": "2.0", "id": "1", "method": "message/send", ... }'
# Or mint a token for the signed-in person and hand it
# to the embed or @distri/react in their browser
curl https://api.distri.dev/v1/token \
-H "Authorization: Bearer $DISTRI_API_KEY" \
-d '{ "scope": "user", "identifier_id": "usr_8812" }'
# → { "access_token": "…" }

Limits

Caps checked before every provider call.

Cap tokens, cost in USD, or both, per day, per rolling week or per month. A call over the cap is refused before it reaches a model, answers 429 with when it resets, and shows as Blocked. The dashboard flags a limit that is used up.

A limit can apply to

  • Workspace
  • Agent
  • Use case
  • Tag
  • End user
  • API key
  • Embed client
The Limits page: a token cap on an agent, cost caps on two end users (one used up) and a workspace cap, each with a usage bar
# Cap one end user at $0.50 a day
curl -X PUT https://api.distri.dev/v1/gateway/limits \
-H "Authorization: Bearer $DISTRI_API_KEY" \
-d '{ "scope": "end_user", "scope_value": "user:api:usr_8812",
"period": "day", "max_cost_usd": 0.5 }'
# The next call over the cap never reaches the model:
HTTP/1.1 429 Too Many Requests
{ "message": "End user user:api:usr_8812's daily limit
of $0.50 is used up. It resets at 00:00 UTC." }

Optimise

See where the spend, tokens and failures go.

Ready-made reports for the questions finance and engineering ask: which agents and end users cost the most, what caching saved, which models fail, and what fills the input. Each exports to CSV. Build your own view in Explore and save it for the workspace.

Spend by agentWhat each agent cost, with its calls and tokens.
Spend by end userThe 20 end users who cost the most.
Spend by modelWhat each model cost.
Spend by sourceDashboard, playground, embed, API, channel, workflow or gateway.
Spend over timeCost, calls and tokens each day.
Cache savings over timeTokens read from the provider’s cache, and what it saved.
Failures by model and agentCalls the provider failed, and who made them.
Calls refused by a limitCalls stopped before they reached a model.
Largest calls by tokensThe 20 calls that used the most tokens.
What fills the inputSystem prompt, tools, skills, history, message, tool results.
The Spend by end user report: a bar chart and a table of spend, calls and tokens per person, with Export CSV

Multiple providers

Any provider. Your keys. A model per agent.

Agents run on the free tier until you add your own key; then the provider account, its rates and its data terms stay yours. Pick a default model for the workspace and a different one for any agent. Test a connection and try models in the playground.

  • OpenAI
  • Anthropic
  • Google Gemini
  • Google Vertex AI
  • Azure AI Foundry
  • AWS Bedrock
  • Alibaba Cloud
  • Z.ai
  • Cloudflare Workers AI
  • fal.ai
  • ElevenLabs
  • OpenAI-compatible
Models and keys: thirteen providers, one configured with its API token, and the models it offers with the workspace default marked

Every model call, accounted for.

The AI gateway is built into every Distri workspace, free tier included.