How to Set Up an LLM Gateway for AI Agents
Set up an LLM gateway in 5 minutes. Route all AI requests through one endpoint for cost tracking, security, and multi-provider support.
What you will learn
- Understand what an LLM gateway does and why you need one
- Set up a gateway and create your first API key
- Route your first LLM request through the gateway
- See costs tracked automatically per request
What Is an LLM Gateway?
An LLM gateway is a proxy that sits between your agents and LLM providers. Instead of agents calling OpenAI, Anthropic, or Google directly, they call the gateway — which routes the request to the right provider, tracks the cost, applies policies, and logs everything.
Think of it like an API gateway for AI. Nginx routes HTTP requests. An LLM gateway routes AI requests — with cost tracking, rate limiting, and policy enforcement built in.
Each agent has its own API key for each provider. Costs are scattered across 5 different billing dashboards. There is no way to enforce rate limits or block a runaway agent.
All agents use one gateway endpoint. Costs are tracked per agent, per provider, per user in a single dashboard. Rate limits, budgets, and kill-switch are enforced automatically.
Setup in 4 Steps
Create a Dobby workspace at dobby-ai.com/auth/signup. Choose your region (IL, EU, or US) — this determines where your data is stored.
Go to Gateway > API Keys and create a new key. Choose the key type: user (100 RPM), service (500 RPM), or temporary (50 RPM, auto-expires).
Install the OpenAI SDK — the gateway is compatible with the standard OpenAI API format. No custom libraries needed.
Make your first request through the gateway. The cost, latency, and token usage are tracked automatically.
Your First Gateway Request
from openai import OpenAI
# Point to Dobby Gateway instead of OpenAI directly
client = OpenAI(
base_url="https://dobby-ai.com/api/v1/gateway",
api_key="gk_user_your_key_here" # Gateway key, not OpenAI key
)
# Use any supported provider — same SDK
response = client.chat.completions.create(
model="gpt-4o", # or claude-sonnet-4-20250514, gemini-2.5-flash, etc.
messages=[{"role": "user", "content": "Summarize this document"}]
)
print(response.choices[0].message.content)The Dobby Gateway supports 13+ LLM providers through a single endpoint. Switch between OpenAI, Anthropic, Google, Mistral, and more by changing the model parameter — no code changes needed.
What Happens Behind the Scenes
- Authentication — your gateway key is validated and rate-limited
- Policy check — org policies, model restrictions, and budget limits are enforced
- Provider routing — the request is forwarded to the correct LLM provider using your configured credentials
- Cost tracking — tokens consumed and cost are calculated and logged
- Audit trail — the full request/response is stored in the immutable log
Every gateway request appears in real-time on the Live page. You can see who is calling which model, how much it costs, and how long it takes — as it happens.