LLM gateway

Every AI call. One bill, one set of rules.

Every team's model calls go through one door, so cost and data rules are set once.

About 3 weeks
to live on real traffic
Founder-led
no handoff to juniors
Fixed price
agreed before work starts
Requests Every model call, by teamLive
Request log 2,412,880 this month
TimeTeamRouteModelCostResult
10:42:11Engineeringcode-reviewLarge model$0.047Done
10:42:09Supportsummarise-ticketSmall model$0.002Fell back
10:42:08Supportsummarise-ticketLarge model$0.032Budget reached
10:42:06Financeinvoice-extractLarge model$0.064Done
10:42:05Supportdraft-replyLarge model$0.035Done
10:42:03Salesdraft-emailSmall model$0.001Done
10:42:02Supportsummarise-ticketLarge model$0.019Done
10:42:01Engineeringcode-reviewLarge model$0.061Done
10:41:58Saleslead-summarySmall model$0.002Done
10:41:57Engineeringtest-writerLarge model$0.052Done
10:41:55Supportdraft-replyLarge model$0.031Done
10:41:54Financeinvoice-extractLarge model$0.058Done
10:41:52Engineeringcode-reviewLarge model$0.044Done
10:41:51Salesdraft-emailSmall model$0.001Done
10:41:49Supportsummarise-ticketLarge model$0.022Done
10:41:48Financepo-lookupSmall model$0.001Done
Team budgets November
Support$6,000 of $6,000
On track Near cap Capped, now on small model
Engineering$7,860 of $12,000
Sales$1,700 of $4,000
Finance$1,380 of $4,000
Providers
Provider one Healthy
Provider two Healthy
The problem

AI arrived team by team

Each team picked a provider, made a key and shipped. Now the bill is one number, the keys are everywhere and nothing stops one runaway job from spending the month.

78%

of AI users bring their own AI tools to work.

Sourceblogs.microsoft.com

1 in 5

organizations had a breach caused by AI tools nobody approved.

Sourcenewsroom.ibm.com

69%

of security leaders suspect or know staff use public AI tools at work.

Sourceinfosecurity-magazine.com

How we fix it

Every model call goes through one door you own.

What changesWithout itWith it
SpendOne bill, and nobody knows whose.A budget and a report per team.
OutagesOne provider fails and features stop.Calls fall back to the next provider.
KeysCopied into many services.Held and rotated in one place.
How it works

What we build

What changes for your teams. We pick the tools to fit the systems you already run.

Team budgets November
Support$6,000 of $6,000
On track Near cap Capped, now on small model
Engineering$7,860 of $12,000
Sales$1,700 of $4,000
Finance$1,380 of $4,000
What changes

Measured against where you started

Example: Acme Inc

What changes at a company like Acme Inc

Illustrative figures, not a client result. Your own baseline is taken in the first week.

MeasureBeforeAfter
Model spend you can trace to a teamUnknownAll of it
Places that hold provider keysDozens of servicesOne
Features down in a provider outageAll on that providerNone, calls fall back
Months with a surprise billMost monthsNone, caps hold

How others built it

Another company built the same kind of system and wrote publicly about how it went.

  • Uber
    Uber’s GenAI Gateway

    What they learned

    One gateway can carry every team’s model calls, with keys, budgets and logs in one place.

    Used by close to 30 customer teams, serving 16 million queries a month.

What you keep

Everything runs in your cloud, and stays yours

You own the config, the tests and the runbooks, so you can swap any part later.

The gateway running in your cloud, configured as code in your repo.

Request log 2,412,880 this month
TimeTeamRouteModelCostResult
10:42:11Engineeringcode-reviewLarge model$0.047Done
10:42:09Supportsummarise-ticketSmall model$0.002Fell back
10:42:08Supportsummarise-ticketLarge model$0.032Budget reached
10:42:06Financeinvoice-extractLarge model$0.064Done
10:42:05Supportdraft-replyLarge model$0.035Done
10:42:03Salesdraft-emailSmall model$0.001Done
10:42:02Supportsummarise-ticketLarge model$0.019Done
10:42:01Engineeringcode-reviewLarge model$0.061Done
10:41:58Saleslead-summarySmall model$0.002Done
10:41:57Engineeringtest-writerLarge model$0.052Done
10:41:55Supportdraft-replyLarge model$0.031Done
10:41:54Financeinvoice-extractLarge model$0.058Done
10:41:52Engineeringcode-reviewLarge model$0.044Done
10:41:51Salesdraft-emailSmall model$0.001Done
10:41:49Supportsummarise-ticketLarge model$0.022Done
10:41:48Financepo-lookupSmall model$0.001Done

A key, a budget and a spend report for each team.

Team budgets November
Support$6,000 of $6,000
On track Near cap Capped, now on small model
Engineering$7,860 of $12,000
Sales$1,700 of $4,000
Finance$1,380 of $4,000
  • Fallback rules and the tests that prove them.

  • A runbook your platform team owns.

Typical building blocks: LiteLLM proxy, run from the pinned Docker image (the PyPI build was compromised on 2026-03-24).

Alternatives: agentgateway (Linux Foundation, also covers MCP and A2A); Agent Router (formerly Envoy AI Gateway) when Envoy already runs; Bifrost.

How the weeks run

Live in about 3 weeks

The work happens in your cloud, with the people who will own it.

  1. Week 1Map every model call and who owns it.
  2. Week 2Put the gateway in your cloud and move the first teams onto it.
  3. Week 3Budgets, fallbacks and traces live, then hand over the runbook.
  4. After thatRunning on real traffic

    One door for every model call.

Before we start

  • Admin access to one cloud account and to your model provider consoles.
  • A named platform owner who signs off the design.
  • About 3 hours a week from that owner, and an hour from each team we move.
Fit

Is this the right first build?

It fits when

  • Every team calls the models its own way.
  • One provider has a bad hour and features stop.
  • A team on your side will own it after we leave.

It does not fit when

  • You want a hosted tool picked and resold to you.
  • You want a strategy deck in place of working software.
  • Nobody on your side can own it after we leave.

What’s included and what isn’t

Included

  • One gateway in one cloud account.
  • Up to 3 model providers, with fallbacks between them.
  • Up to 5 teams moved onto it, each with a key and a budget.
  • A spend report for each of those teams.

Not included

  • Moving every other service: your team does that with the runbook.
  • Changes to the apps that call the models, beyond the address and the key.
  • Model provider contracts and billing.
Fixed quoteOne price, agreed before work starts

About 3 weeks. You get a price range before the call and one fixed number after it. That number is the invoice.

  • If the work takes longer than we planned, we absorb the cost.
  • You pay 50% at the start and 50% at acceptance. Builds of 6 weeks or more pay 40% at the start, 30% at the midpoint and 30% at acceptance.
  • If a build starts within 60 days of your Audit results, the full Audit fee comes off its price.
  • We start three new projects a quarter, so your start date depends on when you book.
Book a demoSee starting prices
FAQ

Questions teams ask before they book

Which gateway do you use?

We pick the one that fits the systems you already run. You own the config, the tests and the runbooks, so you can swap any part later. Most often it is a LiteLLM proxy; agentgateway, Agent Router or Bifrost when they fit better.

Does it run in our cloud?

Yes. The gateway, its logs and its keys stay in your cloud account.

Do teams have to change their code?

Only the address they call and the key they use. The gateway speaks the same interface the providers do.

What happens when a team hits its budget?

The rule you set decides: warn, move to a cheaper model or stop. Nothing is cut off without a rule you approved.

How long does it take?

About 3 weeks for the gateway and the first teams. More teams move onto it after the handover, using the runbook.

LLM gateway, running in your cloud in about 3 weeks.

One door for every model call. You own the config, the tests and the runbooks.