LLM Gateway

LLM Gateway is an open-source, OpenAI-compatible API gateway that routes requests to 250+ models from 40+ providers, with failover, spend caps, cost analytics and optional self-hosting for teams running LLMs in production.

LLM Gateway is an open-source API gateway that sits between an application and the model providers it uses. Instead of keeping separate integrations, keys and dashboards for each vendor, a team points its existing OpenAI SDK code at a new base URL and reaches more than 250 models from over 40 providers, including OpenAI, Anthropic and Google. Switching a model or provider becomes a configuration change rather than a code rewrite. The product is aimed at product and platform teams shipping LLM features, finance teams that need to understand AI spend, and developers who want predictable coding-tool costs. In practice, a user signs up, creates an API key and either tops up credits or brings their own provider keys, which the site says is free. Every request is then logged with its model, tokens, latency, cost and cache status. The dashboard breaks usage down by project, provider, model and API key, and reports error rate and cache hit rate. Spend caps can be set per key or member, and when a provider errors, times out or rate-limits, the gateway retries up to twice on another provider for that model. Beyond the core gateway, the homepage describes five related products: the gateway itself, an observability layer, DevPass (flat-price plans for AI coding with one key across coding tools), a Lounge for chatting with and comparing models across text, image, video and audio, and guardrails. Enterprise adds SSO through SAML 2.0 with SCIM, audit logs, team budgets, guardrails that block, redact or warn on PII, secrets and prompt injection, and provider policies based on country, retention and training rules. Deployment is either hosted or self-managed. The core is licensed under AGPLv3, the enterprise code is source-available, and self-hosting runs on Docker or Kubernetes with a Helm chart. Compared with other routers and proxies, its pitch is combining routing, billing and analytics in one open codebase, with migration guides for LiteLLM, OpenRouter and GitHub Copilot users.