llmgw

Self-hosted routing control plane

One gateway.
Your models.
Explicit routes.

Point Claude Code, Codex, Copilot CLI BYOK, SDKs, and HTTP clients at one base URL. Select an exact provider model or an ordered endpoint that can move past throttling and outages.

Install with Docker Read the docs Download release
Single Go binary OpenAI + Anthropic core Local SQLite state Single node
Keep your coding clientsDocumented HTTP profiles for Claude Code, Codex, and GitHub Copilot CLI
Speak familiar APIsCore Chat, Responses, Messages, models, embeddings, audio, image, and video routes
Route to your backendsHosted APIs, cloud platforms, local runtimes, and custom endpoints
Govern from one placeIdentity, routing policy, gateway limits, usage, audit, and local backup/restore

Addressing

Exact when you need control. Resilient when you need continuity.

A model selector has one explicit meaning. No weighted scheduler, hidden classifier, or fabricated quota signal decides for you.

provider/model

Send a request to one exact configured provider and native model ID. Provider retry policy may retry that target, but the gateway does not move it to another model.

Deterministic aliasesBare provider-native names are accepted only when they identify one unambiguous, policy-eligible exact target.

endpoint

Give an ordered list of pinned targets a stable name. Eligible Chat, Responses fallback, and adapted Messages requests can advance before output starts.

Failover is surface-specificStreaming stops moving after the first byte. Embeddings, audio, image, and video choose one eligible target.

Read routing, retry, adaptation, and cancellation details →

Interfaces

One gateway base URL, multiple client grammars.

Core protocol surfaces stay recognizable to coding CLIs and SDKs. Unsupported or lossy translated behavior fails before provider dispatch.

01 / CLIENT

Claude Code

Anthropic Messages, token counting, model discovery, tools, and streaming with strict compatibility checks.

ANTHROPIC_BASE_URL
02 / CLIENT

Codex

A custom Responses provider with native state semantics when the selected target actually supports them.

wire_api = "responses"
03 / CLIENT

Copilot CLI BYOK

Explicit Chat or Responses wire selection and exact wire model, without silently deleting native tools.

COPILOT_PROVIDER_WIRE_API
04 / API

OpenAI core

Chat Completions, Responses, models, embeddings, transcription, speech, image, and video.

/v1/chat/completions
05 / API

Anthropic core

Native or strictly adapted Messages plus native or marked estimated token counting.

/v1/messages
06 / CONTROL

Console and portal

Embedded provider, endpoint, key, access, usage, policy, alert, audit, and owner workflows.

/console · /portal

Configure a coding client → · Explore the API contract →

Authenticated quickstart

Install the image. Configure your route.

No clone or build required. Put this compose.yaml in your own private installation folder, replace <VERSION> with the latest release number without its leading v, and generate its .env once. Then run docker compose up -d and open http://127.0.0.1:8787/console. Keep the folder, keys and state volume for restarts and upgrades.

  1. Create a human owner and project.
  2. Connect and verify one provider.
  3. Create an ordered endpoint.
  4. Mint a project key for your CLI.

Follow the complete quickstart →

compose.yaml · published image
services:
  gateway:
    image: ghcr.io/kayiik/llm-gateway:<VERSION>
    ports:
      - "127.0.0.1:${LLMGW_PORT:-8787}:8787"
    environment:
      LLMGW_API_KEY: ${LLMGW_API_KEY:?set LLMGW_API_KEY in .env}
      LLMGW_CREDENTIAL_ENCRYPTION_KEY: ${LLMGW_CREDENTIAL_ENCRYPTION_KEY:?set LLMGW_CREDENTIAL_ENCRYPTION_KEY in .env}
      LLMGW_ANONYMOUS_PROVIDER_AUTOMATION: ${LLMGW_ANONYMOUS_PROVIDER_AUTOMATION:-false}
    volumes:
      - state:/state
    restart: unless-stopped

volumes:
  state:

Provider registry

Hosted APIs, private subscriptions, local inference, and custom endpoints.

Registry support describes a configuration path. Your account, deployment, location, model, and entitlement still require a catalog sync and real test completion.

GitHub CopilotClaude CodeOpenAI CodexOpenAIAnthropicGoogle GeminiGoogle AI StudioVertex AI (Agent Platform)OpenRouterGroqMistral AICerebrasAmazon BedrockAzure OpenAIOllamaLocalAIEdge TTSCustom OpenAI-compatibleOpenCode ZenKilo CodeLLM7.ioOVH AI EndpointsPollinations.aiCustom Anthropic-compatible
Local detectionProbe common Ollama, LocalAI, LM Studio, vLLM, Jan, text-generation web UI, and llama.cpp addresses. Detection reports candidates; you choose what to add.
Subscription boundariesPersonal Copilot and Codex credentials stay human-private. Provider terms and OAuth registration limits still apply.

Compare every integration and caveat →

Request-path governance

Identity and limits stay attached to every call.

Humans, services, projects, memberships, and keys resolve before routing. Project and key policy intersect; an empty eligible target set fails closed.

  • Model and provider allowlists
  • RPM and daily/monthly request limits
  • Daily input/output and monthly total tokens
  • Estimated cost and model-credit budgets
  • Persistent usage attribution and retained audit history

Understand the governance path →

Single-node operations

Built to recover, not just route.

Operational state is local and explicit: inspectable backups, bounded history, crash recovery, immutable build identity, and verified release artifacts.

STATE

Verified backups

Checkpoint configured SQLite databases, catalog, config, recognized OAuth cache, and legacy state into a bounded archive.

llmgw backup create
RECOVERY

Journaled restore

Validate before replacement, recover an interrupted prepared transaction, and reject newer unsupported schemas.

backup restore --force
RETENTION

Bounded history

Batch-prune old usage, telemetry, audit, delivered outbox, completed quota periods, logs, and verified backups.

LLMGW_RETENTION_*
PROVENANCE

Build identity

CLI, health, admin state, binaries, and images carry the same version, commit, and commit-derived time.

llmgw version
RELEASE

Supply-chain evidence

Eight archives, SPDX SBOMs, verified checksums, GitHub provenance, and two-architecture image attestations.

SHA256SUMS
ALERTS

Durable outbox

Quota and key-expiry rules enqueue deduplicated work for an operator-owned external delivery worker.

/admin/api/outbox

Open the operations runbook →

Explicit boundaries

Complex where correctness needs it. Deliberately small everywhere else.

  • Single-node SQLite, not clustered HA.
  • Core protocol compatibility, not every vendor API field.
  • Ordered failover, not quota-aware or weighted scheduling.
  • Operator-triggered local backups, not a remote backup service.
  • Trusted reverse-proxy SSO, not an embedded identity provider.
  • Private/LAN provider URLs allowed; SSRF blocking intentionally absent.

Read every limitation and non-goal →

Apache-2.0 · self-hosted

Put one private gateway between your clients and providers.

Run the published Linux amd64/arm64 image, or download a native binary for Windows, Linux or macOS. No source build needed. Start on loopback and configure your provider in the embedded console.

Install with DockerDownload release
0 results