llmgw
Docs/Configuration

Runtime state

Configuration is explicit and layered.

Built-in defaults, one private YAML file, and process environment overrides define the single-node gateway.

Never commit real runtime configuration or credentials. Only placeholder examples belong in the repository.

Precedence

  1. Built-in defaults.
  2. LLMGW_CONFIG, or <state>/config.yaml when unset.
  3. LLMGW_* environment overrides.

LLMGW_CONFIG_SEED copies once when the writable config is missing. Local Compose uses it for the read-only repository seed.

Provider instances

config.yaml
providers:
  local-chat:
    type: ollama
    base_url: http://127.0.0.1:11434
  hosted-chat:
    type: openai_compatible
    registry_id: openrouter
    base_url: https://openrouter.ai/api/v1
    api_key: ${ENV:OPENROUTER_API_KEY}
    timeout: 120

Other fields include region, project, location, default voice, disabled status, and experimental API adaptation. See provider integrations.

Endpoints

config.yaml
endpoints:
  coding:
    failover:
      - { provider: hosted-chat, model: <model-a> }
      - { provider: local-chat, model: <model-b> }

categories remains a deprecated fallback only when endpoints is absent.

Provider resilience policy

Loaded at startupYAML policies defaults apply to every provider, and exact provider IDs can override retry and circuit settings.

Endpoint failover is separately configured under endpoints. Circuit counters and cooldown state are process-local and reset on restart. Use the supported fields in the example configuration and read the routing boundary.

Important environment variables

VariablePurpose
LLMGW_API_KEYStatic administrator/recovery key; LLMGW_API_KEYS adds more. The data plane accepts them without key or project policy and quotas, so give clients project keys.
LLMGW_CREDENTIAL_ENCRYPTION_KEYExactly 32 decoded bytes for private connections and recoverable keys. A start with a key other than the one the database was encrypted with fails.
LLMGW_NEW_CREDENTIAL_ENCRYPTION_KEYRead only by llmgw credentials rekey: the new key it re-encrypts stored credentials with. Rotation removes it afterwards.
LLMGW_STATE_DIRWritable state root.
LLMGW_GOOGLE_ANTIGRAVITY_CLIENT_IDOperator-authorized OAuth client ID for experimental Antigravity connections.
LLMGW_GOOGLE_ANTIGRAVITY_CLIENT_SECRETMatching runtime-only OAuth client secret.
LLMGW_OAUTH_PUBLIC_BASE_URLPublic origin used for browser OAuth. Register <origin>/oauth/callback/google_antigravity as the exact redirect URI.
LLMGW_SSO_*Trusted reverse-proxy identity boundary.
LLMGW_EXTENSION_URL, LLMGW_EXTENSION_SECRETNo longer read: add the optional, separately distributed companion daemon as an extension on the console's Providers page or with POST /admin/api/extensions. While either is set, the gateway logs a startup warning.
LLMGW_LOG_REQUESTSMetadata request logging.
LLMGW_LOG_REQUEST_BODIESUnsafe explicit opt-in for prompt and response body capture.
LLMGW_RETENTION_*Usage, audit, and delivered-outbox windows.
LLMGW_BACKUP_KEEPVerified default-directory archive count.

State layout

gateway.db is authoritative for IAM, keys, encrypted connections, usage, quotas, audit, alerts, and outbox. Other files include catalog.json, telemetry.db, optional usage.db, plaintext compatibility secrets.json, OAuth cache, logs, and backups.

Private stateReal config.yaml, secrets.json, databases, caches, logs, and archives must never be committed.