Know what is live. Know how to recover it.

Use a versioned binary or image for production, bind the gateway privately, and keep all runtime configuration and credentials outside the repository.

Deployment profiles

Published image (recommended)

Use the standalone Compose recipe, pinned to an exact release image, ghcr.io/kayiik/llm-gateway:<VERSION>, from the latest release. No clone, build or config seed is required. From your private installation folder containing compose.yaml and the generate-once .env:

docker compose up -d
docker compose down

The first command starts; the second stops without deleting state. Preserve the folder, Compose project, .env and state:/state volume. Remove stale shell overrides for both keys and the optional host-port variable LLMGW_PORT. The container stays on port 8787, runs as UID/GID 65532 and already has an exec-form /llmgw health healthcheck. Do not replace it with shell/curl commands: the image has no shell.

Never regenerate the encryption key with existing state; back it up separately. The database records a check value sealed with the key, so a start with a different key fails. Use image upgrades, not a rebuild. Never use down -v for routine maintenance.

Native binary

Download, verify and unpack a release binary. It does not automatically read .env. Load saved keys and the same explicit state/config paths for every service and maintenance invocation. Native state is independent of Compose volumes.

Developers: source build

The source-only recipe uses the repository root docker-compose.yml. It builds the worktree, seeds /state/config.yaml from config.local.yaml once, and uses a separate llmgw-state volume. It publishes on 127.0.0.1:8787 unless LLMGW_HOST_PORT in that clone's private .env names another address:port; a port without an address publishes on all interfaces. After preparing that recipe and clearing stale shell overrides:

docker compose --env-file .env up -d --build

Keep source and standalone folders, projects and volumes separate. Console edits and restores change the volume copy, not the seed.

Production Compose with TLS

deploy/docker-compose.prod.yml runs the gateway behind Caddy from the image that LLMGW_IMAGE names. It has no build: section, and Compose stops with an error while the variable is unset. Set it in deploy/.env to an exact published version/digest from release notes, not a floating tag. Setup commands are in deploy/DEPLOY.md; to change images, follow the production Compose upgrade. Retain its actual files, environment, project and state directory; this is not the standalone quickstart.

Health and build identity

curl -fsS https://<GATEWAY_HOST>/health
llmgw version

Both report semantic version, source commit, and commit-derived RFC3339 time. Health proves process liveness. Use provider Test completion for inference readiness.

Request logs

Logging is off by default. Metadata logging covers POST /v1/*. Unsafe body logging requires a second explicit flag and can persist prompts, credentials, PII, and source code. Logs retain the active file and one size-rotated generation and are excluded from backup.

Retention

Retention starts asynchronously after startup and repeats every 24 hours. SQLite deletion is batched. Pending/failed outbox work, current quotas, identities, credentials, policies, and provider state are preserved.

Credential encryption key

The database records a check value sealed with LLMGW_CREDENTIAL_ENCRYPTION_KEY, and a start with any other key fails: the configured credential encryption key does not match the key this database was encrypted with.

To rotate the key, stop the gateway, create a backup and keep the current key with it, and store a new key with the deployment's secrets. Run the command below with the current key still in LLMGW_CREDENTIAL_ENCRYPTION_KEY and the new one in LLMGW_NEW_CREDENTIAL_ENCRYPTION_KEY, then replace the configured key, remove the new-key variable, and start the gateway.

llmgw credentials rekey

The command holds the state lock, so it refuses to run while a gateway does. It re-encrypts provider connections, legacy provider credentials, recoverable key copies and OAuth client profiles in one transaction and prints per-table counts, never values. A wrong current key, a value that does not decrypt, or a failed write changes nothing.

Create and inspect a backup

Check the installed binary's llmgw --help first and use these commands only if it lists them. Stop all writers and keep the service's actual state/config environment before using them.

llmgw backup create
llmgw backup inspect <ARCHIVE>

No-path create writes into the retention-managed state backup directory. A supplied external path is not pruned automatically. For Compose, stop the service and run the backup in a one-shot container of the installed image with the same environment, project and state mount; do not run a host binary against its default home instead. Copy the archive to protected storage outside the volume.

The archive includes runtime config, checkpointed gateway/telemetry/configured savings SQLite, catalog, recognized Copilot cache, and applicable legacy key/secret state. Environment keys and request logs are excluded. Protect archives from disclosure and replacement: checksums detect corruption, not authenticity.

Restore and recovery

llmgw backup inspect <ARCHIVE>
llmgw backup restore <ARCHIVE> --force
llmgw serve

Restore validates first and replaces known state; it is not a merge. An interrupted prepared transaction is rolled back on the next start. A committed transaction resumes cleanup. Newer unsupported schemas are rejected.

Keep the original encryption key separately. Configure archived external savings/cache destinations in the current process before restore.

Create, inspect and restore report credential_key: matches when the configured encryption key opens the archived database's check value, otherwise does not match, not recorded, not configured or invalid. Configure the matching key before starting on restored state.

Release evidence

Stable tags on approved main commits produce eight platform archives, matching SPDX SBOMs, verified checksums, GitHub provenance, and a versioned Linux amd64/arm64 GHCR image with SPDX/SLSA attestations. Manual dispatch is non-publishing and retains no artifact.

For a single download, compare its SHA-256 against the exact filename in SHA256SUMS before unpacking; use Get-FileHash -Algorithm SHA256 on Windows or shasum -a 256 on macOS. The full checksum command below expects the complete artifact set.

sha256sum -c SHA256SUMS
gh attestation verify llmgw_v<VERSION>_linux_amd64.tar.gz \
  --repo kayiik/llm-gateway
docker buildx imagetools inspect \
  ghcr.io/kayiik/llm-gateway:<VERSION>

Rollback

1. Stop all writers; preserve failed upgraded state separately.
2. Restore the original pre-upgrade snapshot into an empty replacement volume, including original WAL and external state. Never overlay migrated state.
3. Restore original ownership, permissions, environment, mounts and encryption key. Keep the same Compose project.
4. Start the matching previous image by version/digest.
5. Verify health, authenticated admin state, and model listing plus one completion for each human and service/project credential scope.

There is no supported database downgrade. Never run the old image against migrated state or leave newer WAL files behind. Rollback discards writes after the snapshot. Keep the original snapshot outside automatic backup retention and never run down -v during an upgrade.