Install the image. Make it yours.

You need Docker Desktop (Linux containers) or Docker Engine with Compose, plus openssl on Linux/macOS or PowerShell 7 on Windows for key generation. Have a provider credential ready, use a local no-auth provider such as Ollama, or explicitly opt into the curated anonymous providers. Prefer no Docker? Download a native binary.

Create your installation folder

Choose a private folder outside any repository and open a terminal there. Create compose.yaml with these contents:

services:
  gateway:
    image: ghcr.io/kayiik/llm-gateway:<VERSION>
    ports:
      - "127.0.0.1:${LLMGW_PORT:-8787}:8787"
    environment:
      LLMGW_API_KEY: ${LLMGW_API_KEY:?set LLMGW_API_KEY in .env}
      LLMGW_CREDENTIAL_ENCRYPTION_KEY: ${LLMGW_CREDENTIAL_ENCRYPTION_KEY:?set LLMGW_CREDENTIAL_ENCRYPTION_KEY in .env}
      LLMGW_ANONYMOUS_PROVIDER_AUTOMATION: ${LLMGW_ANONYMOUS_PROVIDER_AUTOMATION:-false}
    volumes:
      - state:/state
    restart: unless-stopped

volumes:
  state:

The public image supports Linux amd64 and arm64. It already runs /llmgw serve, listens on 0.0.0.0:8787 inside the container, and includes a working exec-form healthcheck. No command or healthcheck override is needed. The volume persists configuration and databases at /state, owned by container UID/GID 65532. No config.yaml seed is required. Anonymous provider automation remains off unless you set its environment value to true.

Replace <VERSION> with the number of the latest release without its leading v; image tags omit the v, while archive names and Git tags keep it. Pin that exact version and follow Upgrading to move to a later one.

Generate .env once

For a new installation only, run one of these in the same folder. Each creates two independent random values and refuses to overwrite .env. If you already have state, restore the original keys instead, even if the file is missing.

(
  set -eu
  umask 077
  test ! -e .env || { printf '%s\n' '.env already exists; keep it.' >&2; exit 1; }
  admin=$(openssl rand -base64 32)
  encryption=$(openssl rand -base64 32)
  set -C
  printf '%s\n' "LLMGW_API_KEY=$admin" \
    "LLMGW_CREDENTIAL_ENCRYPTION_KEY=$encryption" > .env
)

PowerShell 7 on Windows

$ErrorActionPreference = 'Stop'
if (Test-Path -LiteralPath .env) { throw '.env already exists; keep it.' }
$admin = [Convert]::ToBase64String([Security.Cryptography.RandomNumberGenerator]::GetBytes(32))
$encryption = [Convert]::ToBase64String([Security.Cryptography.RandomNumberGenerator]::GetBytes(32))
New-Item -ItemType File -Path .env -Value (
  "LLMGW_API_KEY=$admin`nLLMGW_CREDENTIAL_ENCRYPTION_KEY=$encryption`n"
) | Out-Null

LLMGW_API_KEY is the administrator key. LLMGW_CREDENTIAL_ENCRYPTION_KEY encodes exactly 32 random bytes as base64 and enables encrypted provider credentials and recoverable key copies.

Start the gateway

From the folder containing compose.yaml and .env, in either shell:

docker compose up -d

Compose automatically reads .env and downloads the pinned image if missing. No --build or --env-file is needed here. Shell values override .env: remove stale LLMGW_API_KEY, LLMGW_CREDENTIAL_ENCRYPTION_KEY and LLMGW_PORT shell variables before starting. To choose another host port, add LLMGW_PORT=<HOST_PORT> to .env with your chosen number. This changes only the host mapping, not the container listener.

Open http://127.0.0.1:8787/console, substituting your host port if changed. Sign in with only the LLMGW_API_KEY value from your local file. Never paste the encryption key, file contents or authenticated output into support messages.

Check process and administrator access

The built-in healthcheck checks process liveness. You can also open http://127.0.0.1:8787/health: HTTP 200 with status: ok and build identity means the process is running, not that a provider can serve inference. The console checks administrator access; authenticated /admin/api/state reports auth_required: true.

Configure one working path

1. In Access, create a human principal and project, then assign that human owner or admin.
2. In Providers, connect one API-key, OAuth or local provider.
3. Run Sync catalog.
4. Run Test completion; catalog access alone does not prove inference. Hosted requests may be billable.
5. In Routes, create a named endpoint from catalog-backed exact models.
6. In API keys, mint a project key. Choose the human under Acts as when private OAuth is required. Keys encrypted at issuance can be revealed later by their owner or an administrator.

Send the first request

curl http://127.0.0.1:8787/v1/chat/completions \
  -H 'Authorization: Bearer <PROJECT_KEY>' \
  -H 'Content-Type: application/json' \
  -d '{"model":"<MODEL_SELECTOR>","messages":[{"role":"user","content":"Reply with: ok"}],"max_tokens":64}'

Use your chosen host port. Choose a Chat-capable <MODEL_SELECTOR> from GET /v1/models. An endpoint name stands alone, not as endpoint/model. Model listing does not prove support for every API surface. See client setup and provider limitations.

Restart, upgrade, or stop

Keep the same installation folder, .env, Compose project and state volume. Renaming or moving the folder can change the project name and select a fresh volume. Never repeat key generation.

# Start again with the saved configuration:
docker compose up -d
# Stop without deleting state:
docker compose down

down retains state; down -v deletes it. Before upgrading, back up offline, then change the image pin, pull and start. Keep the original encryption key and folder through every upgrade.

Alternative: native binary

No Docker or build tools required. This is a separate installation, not a way to access the Compose volume. In a private folder, download the archive for your OS and SHA256SUMS from the latest release. Archive names carry the release tag, shown here as v<VERSION>:

• Windows: llmgw_v<VERSION>_windows_amd64.zip (x64) or llmgw_v<VERSION>_windows_arm64.zip (ARM64)
• Linux: llmgw_v<VERSION>_linux_amd64.tar.gz (x64), llmgw_v<VERSION>_linux_arm64.tar.gz (ARM64), or llmgw_v<VERSION>_linux_riscv64.tar.gz (RISC-V 64)
• macOS: llmgw_v<VERSION>_darwin_amd64.tar.gz (Intel) or llmgw_v<VERSION>_darwin_arm64.tar.gz (Apple Silicon)
• FreeBSD: llmgw_v<VERSION>_freebsd_amd64.tar.gz (x64)

1. Calculate the archive's SHA-256 with sha256sum <ARCHIVE> (Linux), shasum -a 256 <ARCHIVE> (macOS), or Get-FileHash <ARCHIVE> -Algorithm SHA256 (PowerShell). Compare it with the exact filename in SHA256SUMS; stop if different.
2. Unpack with tar -xzf <ARCHIVE> or Expand-Archive <ARCHIVE> -DestinationPath . on Windows.
3. Generate this fresh installation's .env once as above. The binary does not automatically load it. Use the native environment-loading commands for POSIX and PowerShell before each start. Set persistent LLMGW_STATE_DIR and LLMGW_CONFIG paths, plus LLMGW_HOST=127.0.0.1 and your chosen LLMGW_PORT.
4. Run ./llmgw serve or .\llmgw.exe serve, then follow console setup above.

No initial config file is needed. Preserve the native state directory, .env and environment settings through restarts and upgrades. Service managers must load the same keys and absolute paths; without overrides native state defaults to ~/.llmgw.

Developers: from source

Only clone when you want to build the code yourself. This alternative uses a separate folder, Compose project and volume, not the standalone installation above. Replace <VERSION> as above.

git clone --branch 'v<VERSION>' https://github.com/kayiik/llm-gateway.git
cd llm-gateway

For a fresh source Compose installation, generate a private .env in this clone once, keep it out of Git, and copy llmgw.config.example.yaml to config.local.yaml without overwriting an existing file. Clear stale shell overrides for LLMGW_HOST_PORT and both keys.

docker compose --env-file .env up -d --build

The root docker-compose.yml publishes on 127.0.0.1:8787. To change that, set LLMGW_HOST_PORT, not standalone LLMGW_PORT, to address:port; a port without an address publishes on all interfaces. It seeds /state/config.yaml once from the read-only config file. Console edits update the volume copy. Its volume is llmgw-state, not standalone state. Stop with docker compose --env-file .env down.

For a Go source run instead, install the Go release that go/go.mod names, load the intended native keys and absolute state/config paths, and run go run ./cmd/llmgw serve from go/. See the full source recipe.

Common failures