Podman Quadlet: LiteLLM

Overview

LiteLLM is an LLM gateway (proxy server) that exposes 100+ LLM providers behind a single, OpenAI-compatible API, with authentication, budgets, rate limiting and a management UI.

This cookbook:

Runs LiteLLM in proxy mode using the non-root container image (ghcr.io/berriai/litellm-non_root), which runs as UID/GID 65534. UID/GID mapping remaps it to the dedicated litellm user (10024) and the itix-svc group (10000) on the host.
Reads its behaviour from a configuration file (config.yaml).
Reads secrets and passwords (master key, UI credentials, database URL) from a separate config.env file injected as environment variables.
Redirects the root path (/) to the Admin UI (/ui) and serves the API docs under /docs.
Uses PostgreSQL as its backend (requires the postgresql cookbook) to persist keys, models and spend logs.
Is published through Traefik (requires the traefik cookbook).

Prerequisites

The postgresql cookbook must be installed and running.
The traefik cookbook must be installed and running.
Configuration files /etc/quadlets/litellm/config.yaml and /etc/quadlets/litellm/config.env must exist (copied from the examples).

Configuration

config.yaml — proxy configuration (model_list, general_settings, ...). See https://docs.litellm.ai/docs/proxy/config_settings.

config.env — secrets injected as environment variables:

Variable	Purpose
`LITELLM_MASTER_KEY`	Master key for the proxy (must start with `sk-`)
`UI_USERNAME`	Username to sign in on the Admin UI
`UI_PASSWORD`	Password to sign in on the Admin UI
`DATABASE_URL`	PostgreSQL connection string

The root redirect (ROOT_REDIRECT_URL=/ui) and docs path (DOCS_URL=/docs) are set directly in the Quadlet file.

Ports

TCP 4000: LiteLLM proxy / Admin UI (bound on 127.0.0.1, exposed through Traefik).

UID / GID

User litellm: UID 10024
Group itix-svc: GID 10000
Inside the container the process runs as 65534:65534 (mapped to the host IDs above).

Usage

In a separate terminal, follow the logs.

sudo make tail-logs

Install the Podman Quadlets and start LiteLLM.

sudo make clean install

You should see the litellm.service waiting for PostgreSQL to be available, then running its database migrations and starting up.

Verify LiteLLM is running using its liveness endpoint:

curl -sSf http://127.0.0.1:4000/health/liveliness

Then browse to the service through Traefik (the root path redirects to /ui):

curl --resolve litellm:80:127.0.0.1 -L http://litellm/

Finally, remove the quadlets, their configuration and their data.

sudo make uninstall clean

3.0 KiB Raw Blame History