mosquitto: add Eclipse Mosquitto MQTT broker cookbook

Add a `mosquitto` cookbook running Eclipse Mosquitto 2 (multi-arch
docker.io/library/eclipse-mosquitto:2) as a rootless, non-root container
(UID 10033 / GID 10000) serving as the MQTT backbone of a home-automation
stack.

Design highlights:

- Plain MQTT on 127.0.0.1:1883 only; TLS is terminated at Traefik (:443,
  SNI-routed) via a TCP router hook fragment (other/traefik/mosquitto.yaml).
  Depends on the traefik cookbook.
- Anonymous access refused; password_file and acl_file are operator-provided
  (working examples shipped under config/examples/, installed 0640 10033:10000).
  No credential is baked into a packaged file.
- Retained messages and sessions persist to the virtiofs precious-data mount
  (/var/lib/virtiofs/data/mosquitto), purged on uninstall via a tmpfiles `d$`
  entry.
- log_dest stdout so journalctl is the single place to look.
- include_dir conf.d shipped as an empty packaged directory (mosquitto fails if
  it is missing); hooks.mk lets dependent cookbooks drop conf.d fragments.
- Health check is an unauthenticated `nc -z 127.0.0.1 1883` TCP connect, so it
  needs no secret that may not exist yet.

Verified: make dryrun, whole-repo `make package`, and a podman-level functional
run (non-root startup, loopback-only bind, anonymous refused, authenticated
pub/sub round trip, retained message surviving a restart, health probe).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

cookbooks/mosquitto/Makefile

@@ -0,0 +1,26 @@
+##
+## Makefile for Mosquitto quadlet
+##
+
+DEPENDENCIES = traefik
+
+# Mosquitto runs as UID 10033 (mosquitto) / GID 10000 (itix-svc) on the host
+PROJECT_UID = 10033
+PROJECT_GID = 10000
+
+# The conf.d directory must exist even when empty: mosquitto's include_dir fails if the
+# directory is missing. Ship it as a packaged (persistent) directory.
+TARGET_FILES += $(TARGET_CHROOT)/etc/quadlets/mosquitto/conf.d
+
+# Include common Makefile
+include ../../scripts/common.mk
+
+$(TARGET_CHROOT)/etc/quadlets/mosquitto/conf.d:
+	install -d -m 0755 -o $(PROJECT_UID) -g $(PROJECT_GID) -D $@
+
+# The example password and ACL files stand in for the operator-provided secrets. Install
+# them with the same 0640 mode and 10033:10000 ownership that the operator must use,
+# overriding the default 0644 config-install rule for these two files.
+TARGET_MOSQUITTO_SECRET_EXAMPLES = $(patsubst config/examples/%, $(TARGET_CHROOT)/etc/quadlets/mosquitto/%, $(wildcard config/examples/passwd config/examples/acl))
+$(TARGET_MOSQUITTO_SECRET_EXAMPLES): $(TARGET_CHROOT)/etc/quadlets/mosquitto/%: config/examples/% $(TARGET_CHROOT)/etc/quadlets/mosquitto
+	install -m 0640 -o $(PROJECT_UID) -g $(PROJECT_GID) $< $@

cookbooks/mosquitto/README.md

@@ -0,0 +1,187 @@
+# Podman Quadlet: Mosquitto
+
+## Overview
+
+[Eclipse Mosquitto](https://mosquitto.org/) is a lightweight MQTT broker. This cookbook
+runs it as a Podman Quadlet to serve as the MQTT backbone of a home-automation stack
+(Home Assistant, `zigbee2mqtt`, ESPHome, ...).
+
+This cookbook:
+
+- Runs Mosquitto as a rootless container (UID 10033) with minimal privileges.
+- Binds a plain-MQTT listener on **loopback only** (`127.0.0.1:1883`). Nothing listens on
+  the LAN by default.
+- Terminates TLS **at Traefik** (port 443, demultiplexed by SNI), not inside the broker —
+  it depends on the `traefik` cookbook.
+- Refuses anonymous access; authentication and topic authorization come from operator-provided
+  files.
+- Stores retained messages and client sessions on the virtiofs precious-data mount
+  (`/var/lib/virtiofs/data/mosquitto/`), so the host's ZFS snapshots and backups pick them up.
+- Logs everything to stdout, so `journalctl` is the single place to look.
+- Supports automatic container image updates via Podman auto-update.
+
+The image is `docker.io/library/eclipse-mosquitto:2`, a multi-architecture manifest
+(amd64 + arm64, verified 2026-09-19, version 2.1.2), as required by the aarch64 target.
+
+## Prerequisites
+
+- The `traefik` cookbook must be installed and running (installed automatically as a
+  dependency).
+- The virtiofs data mount `/var/lib/virtiofs/data` must be present (provided by the
+  `fcos-with-virtiofs` template / the `base` cookbook).
+- The operator must provide a password file and an ACL file (see below). Until the
+  password file exists, the broker is intentionally **not** started (the units carry a
+  `ConditionPathExists=/etc/quadlets/mosquitto/passwd` guard).
+
+## Files the operator must provide
+
+Both files are provided by the operator (e.g. an Ansible role from a vault) and are **not**
+part of the package. Working examples ship under `config/examples/` for development only.
+
+| Path                                | Mode | Owner        | Purpose                                   |
+|-------------------------------------|------|--------------|-------------------------------------------|
+| `/etc/quadlets/mosquitto/passwd`    | 0640 | `10033:10000`| MQTT username/password database           |
+| `/etc/quadlets/mosquitto/acl`       | 0640 | `10033:10000`| Per-user topic authorization              |
+
+Site-specific Traefik routing (real SNI host name + certificate resolver) is injected
+downstream as an overlay into `/etc/quadlets/traefik/conf.d/` — see
+[TLS at Traefik](#tls-at-traefik) below. No site value is baked into this cookbook.
+
+### Generating a password entry
+
+Use `mosquitto_passwd` (available inside the broker image, or from the `mosquitto-clients`
+package):
+
+```sh
+# Create/replace the file and add the first user:
+mosquitto_passwd -c -b /etc/quadlets/mosquitto/passwd homeassistant 'S3cr3t!'
+# Add more users (omit -c so the file is not truncated):
+mosquitto_passwd -b /etc/quadlets/mosquitto/passwd zigbee2mqtt 'An0th3r!'
+```
+
+Then make sure ownership and mode are correct:
+
+```sh
+chown 10033:10000 /etc/quadlets/mosquitto/passwd
+chmod 0640 /etc/quadlets/mosquitto/passwd
+```
+
+The ACL file uses Mosquitto's ACL syntax; see `config/examples/acl` for a starting point.
+Because `acl_file` is set and anonymous access is refused, access is **default-deny**: an
+authenticated user can only touch the topics granted to it.
+
+## TLS at Traefik
+
+The broker speaks **plain MQTT** on loopback. Traefik terminates TLS on port 443 and
+demultiplexes connections by SNI, forwarding the plaintext stream to `127.0.0.1:1883`. This
+is expressed as a Traefik **TCP** router (not an HTTP one), shipped as a hook fragment in
+`other/traefik/mosquitto.yaml` and installed to `/etc/quadlets/traefik/conf.d/mosquitto.yaml`
+during development.
+
+What this implies for clients:
+
+- All clients are **remote** and connect over **TLS on port 443** using the broker's SNI host
+  name (e.g. `mqtt.itix.fr` in production, `mqtt` in the shipped example), *not* to port 1883.
+  Example round trip through Traefik:
+
+  ```sh
+  mosquitto_sub --cafile /path/to/ca.crt -h mqtt -p 443 -u homeassistant -P '…' -t 'test/#' &
+  mosquitto_pub --cafile /path/to/ca.crt -h mqtt -p 443 -u homeassistant -P '…' -t 'test/x' -m hello
+  ```
+
+- **Credentials travel in clear text between Traefik and the broker** over the loopback
+  interface (Traefik has already terminated TLS). This is acceptable because loopback traffic
+  never leaves the host, but it is stated here so it is not a surprise.
+
+- The shipped fragment uses a bare `tls: {}`, which makes Traefik serve its own self-signed
+  certificate — good enough to test the path end-to-end. In production the operator overlays a
+  fragment with the real SNI host name and `certResolver: le` (the certificate resolver is a
+  site value and is kept commented out in the package).
+
+A TCP router whose `HostSNI` is not `` `*` `` **requires** a `tls` section, a TCP service uses
+`address:` (never `url:`), and this router coexists on the `https` entry point with the HTTP
+routers of the other cookbooks, which match `HostSNI(`*`)` implicitly.
+
+### Making the broker listen on the LAN (optional)
+
+If an operator nevertheless wants the broker to accept connections directly on the LAN, they
+can drop a fragment such as `listener 8883 0.0.0.0` (plus its own TLS/authentication settings)
+into `/etc/quadlets/mosquitto/conf.d/`. This cookbook ships no LAN listener and no firewall
+rules.
+
+## Ports
+
+| Port | Bind        | Purpose                                             |
+|------|-------------|-----------------------------------------------------|
+| 1883 | `127.0.0.1` | Plain MQTT listener (loopback only)                 |
+| 443  | Traefik     | TLS-terminated MQTT ingress (SNI-routed by Traefik) |
+
+Port 1883 is above 1024, so the non-root container binds it without any capability.
+
+## UID and GID
+
+- User: `mosquitto`, UID **10033**
+- Group: `itix-svc`, GID **10000**
+
+## Health check
+
+The health probe is a bare TCP connect to the listener (`nc -z 127.0.0.1 1883`). It is
+preferred over a credentialed `mosquitto_sub` on `$SYS/#` because it needs **no secret** —
+the operator's password file may legitimately not be in place yet, and the probe must not
+depend on it. As a side effect, each probe produces a benign `New connection` /
+`disconnected: connection closed by client` log pair in the journal; raise `HealthInterval`
+or set `connection_messages false` in a `conf.d` fragment if that noise is unwanted.
+
+## Deviations from the repository conventions
+
+- **No TLS inside the container and no `lego` dependency.** TLS is Traefik's job, mirroring
+  the production montage (Traefik `HostSNI` + `certResolver: le` in front of port 1883). The
+  Traefik hook is a **TCP** router rather than the usual HTTP one.
+- **The whole `/etc/quadlets/mosquitto` directory is mounted read-only** into the container
+  (at the same path) rather than mounting individual files, so that `mosquitto.conf`, the
+  `conf.d/` fragments and the operator-provided `passwd`/`acl` all appear without enumerating
+  them, and host and container paths stay identical.
+- **Health check is an unauthenticated TCP connect** (see above), not the credentialed
+  `$SYS/#` subscription some deployments use.
+- **The `conf.d/` directory is packaged as an empty directory** because Mosquitto's
+  `include_dir` fails if the directory is missing (verified against 2.1.2; it is also
+  non-recursive and only reads `*.conf`).
+
+## Usage
+
+In a separate terminal, follow the logs.
+
+```sh
+sudo make tail-logs
+```
+
+Install the Podman Quadlets and start Mosquitto (pulls in `traefik`).
+
+```sh
+sudo make clean install
+```
+
+Verify the broker is listening on loopback (and nowhere else):
+
+```sh
+ss -ltnp | grep 1883
+```
+
+Publish/subscribe round trip with a password (using the shipped development credentials):
+
+```sh
+mosquitto_sub -h 127.0.0.1 -p 1883 -u homeassistant -P homeassistant -t 'test/#' &
+mosquitto_pub -h 127.0.0.1 -p 1883 -u homeassistant -P homeassistant -t 'test/x' -m hello
+```
+
+Anonymous access must be refused:
+
+```sh
+mosquitto_pub -h 127.0.0.1 -p 1883 -t 'test/x' -m nope   # -> Connection Refused: not authorised.
+```
+
+Finally, remove the quadlets, their configuration and their data.
+
+```sh
+sudo make I_KNOW_WHAT_I_AM_DOING=yes uninstall clean
+```

cookbooks/mosquitto/config/examples/acl

@@ -0,0 +1,18 @@
+# Example Mosquitto ACL file (DEVELOPMENT ONLY - do NOT use in production).
+#
+# In production this file is provided by the operator at
+# /etc/quadlets/mosquitto/acl (mode 0640, owned by 10033:10000).
+#
+# With acl_file set and allow_anonymous false, access is default-deny: an
+# authenticated user can only touch the topics granted to it here. Tighten these
+# rules per client in production (e.g. restrict zigbee2mqtt to the zigbee2mqtt/#
+# tree). The examples below grant each development user full access.
+
+user homeassistant
+topic readwrite #
+
+user zigbee2mqtt
+topic readwrite #
+
+user esphome
+topic readwrite #

cookbooks/mosquitto/config/examples/passwd

@@ -0,0 +1,15 @@
+# Example Mosquitto password file (DEVELOPMENT ONLY - do NOT use in production).
+#
+# In production this file is provided by the operator at
+# /etc/quadlets/mosquitto/passwd (mode 0640, owned by 10033:10000). Generate an
+# entry with:
+#
+#   mosquitto_passwd -b /etc/quadlets/mosquitto/passwd <username> <password>
+#
+# The username:password pairs below are, respectively:
+#   homeassistant / homeassistant
+#   zigbee2mqtt   / zigbee2mqtt
+#   esphome       / esphome
+homeassistant:$7$1000$0ca+E0ECJyUjNmtIarQIiUR0zIfkBvRuTRP/SmXyLvKIhi+YJEAeUTMfqJJWE/cpB/SqombhtS51GGXTUvm/JA==$lmvIk0V2EuBG9kzWtuIzaF58keiwX6hVCnMz/P5u2SEAtw2TneNi58IYf6YNubR88etNa26MQ9rOJVCI7VqOeA==
+zigbee2mqtt:$7$1000$KFNPZOqPGZ5tdJuX1L2Nhckm1HiQYW4rUnyB6EcZcZdckMHJaTZmhBfO5KpB13wEdO+Muyff4j2c5/iBX/Hbmg==$XvtozXmh541JUhwvFeKfr/usnmoR3VDTxPz47oAvoLJjpZupNeRmPk9JsYsVOKy55eoO9qKzeleCH2h1y0+6kA==
+esphome:$7$1000$DSDWpCqjOEWOdyQerw1yMeLAXiJAscic5GbTZ4KFrW6R1UuB+MxrRT26g1v0sQZrLXPMLc8ev/942o2yDQwGpw==$w8br+ek/wfYug0cw/43vklsnPypQVR6yhvc/WQThDK0+EvxwniJ73gQXCueL9D82wWMv26Bm9tBHpgik/RxvLQ==

cookbooks/mosquitto/config/mosquitto.conf

@@ -0,0 +1,34 @@
+# Mosquitto broker configuration (packaged, treated as read-only).
+#
+# This file ships with the cookbook. Site-specific configuration and fragments
+# contributed by dependent cookbooks go into the conf.d directory referenced by
+# the include_dir directive at the end of this file.
+
+# Send every log line to stdout so journalctl is the single place to look.
+# Do NOT configure log_dest file / do not mount /mosquitto/log.
+log_dest stdout
+
+# Persist retained messages and client sessions to the precious data directory,
+# which is virtiofs-backed and picked up by the host's ZFS snapshots and backups.
+persistence true
+persistence_location /var/lib/virtiofs/data/mosquitto/
+
+# Default listener: plain MQTT, bound to loopback only. Nothing listens on the LAN.
+# TLS is terminated by Traefik on :443 and demultiplexed by SNI (see the README and
+# other/traefik/mosquitto.yaml); the broker itself never speaks TLS.
+listener 1883 127.0.0.1
+
+# Authentication: anonymous access is refused. The password file is provided by the
+# operator (Ansible, from a vault) at the path below; a working example ships under
+# config/examples/ for development only.
+allow_anonymous false
+password_file /etc/quadlets/mosquitto/passwd
+
+# Topic authorization, also operator-provided (example under config/examples/).
+acl_file /etc/quadlets/mosquitto/acl
+
+# Site configuration and dependent-cookbook fragments are dropped here.
+# Caveats of mosquitto's include_dir (verified against 2.1.2): it is NOT recursive,
+# it only reads files matching *.conf, and it FAILS if the directory is missing.
+# The directory is therefore created unconditionally (packaged as an empty directory).
+include_dir /etc/quadlets/mosquitto/conf.d

cookbooks/mosquitto/hooks.mk

@@ -0,0 +1,17 @@
+# Mosquitto configuration fragments
+# This hook lets a dependent cookbook drop a configuration fragment into the broker's
+# include_dir by placing a file named other/mosquitto/<name>.conf in its own directory.
+# It lands in /etc/quadlets/mosquitto/conf.d/<name>.conf and is loaded by mosquitto.
+
+# Define the target files deployed to the target system when the hook is used by a
+# dependent cookbook.
+TARGET_MOSQUITTO_FILES = $(patsubst other/mosquitto/%.conf, $(TARGET_CHROOT)/etc/quadlets/mosquitto/conf.d/%.conf, $(wildcard other/mosquitto/*.conf))
+
+# Those fragments are examples for the dependent cookbooks and thus are not part of the
+# final package.
+TARGET_EXAMPLE_FILES += $(TARGET_MOSQUITTO_FILES)
+
+# Define the installation rule for the target files. Owned by the broker user so it can
+# read them (they are mounted read-only into the container).
+$(TARGET_CHROOT)/etc/quadlets/mosquitto/conf.d/%.conf: other/mosquitto/%.conf
+	install -D -m 0644 -o 10033 -g 10000 $< $@

cookbooks/mosquitto/mosquitto.container

@@ -0,0 +1,57 @@
+[Unit]
+Description=Mosquitto MQTT broker
+Documentation=https://mosquitto.org/documentation/
+After=network.target var-lib-virtiofs-data.mount
+Requires=var-lib-virtiofs-data.mount
+
+# Only start once Mosquitto has been configured (the operator provides the password file)
+ConditionPathExists=/etc/quadlets/mosquitto/passwd
+
+# Start/stop this unit when the target is started/stopped
+PartOf=mosquitto.target
+
+[Container]
+ContainerName=mosquitto
+Image=mosquitto.image
+AutoUpdate=registry
+
+# No need for root privileges: the broker binds 1883 (> 1024) as an unprivileged user.
+# The upstream entrypoint only chowns /mosquitto when it runs as root; as UID 10033 it
+# just execs the broker, so every directory it touches must already be owned by 10033:10000
+# (that is what tmpfiles.d and the config install ownership take care of).
+User=10033
+Group=10000
+
+# Network configuration (host networking is the convention across this repository)
+Network=host
+
+# Run the broker against the packaged configuration. Host and container paths are kept
+# identical so that password_file, acl_file, persistence_location and include_dir all
+# resolve to the same location inside and outside the container.
+Exec=/usr/sbin/mosquitto -c /etc/quadlets/mosquitto/mosquitto.conf
+
+# Configuration (read-only): mosquitto.conf, conf.d/ fragments and the operator-provided
+# passwd / acl files all live under /etc/quadlets/mosquitto.
+Volume=/etc/quadlets/mosquitto:/etc/quadlets/mosquitto:ro,Z
+
+# Precious data (retained messages + client sessions) on the virtiofs mount, so the
+# host's ZFS snapshots and backups pick it up.
+Volume=/var/lib/virtiofs/data/mosquitto:/var/lib/virtiofs/data/mosquitto:z
+
+# Health check: a bare TCP connect to the listener. It needs no credentials (which the
+# operator may not have provided yet), which is why it is preferred over a credentialed
+# mosquitto_sub on $SYS/#.
+HealthCmd=nc -z 127.0.0.1 1883
+HealthInterval=30s
+HealthTimeout=5s
+HealthStartPeriod=10s
+HealthRetries=3
+
+[Service]
+Restart=always
+RestartSec=10
+TimeoutStartSec=120
+TimeoutStopSec=30
+
+[Install]
+WantedBy=mosquitto.target

cookbooks/mosquitto/mosquitto.image

@@ -0,0 +1,11 @@
+[Unit]
+Description=podman pull docker.io/library/eclipse-mosquitto
+Documentation=https://hub.docker.com/_/eclipse-mosquitto
+
+# Only pull if Mosquitto has been configured (the operator provides the password file)
+ConditionPathExists=/etc/quadlets/mosquitto/passwd
+
+[Image]
+# The ":2" tag is a multi-architecture manifest (amd64 + arm64, checked 2026-09-19,
+# version 2.1.2), which is required to run on the aarch64 target.
+Image=docker.io/library/eclipse-mosquitto:2

cookbooks/mosquitto/mosquitto.target

@@ -0,0 +1,14 @@
+[Unit]
+Description=Mosquitto Service Target
+Documentation=man:systemd.target(5)
+Requires=traefik.target mosquitto.service
+After=traefik.target mosquitto.service
+
+# Allow isolation - can stop/start this target independently
+AllowIsolate=yes
+
+# Only start once Mosquitto has been configured (the operator provides the password file)
+ConditionPathExists=/etc/quadlets/mosquitto/passwd
+
+[Install]
+WantedBy=multi-user.target

cookbooks/mosquitto/other/traefik/mosquitto.yaml

@@ -0,0 +1,31 @@
+# Traefik dynamic configuration contributed by the mosquitto cookbook.
+#
+# TLS is terminated HERE (on the https / :443 entry point) and the plaintext MQTT
+# stream is forwarded to the broker on loopback. This is a TCP router, not an HTTP
+# one. It coexists on the https entry point with the HTTP routers of the other
+# cookbooks, which match HostSNI(`*`) implicitly.
+#
+# In production the SNI host name and the certificate resolver are site values,
+# injected by the operator via an overlay into /etc/quadlets/traefik/conf.d/. This
+# shipped fragment uses the bare SNI `mqtt` and a self-signed certificate so it is
+# testable as-is.
+tcp:
+  routers:
+    mosquitto:
+      # A TCP router whose HostSNI is not `*` REQUIRES a tls section (below).
+      rule: "HostSNI(`mqtt`)"
+      entryPoints:
+      - https
+      service: "mosquitto"
+      # Bare `tls: {}` makes Traefik serve its own self-signed certificate, which is
+      # what makes this fragment testable as shipped. In production, uncomment the
+      # certResolver below to obtain a real Let's Encrypt certificate (site value).
+      tls: {}
+      #tls:
+      #  certResolver: le
+  services:
+    mosquitto:
+      loadBalancer:
+        # A TCP service uses `address:`, never `url:`.
+        servers:
+        - address: "127.0.0.1:1883"

cookbooks/mosquitto/overlay.bu

@@ -0,0 +1,9 @@
+variant: fcos
+version: 1.4.0
+passwd:
+  users:
+  - name: mosquitto
+    uid: 10033
+    gecos: Mosquitto MQTT broker
+    home_dir: /var/lib/quadlets/mosquitto
+    primary_group: itix-svc

cookbooks/mosquitto/tmpfiles.d/mosquitto.conf

@@ -0,0 +1 @@
+d$ /var/lib/virtiofs/data/mosquitto 0700 10033 10000 -