The server: front, mta and core Deployments, Services, and the new store's seed #16

Merged
jcoffey-dev merged 2 commits from feat/server-deployments into main 2026-11-18 19:36:44 +00:00
Owner

K-C2 (tracker #14). Stacked on #15: merge that first, then this one's base becomes main.

What it adds

  • Three Deployments on one data store:
    • front (role peer): client protocols and web.
    • mta (role peer-mta): as front, plus outbound delivery.
    • core (no role): everything, exactly 1 replica. The schema refuses 2, and its rollout is maxSurge 0 so two never run at once.
  • server.dataStore becomes config.json; its secrets are File references to Secrets mounted under /etc/inbuxa/secrets.
  • Pods run non-root with every capability dropped, a read-only root and seccomp RuntimeDefault. The service-account token isn't mounted.
  • Probes: startup and liveness on /healthz/live, readiness on /healthz/ready.
  • A mail LoadBalancer Service (front + mta, externalTrafficPolicy: Local), an http ClusterIP Service (front + core), PDBs, and spreading across nodes (hard) and zones (soft).

Standard ports, no privilege. The pods set the namespaced sysctl net.ipv4.ip_unprivileged_port_start to the lowest listener port. Pod Security "restricted" allows that sysctl. I first used high ports (2525 and so on) and found SMTP has to stay on 25: the server's default rules tell mail from the internet from submissions by local_port == 25, so on 2525 inbound mail got "503 You must authenticate first". The schema now allows only 25 or 0 for SMTP. kube-linter's privileged-ports and unsafe-sysctls are excluded with reasons (its sysctl list predates 1.22).

Tested on kind (k8s 1.37, 3 workers, CNPG + NATS + Garage + MetalLB), image built from server main + #449:

  • 5 pods ready, spread over the 3 workers. Each listens on 25, 443, 465, 993, 4190 and 8080 with CapEff 0.
  • Mail test (inbuxa-k8s-lab lab/chart/mailtest.py): SMTP on the LoadBalancer's port 25 → found over IMAPS on 993 after 0.5 s. The client's own address reached the pod. The message lands in Junk: the lab has no DNS to pass SPF/DKIM, and the filter is right.
  • tools/check.sh: kubeconform 9/9 on 1.35, 1.36 and 1.37; kube-linter clean; helm-unittest 22/22.

Setting up a new store: server.seed (needs server #451, --seed). A pre-install Job runs inbuxa --seed before any server pod starts. The seed is a JMAP request body from values or an existing ConfigMap. It creates the domain, administrator, blob store, coordinator and cluster roles. Its secrets come from seed.secretMounts, mounted only in the Job. backoffLimit: 0, so a partial run is never repeated. The Job brings its own config.json, because pre-install hooks run before the chart's ConfigMap exists. If existingConfigMap names the chart's own seed ConfigMap, the chart refuses it (found on kind: the hook replaced the user's ConfigMap).

Tested on kind with the seed: empty database → one helm install → 5 pods ready in 12 s, all starting at once, 0 restarts. Mail in on 25 found over IMAPS. The seeded admin signs in. Mail alice submits is DKIM-signed for d=lab.test, so the seeded domain got its keys. tools/check.sh: 31/31 unit tests, lint clean.

This also ends the startup race I saw earlier (every pod writing the defaults of an empty store at once): the seed run writes them before any pod starts.

Also found: the release-features build of main was broken (#449, server).

K-C2 (tracker #14). Stacked on #15: merge that first, then this one's base becomes `main`. **What it adds** - Three Deployments on one data store: - `front` (role `peer`): client protocols and web. - `mta` (role `peer-mta`): as front, plus outbound delivery. - `core` (no role): everything, exactly 1 replica. The schema refuses 2, and its rollout is maxSurge 0 so two never run at once. - `server.dataStore` becomes `config.json`; its secrets are File references to Secrets mounted under `/etc/inbuxa/secrets`. - Pods run non-root with every capability dropped, a read-only root and seccomp RuntimeDefault. The service-account token isn't mounted. - Probes: startup and liveness on `/healthz/live`, readiness on `/healthz/ready`. - A `mail` LoadBalancer Service (front + mta, `externalTrafficPolicy: Local`), an `http` ClusterIP Service (front + core), PDBs, and spreading across nodes (hard) and zones (soft). **Standard ports, no privilege.** The pods set the namespaced sysctl `net.ipv4.ip_unprivileged_port_start` to the lowest listener port. Pod Security "restricted" allows that sysctl. I first used high ports (2525 and so on) and found SMTP has to stay on 25: the server's default rules tell mail from the internet from submissions by `local_port == 25`, so on 2525 inbound mail got "503 You must authenticate first". The schema now allows only 25 or 0 for SMTP. kube-linter's `privileged-ports` and `unsafe-sysctls` are excluded with reasons (its sysctl list predates 1.22). **Tested on kind** (k8s 1.37, 3 workers, CNPG + NATS + Garage + MetalLB), image built from server main + #449: - 5 pods ready, spread over the 3 workers. Each listens on 25, 443, 465, 993, 4190 and 8080 with CapEff 0. - Mail test (inbuxa-k8s-lab `lab/chart/mailtest.py`): SMTP on the LoadBalancer's port 25 → found over IMAPS on 993 after 0.5 s. The client's own address reached the pod. The message lands in Junk: the lab has no DNS to pass SPF/DKIM, and the filter is right. - `tools/check.sh`: kubeconform 9/9 on 1.35, 1.36 and 1.37; kube-linter clean; helm-unittest 22/22. **Setting up a new store: `server.seed`** (needs server #451, `--seed`). A pre-install Job runs `inbuxa --seed` before any server pod starts. The seed is a JMAP request body from values or an existing ConfigMap. It creates the domain, administrator, blob store, coordinator and cluster roles. Its secrets come from `seed.secretMounts`, mounted only in the Job. `backoffLimit: 0`, so a partial run is never repeated. The Job brings its own `config.json`, because pre-install hooks run before the chart's ConfigMap exists. If `existingConfigMap` names the chart's own seed ConfigMap, the chart refuses it (found on kind: the hook replaced the user's ConfigMap). **Tested on kind with the seed:** empty database → one `helm install` → 5 pods ready in 12 s, all starting at once, 0 restarts. Mail in on 25 found over IMAPS. The seeded admin signs in. Mail alice submits is DKIM-signed for `d=lab.test`, so the seeded domain got its keys. `tools/check.sh`: 31/31 unit tests, lint clean. This also ends the startup race I saw earlier (every pod writing the defaults of an empty store at once): the seed run writes them before any pod starts. **Also found:** the release-features build of main was broken (#449, server).
jcoffey-dev changed title from The server: front, mta and core Deployments, mail and http Services to The server: front, mta and core Deployments, Services, and the new store's seed 2026-11-18 18:59:45 +00:00
jcoffey-dev changed target branch from feat/repo-setup to main 2026-11-18 19:36:10 +00:00
jcoffey-dev added 2 commits 2026-11-18 19:36:10 +00:00
Three Deployments on one data store (config.json from values, secrets
mounted from Secrets): front for client protocols and web, mta adding
outbound delivery, core running everything with exactly one replica and
a rollout that never runs two.

Pods run non-root, every capability dropped, read-only root, seccomp
RuntimeDefault. They listen on the standard mail ports through the
namespaced sysctl net.ipv4.ip_unprivileged_port_start, which Pod
Security "restricted" allows. SMTP has to stay on 25: the server's
default rules tell inbound mail from submissions by local port 25.

A mail LoadBalancer Service (front and mta, externalTrafficPolicy Local)
and an http ClusterIP Service (front and core), PodDisruptionBudgets and
spreading across nodes and zones.
server.seed: set the new store up on install
ci / chart (pull_request) Successful in 46s
7baae02089
A pre-install Job runs the server's --seed with the seed from values or
an existing ConfigMap, before any server pod starts: the domain,
administrator, blob store, coordinator and cluster roles the groups
name. Its secrets come from seed.secretMounts, mounted only in the Job.
backoffLimit 0: a run that stopped partway must not apply part of the
seed twice. The Job brings its own config.json, since pre-install hooks
run before the chart's ConfigMap exists.
jcoffey-dev merged commit 4db84e7fe7 into main 2026-11-18 19:36:44 +00:00
jcoffey-dev deleted branch feat/server-deployments 2026-11-18 19:36:44 +00:00
Sign in to join this conversation.