docs/spec/SPEC.md

@@ -21,15 +21,20 @@ setup and recovery.
   together.
 - No web interface on the mail host. Both front ends talk to the server over
   JMAP and OAuth, and can run beside it or on another machine.
-- Stay close enough to upstream that its releases can be taken quickly.
+- ~~Stay close enough to upstream that its releases can be taken quickly.~~
+  Ended with the last import, 0.16.27 (2026-10-27): inbuxa diverges fully
+  from 27.1.
 
 **Non-goals**
 
 - Carrying, adapting or reading Stalwart Enterprise code (see §3).
 - Using Stalwart's name or logo anywhere in the product (see §2.4).
-- Changing Stalwart's storage formats beyond what upstream changes. Existing
-  Stalwart 0.16 data, including data written by Enterprise features, must
-  open unchanged (see §7).
+- ~~Changing Stalwart's storage formats beyond what upstream changes.~~
+  Reversed 2026-10-29 (John): from 27.1 the store layout is inbuxa's own,
+  marked with its own store version, and Stalwart is a migration source. A
+  Stalwart 0.16.27 store stays readable as a frozen import format
+  (`import-0.16.md`). INBUXA's own Enterprise-era data was carried over at
+  its cutover (§7).
 
 ## 2. The fork itself
 

docs/spec/import-0.16.md

@@ -0,0 +1,120 @@
+# Importing a Stalwart 0.16.27 store
+
+Status: decided 2026-10-29 (John); checked against 27.1-rc.4 the same day.
+
+From 27.1 the store's layout is inbuxa's own, marked with its own store
+version (`INBUXA_STORE_VERSION`, beside the 0.16 schema version). Stalwart is
+a migration source like any other server. What inbuxa keeps is the ability to
+read one Stalwart format, frozen at **0.16.27**, the last release it imported.
+That format is defined here and checked on every release candidate. Stalwart
+1.0 changed its own layout and isn't read by this route. It needs a converter
+of its own (inbuxa-migrate, planned).
+
+This replaces "the store format is upstream's and unchanged" in `SPEC.md` §1
+and `migration.md`. inbuxa still opens a 0.16.27 store in place today,
+because store version 1 is the 0.16 layout plus inbuxa's own subspace. That
+holds until the first layout change bumps the version. The supported route
+is the one below: export, then import into a new store.
+
+## The route
+
+1. Stop Stalwart 0.16.27 and export its store with its own binary:
+   `stalwart --config <its config> --export <dir>`.
+2. Import into an empty inbuxa store: `inbuxa --config <config> --import <dir>`.
+   The export holds the data, registry, blob, changelog, queue, report and
+   task subspaces. The full-text index isn't in it, so the import queues a
+   reindex.
+3. Start inbuxa. The source isn't touched, so rolling back means starting
+   Stalwart again.
+
+An export goes from any backend to any other. A RocksDB 0.16 store can become
+a PostgreSQL inbuxa one, or the other way round.
+
+## What carries over unchanged
+
+Checked by `tools/import-check` against the fixture in
+`tests/resources/import-0.16.27/`:
+
+- accounts with their passwords, descriptions, aliases, quotas, group
+  membership and roles, including custom roles that inherit built-in ones;
+- groups and their shared mailboxes;
+- tenants, their domains and their accounts;
+- domains and their DKIM keys (Ed25519 and RSA);
+- every built-in role's permissions;
+- mailboxes and their tree, mail with its keywords, threads, attachments,
+  sizes and received dates;
+- mailbox sharing between accounts;
+- calendars and events, with recurrence and time zones;
+- address books and cards;
+- Sieve scripts (active and inactive) and the vacation response.
+
+## What changes when inbuxa first starts
+
+Each of these is done in place on first start, and each is listed in
+`tests/resources/import-0.16.27/known-differences.txt` or tested on its own:
+
+- **Store version:** written as 1 (`crates/migration`).
+- **Roles:** inbuxa adds its Compliance Officer role. The built-in System
+  Administrator and Tenant Administrator roles gain inbuxa's own permissions
+  (account locks, audit, legal holds, journaling, DLP, mail rules,
+  deliverability, scheduled reports, security acceptance, AI explanations).
+  Nothing is taken away.
+- **Spam filter blobs** move from their pre-rename keys (`crates/migration`,
+  `rename_spam_blobs`).
+- **The web interface's OAuth client** `stalwart-webui` is retired, and an
+  application naming it moves to `inbuxa-webui` (`first_party.rs`, test
+  `renamed_identifiers`).
+- **Sieve:** scripts that `require "vnd.stalwart.expressions"` or
+  `"vnd.stalwart.while"` keep running and can be edited and saved again. The
+  old names are aliases (#212).
+- **Search:** the index is rebuilt. Each account's reindex is spread over the
+  first five minutes after start, so imported mail can be missing from
+  search results until then. New mail is indexed as it arrives.
+
+## What the operator has to change
+
+- **Clients signing in to JMAP with a password.** inbuxa takes tokens on JMAP
+  and the management API, never passwords (contract C-23). IMAP, POP3,
+  SMTP, CalDAV and CardDAV keep passwords. `INBUXA_HTTP_BASIC_AUTH=all`
+  restores passwords on JMAP for an operator who needs it while clients move
+  to OAuth.
+- **The 0.16 web interface.** Its application record carries over and points
+  at upstream's `webui` release on GitHub. inbuxa keeps an imported
+  application and fetches it (SPEC.md §5.3), so it serves that interface at
+  `/admin` unless the operator removes it. Open: whether an import should
+  disable it (see below).
+
+## Not covered
+
+- **Enterprise data.** The fixture is a Community store, and a Community
+  server has no Enterprise objects to carry. inbuxa's own cutover from an
+  Enterprise store is recorded in `compat-tests.md`.
+- **Stalwart releases other than 0.16.27.** An older 0.16 upgrades to 0.16.27
+  with Stalwart first (stalwart-migrator does this). 1.0 and later aren't
+  read by this route.
+
+## The check
+
+`tools/import-check/check.sh <image>` runs on every release candidate. It
+needs docker and about seven minutes, most of it waiting for the reindex.
+It fails on any difference that isn't in `known-differences.txt`, and on
+any of the after-import checks: delivery through the imported Sieve filter,
+search over imported mail, and re-saving a pre-rename filter. A known
+difference that stops happening is reported as `GONE`, so the list can't go
+stale without anyone noticing.
+
+The CI workflows compile the test suites but don't run them (by design), and
+they don't run this check either.
+
+`tools/import-check/make-fixture.sh` rebuilds the fixture from the official
+0.16.27 image, pinned by digest. Run it only when the fixture needs more in
+it. The server it builds has no internet access, so upstream's web
+interface bundle and the data files it downloads at first start never reach
+the fixture. The fixture holds only what `populate.py` creates, plus what
+0.16.27 creates at setup.
+
+## Open
+
+- Whether importing a 0.16 store should disable the leftover web interface
+  application (keeping the record, so the operator can switch it back on),
+  rather than serving upstream's interface at `/admin`.

docs/spec/migration.md

@@ -86,9 +86,11 @@ change a password, re-authorize a client, or re-point a DNS record. That is
 the promise the tool makes, and the verification below is what checks it
 rather than assuming it.
 
-The store format is upstream's and unchanged (SPEC.md §2.6), which is what
-makes this a copy rather than a conversion. A migration that also crosses an
-upstream version is that upgrade first, then this.
+~~The store format is upstream's and unchanged (SPEC.md §2.6), which is what
+makes this a copy rather than a conversion.~~ Since 2026-10-29 the store
+layout is inbuxa's own. A Stalwart 0.16.27 store is read as a frozen import
+format, by export and import (`import-0.16.md`). A source on an older 0.16
+is upgraded to 0.16.27 first, then imported.
 
 ## Shape
 

tests/resources/import-0.16.27/admin.json

@@ -0,0 +1 @@
+{"username": "[email protected]", "secret": "wZoDr0W3jv30PGK3"}

tests/resources/import-0.16.27/known-differences.txt

@@ -0,0 +1,42 @@
+# What inbuxa adds when it first starts on a 0.16.27 store: its own
+# Compliance Officer role, and its own permissions on the built-in
+# administrator roles. Read by tools/import-check/compare.py; one
+# difference per line, as it prints them.
++ /admin/roles/Compliance Officer
++ /admin/roles/System Administrator/enabled sysAccountLockCreate
++ /admin/roles/System Administrator/enabled sysAccountLockDestroy
++ /admin/roles/System Administrator/enabled sysAccountLockGet
++ /admin/roles/System Administrator/enabled sysAccountLockUpdate
++ /admin/roles/System Administrator/enabled sysAiExplain
++ /admin/roles/System Administrator/enabled sysAuditExport
++ /admin/roles/System Administrator/enabled sysAuditGet
++ /admin/roles/System Administrator/enabled sysAuditSettingsUpdate
++ /admin/roles/System Administrator/enabled sysComplianceGet
++ /admin/roles/System Administrator/enabled sysDeliverabilityCheck
++ /admin/roles/System Administrator/enabled sysDeliverabilityGet
++ /admin/roles/System Administrator/enabled sysDeliverabilityUpdate
++ /admin/roles/System Administrator/enabled sysDlpPolicyGet
++ /admin/roles/System Administrator/enabled sysDlpPolicyUpdate
++ /admin/roles/System Administrator/enabled sysDlpReviewGet
++ /admin/roles/System Administrator/enabled sysDlpReviewUpdate
++ /admin/roles/System Administrator/enabled sysJournalGet
++ /admin/roles/System Administrator/enabled sysJournalUpdate
++ /admin/roles/System Administrator/enabled sysLegalHoldCreate
++ /admin/roles/System Administrator/enabled sysLegalHoldExport
++ /admin/roles/System Administrator/enabled sysLegalHoldGet
++ /admin/roles/System Administrator/enabled sysLegalHoldUpdate
++ /admin/roles/System Administrator/enabled sysMailRuleGet
++ /admin/roles/System Administrator/enabled sysMailRuleUpdate
++ /admin/roles/System Administrator/enabled sysScheduledReportGet
++ /admin/roles/System Administrator/enabled sysScheduledReportUpdate
++ /admin/roles/System Administrator/enabled sysSecurityAccept
++ /admin/roles/Tenant Administrator/enabled sysAccountLockCreate
++ /admin/roles/Tenant Administrator/enabled sysAccountLockDestroy
++ /admin/roles/Tenant Administrator/enabled sysAccountLockGet
++ /admin/roles/Tenant Administrator/enabled sysAccountLockUpdate
++ /admin/roles/Tenant Administrator/enabled sysAuditExport
++ /admin/roles/Tenant Administrator/enabled sysAuditGet
++ /admin/roles/Tenant Administrator/enabled sysComplianceGet
++ /admin/roles/Tenant Administrator/enabled sysDeliverabilityGet
++ /admin/roles/Tenant Administrator/enabled sysScheduledReportGet
++ /admin/roles/Tenant Administrator/enabled sysScheduledReportUpdate

tools/import-check/after.py

@@ -0,0 +1,66 @@
+#!/usr/bin/env python3
+# SPDX-FileCopyrightText: 2026 Coffey Labs LLC
+# SPDX-License-Identifier: AGPL-3.0-only
+"""What happens on the imported store after the move: delivery through the
+imported Sieve filter, search over imported mail, and re-saving a filter that
+uses a pre-rename extension name."""
+import json, smtplib, sys, time, os
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from jmap import Jmap
+
+BASE = sys.argv[1]
+alice = Jmap(BASE, "[email protected]", "alice-fixture-pass-1")
+acct = alice.account_id()
+
+failed = []
+
+
+def ok(label, cond, detail=""):
+    print(("PASS " if cond else "FAIL ") + label + (f": {detail}" if detail and not cond else ""))
+    if not cond:
+        failed.append(label)
+
+# 1. Delivery: the imported filter files "kickoff" mail into Projects.
+with smtplib.SMTP("127.0.0.1", 18025, timeout=300) as s:
+    s.sendmail("[email protected]", ["[email protected]"],
+               b"From: [email protected]\r\nTo: [email protected]\r\nSubject: Second kickoff\r\n"
+               b"Message-ID: <[email protected]>\r\n\r\nAgain.\r\n")
+found = []
+for _ in range(30):
+    r = alice.call(("Email/query", {"accountId": acct, "filter": {"subject": "Second kickoff"}}),
+                   ("Email/get", {"accountId": acct, "#ids": {"resultOf": "0", "name": "Email/query", "path": "/ids"},
+                                  "properties": ["mailboxIds"]}),
+                   ("Mailbox/get", {"accountId": acct, "ids": None, "properties": ["name"]}))
+    found = r[1]["list"]
+    if found:
+        break
+    time.sleep(1)
+names = {m["id"]: m["name"] for m in r[2]["list"]}
+ok("delivery after import", bool(found))
+ok("imported filter ran", bool(found) and [names[b] for b in found[0]["mailboxIds"]] == ["Projects"],
+   found and [names[b] for b in found[0]["mailboxIds"]])
+
+# 2. Search over imported mail, once the queued reindex has run.
+hits = []
+# Each account's reindex is spread over the first five minutes after start.
+for _ in range(400):
+    hits = alice.call(("Email/query", {"accountId": acct, "filter": {"text": "quarterly"}}))[0]["ids"]
+    if len(hits) >= 2:
+        break
+    time.sleep(1)
+ok("search finds imported mail", len(hits) >= 2, f"{len(hits)} of 2 found")
+
+# 3. Re-saving the imported filter, which says vnd.stalwart.expressions.
+s = alice.call(("SieveScript/get", {"accountId": acct, "ids": None}))[0]["list"]
+script = next(x for x in s if x["name"] == "filters")
+blob = alice.upload(b'require ["fileinto", "vnd.stalwart.expressions"];\r\nlet "who" "envelope.from";\r\n'
+                    b'if header :contains "subject" "kickoff" { fileinto "Projects"; }\r\n', "application/sieve")
+r = alice.call(("SieveScript/set", {"accountId": acct, "update": {script["id"]: {"blobId": blob}}}))[0]
+ok("re-saving a pre-rename filter", not r.get("notUpdated"), r.get("notUpdated"))
+
+# 4. The 0.16 web interface left in the registry.
+A = json.load(open(os.path.join(os.environ["FIXTURE"], "admin.json")))
+adm = Jmap(BASE, A["username"], A["secret"], ["urn:inbuxa:jmap:registry"])
+apps = adm.call(("x:Application/get", {"ids": None}))[0]["list"]
+print("INFO applications:", [(a.get("description"), a.get("enabled"), a.get("resourceUrl")) for a in apps])
+sys.exit(1 if failed else 0)

tools/import-check/bootstrap.py

@@ -0,0 +1,14 @@
+#!/usr/bin/env python3
+# SPDX-FileCopyrightText: 2026 Coffey Labs LLC
+# SPDX-License-Identifier: AGPL-3.0-only
+"""Run the 0.16.27 setup wizard over JMAP and save the administrator it creates."""
+import sys, json, os
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from jmap import Jmap
+j = Jmap(sys.argv[1], "admin", os.environ["BOOT_PASSWORD"], ["urn:stalwart:jmap"])
+r = j.call(("x:Bootstrap/set", {"update": {"singleton": {
+    "serverHostname": "mail.example.org", "defaultDomain": "example.org",
+    "requestTlsCertificate": False, "generateDkimKeys": True,
+    "tracer": {"@type": "Stdout", "enable": True, "level": "info", "ansi": False,
+               "multiline": False, "lossy": False, "events": {}, "eventsPolicy": "exclude"}}}}))
+json.dump(j.set_ok(r[0], "updated")["singleton"], open(os.path.join(os.environ["FIXTURE"], "admin.json"), "w"), indent=2)

tools/import-check/check.sh

@@ -0,0 +1,46 @@
+#!/usr/bin/env bash
+# SPDX-FileCopyrightText: 2026 Coffey Labs LLC
+# SPDX-License-Identifier: AGPL-3.0-only
+#
+# Check that an inbuxa image still reads a Stalwart 0.16.27 store:
+#
+#   tools/import-check/check.sh registry.coffeylabs.org/inbuxa/inbuxa-server:27.1-rc.5
+#
+# Imports tests/resources/import-0.16.27/export into a fresh store with
+# `inbuxa --import`, starts the image on it, and compares what it serves with
+# what 0.16.27 served (expected.json, less known-differences.txt). Then it
+# checks what happens after the move: mail delivered through the imported
+# Sieve filter, search over imported mail once the reindex has run (up to
+# five minutes), and re-saving a filter that uses a pre-rename extension name.
+# Run it on every release candidate; see docs/spec/import-0.16.md.
+#
+# Needs docker, python3, loopback ports 18091 and 18025, and DNS (delivery
+# runs the SPF/DKIM/DMARC checks).
+set -euo pipefail
+HERE=$(cd "$(dirname "$0")" && pwd)
+ROOT=$(cd "$HERE/../.." && pwd)
+export FIXTURE="${FIXTURE:-$ROOT/tests/resources/import-0.16.27}"
+IMAGE=${1:?usage: check.sh <inbuxa image>}
+NAME=inbuxa-import-check URL=http://127.0.0.1:18091
+WORK=$(mktemp -d)
+trap 'docker rm -f $NAME >/dev/null 2>&1 || true; rm -rf "$WORK"' EXIT
+mkdir -p "$WORK/etc" "$WORK/data" && chmod 777 "$WORK/etc" "$WORK/data"
+echo '{"@type":"RocksDb","path":"/var/lib/inbuxa/"}' > "$WORK/etc/config.json" && chmod 644 "$WORK/etc/config.json"
+
+docker run --rm -v "$WORK/etc:/etc/inbuxa" -v "$WORK/data:/var/lib/inbuxa" -v "$FIXTURE/export:/import:ro" \
+  $IMAGE --config /etc/inbuxa/config.json --import /import | tail -1
+# Basic auth on JMAP is refused by default (contract C-23); the check signs in
+# with the fixture's passwords. GitHub stays unreachable, so the 0.16 web
+# interface the store still lists is never fetched.
+docker run -d --name $NAME -e INBUXA_HTTP_BASIC_AUTH=all --add-host github.com:127.0.0.1 \
+  -p 127.0.0.1:18091:8080 -p 127.0.0.1:18025:25 \
+  -v "$WORK/etc:/etc/inbuxa" -v "$WORK/data:/var/lib/inbuxa" $IMAGE >/dev/null
+for _ in $(seq 1 60); do curl -sf -o /dev/null $URL/healthz/live && break; sleep 1; done
+sleep 5
+
+python3 -I "$HERE/snapshot.py" $URL > "$WORK/actual.json"
+status=0
+python3 -I "$HERE/compare.py" "$FIXTURE/expected.json" "$WORK/actual.json" "$FIXTURE/known-differences.txt" || status=1
+python3 -I "$HERE/after.py" $URL || status=1
+docker logs $NAME 2>&1 | grep -E " (ERROR|WARN) " | grep -vE "resource.error|tls.no-certificates|DnsResolver" | head -20 || true
+exit $status

tools/import-check/compare.py

@@ -0,0 +1,44 @@
+#!/usr/bin/env python3
+# SPDX-FileCopyrightText: 2026 Coffey Labs LLC
+# SPDX-License-Identifier: AGPL-3.0-only
+"""Diff two fixture snapshots. Every difference is printed; the ones listed in
+the known-differences file (what inbuxa adds on start) are marked and don't
+fail the check, and a listed one that no longer happens is reported too.
+
+    compare.py expected.json actual.json [known-differences.txt]
+"""
+import json, sys
+
+
+def differences(x, y, path=""):
+    if isinstance(x, dict) and isinstance(y, dict):
+        for k in sorted(set(x) | set(y)):
+            p = f"{path}/{k}"
+            if k not in x:
+                yield f"+ {p}"
+            elif k not in y:
+                yield f"- {p}"
+            else:
+                yield from differences(x[k], y[k], p)
+    elif isinstance(x, list) and isinstance(y, list) and all(isinstance(v, str) for v in x + y):
+        for v in sorted(set(x) - set(y)):
+            yield f"- {path} {v}"
+        for v in sorted(set(y) - set(x)):
+            yield f"+ {path} {v}"
+    elif x != y:
+        yield f"~ {path} {json.dumps(x)} => {json.dumps(y)}"
+
+
+expected, actual = (json.load(open(f)) for f in sys.argv[1:3])
+known = set()
+if len(sys.argv) > 3:
+    known = {line.strip() for line in open(sys.argv[3]) if line.strip() and not line.startswith("#")}
+
+found = list(differences(expected, actual))
+unexpected = [d for d in found if d not in known]
+for d in found:
+    print(("  known " if d in known else "  NEW   ") + d)
+for d in sorted(known - set(found)):
+    print("  GONE  " + d)
+print(f"{len(found)} difference(s), {len(unexpected)} not known, {len(known - set(found))} known but gone")
+sys.exit(1 if unexpected else 0)

tools/import-check/jmap.py

@@ -0,0 +1,69 @@
+#!/usr/bin/env python3
+# SPDX-FileCopyrightText: 2026 Coffey Labs LLC
+# SPDX-License-Identifier: AGPL-3.0-only
+"""Tiny stdlib JMAP client for the 0.16.27 import fixture rig."""
+import base64, json, urllib.request, urllib.error
+
+CAPS = [
+    "urn:ietf:params:jmap:core", "urn:ietf:params:jmap:mail",
+    "urn:ietf:params:jmap:submission", "urn:ietf:params:jmap:sieve",
+    "urn:ietf:params:jmap:calendars", "urn:ietf:params:jmap:contacts",
+    "urn:ietf:params:jmap:principals", "urn:ietf:params:jmap:vacationresponse",
+]
+
+
+class Jmap:
+    def __init__(self, base, user, password, extra_caps=()):
+        self.base, self.user = base.rstrip("/"), user
+        tok = base64.b64encode(f"{user}:{password}".encode()).decode()
+        self.auth = {"Authorization": f"Basic {tok}"}
+        self.caps = list(CAPS) + list(extra_caps)
+        self._session = None
+
+    def _req(self, method, path, body=None, ctype="application/json"):
+        url = path if path.startswith("http") else self.base + path
+        data = body if isinstance(body, (bytes, type(None))) else json.dumps(body).encode()
+        req = urllib.request.Request(url, data=data, method=method,
+                                     headers={**self.auth, "Content-Type": ctype})
+        try:
+            with urllib.request.urlopen(req, timeout=60) as r:
+                raw = r.read()
+        except urllib.error.HTTPError as e:
+            raise RuntimeError(f"{method} {path}: HTTP {e.code}: {e.read()[:500]!r}")
+        return json.loads(raw) if raw else None
+
+    @property
+    def session(self):
+        if self._session is None:
+            self._session = self._req("GET", "/jmap/session")
+        return self._session
+
+    def account_id(self, cap="urn:ietf:params:jmap:mail"):
+        return self.session["primaryAccounts"].get(cap) or next(iter(self.session["accounts"]))
+
+    def call(self, *calls):
+        out = self._req("POST", "/jmap/", {"using": self.caps, "methodCalls": [
+            [name, args, str(i)] for i, (name, args) in enumerate(calls)]})
+        res = []
+        for name, args, _ in out["methodResponses"]:
+            if name == "error":
+                raise RuntimeError(f"JMAP error: {json.dumps(args)[:800]}")
+            res.append(args)
+        return res
+
+    def set_ok(self, resp, key="created"):
+        bad = {k: v for k, v in resp.items() if k.startswith("not") and v}
+        if bad:
+            raise RuntimeError(f"set failed: {json.dumps(bad)[:1200]}")
+        return resp.get(key) or {}
+
+    def upload(self, data, ctype="application/octet-stream", account=None):
+        account = account or self.account_id()
+        url = self.local(self.session["uploadUrl"].replace("{accountId}", account))
+        return self._req("POST", url, data, ctype)["blobId"]
+
+    def local(self, url):
+        # The session names the public hostname; the rig is on loopback.
+        import urllib.parse
+        u = urllib.parse.urlsplit(url)
+        return self.base + urllib.parse.urlunsplit(("", "", u.path, u.query, ""))

tools/import-check/make-fixture.sh

@@ -0,0 +1,47 @@
+#!/usr/bin/env bash
+# SPDX-FileCopyrightText: 2026 Coffey Labs LLC
+# SPDX-License-Identifier: AGPL-3.0-only
+#
+# Build the Stalwart 0.16.27 import fixture in tests/resources/import-0.16.27/:
+# a fresh 0.16.27 server from the official image, set up and filled over JMAP
+# (populate.py), snapshotted (expected.json), then exported with its own
+# --export. The export is what an operator moving from 0.16.27 hands to
+# `inbuxa --import`.
+#
+# The server gets no internet: GitHub resolves to loopback and DNS goes
+# nowhere, so the web interface and data files it fetches at first start
+# never land in the fixture. Only needed when the fixture itself changes;
+# check.sh is what runs against a release.
+set -euo pipefail
+HERE=$(cd "$(dirname "$0")" && pwd)
+ROOT=$(cd "$HERE/../.." && pwd)
+export FIXTURE="${FIXTURE:-$ROOT/tests/resources/import-0.16.27}"
+IMAGE=stalwartlabs/stalwart@sha256:c429b311c93db5606598abc1c68225a85a6953183848ae01bee9c6a98885836c # v0.16.27
+NAME=inbuxa-import-fixture PORT=18090 URL=http://127.0.0.1:18090
+WORK=$(mktemp -d)
+trap 'docker rm -f $NAME >/dev/null 2>&1 || true; rm -rf "$WORK"' EXIT
+mkdir -p "$WORK/etc" "$WORK/data" "$WORK/export" && chmod 777 "$WORK"/*
+export BOOT_PASSWORD=$(head -c 18 /dev/urandom | base64 | tr -d '/+=')
+
+wait_live() {
+  for _ in $(seq 1 60); do curl -sf -o /dev/null $URL/healthz/live && return; sleep 1; done
+  echo "server never came up" >&2; docker logs $NAME | tail -20 >&2; exit 1
+}
+run() {
+  docker rm -f $NAME >/dev/null 2>&1 || true
+  docker run -d --name $NAME --dns 127.0.0.1 --add-host github.com:127.0.0.1 \
+    --add-host objects.githubusercontent.com:127.0.0.1 -p 127.0.0.1:$PORT:8080 \
+    -v "$WORK/etc:/etc/stalwart" -v "$WORK/data:/var/lib/stalwart" "$@" $IMAGE >/dev/null
+  wait_live
+}
+
+run -e STALWART_RECOVERY_ADMIN="admin:$BOOT_PASSWORD"
+python3 -I "$HERE/bootstrap.py" $URL
+run
+python3 -I "$HERE/populate.py" $URL
+python3 -I "$HERE/snapshot.py" $URL > "$FIXTURE/expected.json"
+docker rm -f $NAME >/dev/null
+docker run --rm -v "$WORK/etc:/etc/stalwart" -v "$WORK/data:/var/lib/stalwart" -v "$WORK/export:/export" \
+  $IMAGE --config /etc/stalwart/config.json --export /export >/dev/null
+rm -rf "$FIXTURE/export" && cp -r "$WORK/export" "$FIXTURE/export"
+echo "fixture written: $(du -sh "$FIXTURE/export" | cut -f1) export, expected.json, admin.json"

tools/import-check/populate.py

@@ -0,0 +1,147 @@
+#!/usr/bin/env python3
+# SPDX-FileCopyrightText: 2026 Coffey Labs LLC
+# SPDX-License-Identifier: AGPL-3.0-only
+"""Fill a fresh 0.16.27 server with a small, varied data set for the import fixture."""
+import json, sys, os
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from jmap import Jmap
+
+BASE = sys.argv[1] if len(sys.argv) > 1 else "http://127.0.0.1:18090"
+A = json.load(open(os.path.join(os.environ["FIXTURE"], "admin.json")))
+adm = Jmap(BASE, A["username"], A["secret"], ["urn:stalwart:jmap"])
+PW = {"alice": "alice-fixture-pass-1", "bob": "bob-fixture-pass-2"}
+
+domain = adm.call(("x:Domain/query", {}))[0]["ids"][0]
+
+# Bootstrap makes an Ed25519 key, then an RSA one a moment later.
+import time
+for _ in range(60):
+    if len(adm.call(("x:DkimSignature/query", {}))[0]["ids"]) >= 2:
+        break
+    time.sleep(1)
+else:
+    raise SystemExit("DKIM keys never appeared")
+
+# A tenant with its own domain and one user (multi-tenancy is AGPL in 0.16).
+r = adm.call(("x:Tenant/set", {"create": {"t": {"name": "acme"}}}))
+tenant = adm.set_ok(r[0])["t"]["id"]
+r = adm.call(("x:Domain/set", {"create": {"d": {"name": "acme.example.com", "memberTenantId": tenant}}}))
+acme_domain = adm.set_ok(r[0])["d"]["id"]
+
+
+def user(name, extra=None):
+    u = {"@type": "User", "name": name, "domainId": domain, "description": f"{name.title()} Example",
+         "credentials": {"0": {"@type": "Password", "secret": PW[name]}},
+         "roles": {"@type": "User"}, "permissions": {"@type": "Inherit"}, "quotas": {},
+         "aliases": {}, "memberGroupIds": {}, "encryptionAtRest": {"@type": "Disabled"}}
+    u.update(extra or {})
+    return u
+
+
+existing = [x["id"] for x in adm.call(("x:Role/get", {"ids": None}))[0]["list"] if x.get("description") == "Helpdesk"]
+r = [{"created": {"helpdesk": {"id": existing[0]}}}] if existing else adm.call(
+    ("x:Role/set", {"create": {"helpdesk": {
+        "description": "Helpdesk", "roleIds": {},
+        "enabledPermissions": {"sysAccountGet": True, "sysAccountQuery": True, "sysQueuedMessageGet": True},
+        "disabledPermissions": {}}}}),
+)
+role = adm.set_ok(r[0])["helpdesk"]["id"]
+user_role = next(x["id"] for x in adm.call(("x:Role/get", {"ids": None}))[0]["list"] if x.get("description") == "User")
+r = adm.call(("x:Account/set", {"create": {"sales": {
+    "@type": "Group", "name": "sales", "domainId": domain, "description": "Sales team",
+    "roles": {"@type": "Default"}, "permissions": {"@type": "Inherit"}, "quotas": {}, "aliases": {}}}}))
+group = adm.set_ok(r[0])["sales"]["id"]
+r = adm.call(("x:Account/set", {"create": {
+    "alice": user("alice", {"memberGroupIds": {group: True},
+                            "aliases": {"0": {"name": "a.example", "domainId": domain, "enabled": True,
+                                              "description": None}},
+                            "roles": {"@type": "Custom", "roleIds": {user_role: True, role: True}}}),
+    "bob": user("bob", {"memberGroupIds": {group: True}, "quotas": {"maxDiskQuota": 50_000_000}}),
+}}))
+ids = adm.set_ok(r[0])
+r = adm.call(("x:Account/set", {"create": {"carol": {
+    "@type": "User", "name": "carol", "domainId": acme_domain, "description": "Carol Acme",
+    "memberTenantId": tenant, "credentials": {"0": {"@type": "Password", "secret": "carol-fixture-pass-3"}},
+    "roles": {"@type": "User"}, "permissions": {"@type": "Inherit"}, "quotas": {}, "aliases": {},
+    "memberGroupIds": {}, "encryptionAtRest": {"@type": "Disabled"}}}}))
+adm.set_ok(r[0])
+print("accounts", {k: v["id"] for k, v in ids.items()}, "group", group, "role", role)
+
+MSGS = [
+    ("Quarterly numbers", "[email protected]", "[email protected]", "<[email protected]>", None,
+     "Here are the numbers for the quarter.", ["$seen"], "Inbox"),
+    ("Re: Quarterly numbers", "[email protected]", "[email protected]", "<[email protected]>", "<[email protected]>",
+     "Thanks, looks good.", ["$seen", "$answered"], "Inbox"),
+    ("Project kickoff", "[email protected]", "[email protected]", "<[email protected]>", None,
+     "Kickoff is Monday at 10.", ["$flagged"], "Projects"),
+    ("Invoice attached", "[email protected]", "[email protected]", "<[email protected]>", None,
+     None, [], "Inbox"),
+]
+
+
+def rfc822(subject, frm, to, mid, reply, body, attach):
+    head = (f"From: {frm}\r\nTo: {to}\r\nSubject: {subject}\r\nMessage-ID: {mid}\r\n"
+            f"Date: Mon, 26 Oct 2026 09:00:00 +0000\r\nMIME-Version: 1.0\r\n")
+    if reply:
+        head += f"In-Reply-To: {reply}\r\nReferences: {reply}\r\n"
+    if not attach:
+        return (head + "Content-Type: text/plain; charset=utf-8\r\n\r\n" + body + "\r\n").encode()
+    return (head + 'Content-Type: multipart/mixed; boundary="b1"\r\n\r\n--b1\r\n'
+            "Content-Type: text/plain; charset=utf-8\r\n\r\nInvoice for October.\r\n--b1\r\n"
+            'Content-Type: application/pdf; name="invoice.pdf"\r\nContent-Disposition: attachment; '
+            'filename="invoice.pdf"\r\nContent-Transfer-Encoding: base64\r\n\r\n'
+            "JVBERi0xLjQKJcOkw7zDtsOfCjEgMCBvYmoKPDw+PgplbmRvYmoKdHJhaWxlcgo8PD4+CiUlRU9GCg==\r\n"
+            "--b1--\r\n").encode()
+
+
+def fill(name):
+    j = Jmap(BASE, f"{name}@example.org", PW[name])
+    acct = j.account_id()
+    boxes = {m["name"]: m["id"] for m in j.call(("Mailbox/get", {"accountId": acct, "ids": None}))[0]["list"]}
+    r = j.call(("Mailbox/set", {"accountId": acct, "create": {"p": {"name": "Projects"}}}))
+    boxes["Projects"] = j.set_ok(r[0])["p"]["id"]
+    r = j.call(("Mailbox/set", {"accountId": acct, "create": {"c": {"name": "2026", "parentId": boxes["Projects"]}}}))
+    boxes["2026"] = j.set_ok(r[0])["c"]["id"]
+    imports = {}
+    for i, (subj, frm, to, mid, reply, body, kw, box) in enumerate(MSGS):
+        blob = j.upload(rfc822(subj, frm, to, mid, reply, body, body is None), "message/rfc822")
+        imports[str(i)] = {"blobId": blob, "mailboxIds": {boxes[box]: True},
+                           "keywords": {k: True for k in kw}, "receivedAt": f"2026-10-26T09:0{i}:00Z"}
+    j.set_ok(j.call(("Email/import", {"accountId": acct, "emails": imports}))[0])
+
+    cals = j.call(("Calendar/get", {"accountId": acct, "ids": None}))[0]["list"]
+    r = j.call(("CalendarEvent/set", {"accountId": acct, "create": {"e": {
+        "calendarIds": {cals[0]["id"]: True}, "@type": "Event", "title": "Team standup",
+        "start": "2026-11-02T09:30:00", "timeZone": "America/Phoenix", "duration": "PT15M",
+        "recurrenceRule": {"@type": "RecurrenceRule", "frequency": "weekly", "byDay": [{"@type": "NDay", "day": "mo"}]}}}}))
+    j.set_ok(r[0])
+    books = j.call(("AddressBook/get", {"accountId": acct, "ids": None}))[0]["list"]
+    r = j.call(("ContactCard/set", {"accountId": acct, "create": {"c": {
+        "addressBookIds": {books[0]["id"]: True}, "@type": "Card", "version": "1.0", "kind": "individual",
+        "name": {"@type": "Name", "full": "Carol Client"},
+        "emails": {"e1": {"@type": "EmailAddress", "address": "[email protected]"}}}}}))
+    j.set_ok(r[0])
+
+    script = ('require ["fileinto", "vnd.stalwart.expressions"];\r\n'
+              'let "who" "envelope.from";\r\n'
+              'if header :contains "subject" "kickoff" { fileinto "Projects"; }\r\n').encode()
+    blob = j.upload(script, "application/sieve")
+    r = j.call(("SieveScript/set", {"accountId": acct, "create": {"s": {"name": "filters", "blobId": blob}},
+                                     "onSuccessActivateScript": "#s"}))
+    j.set_ok(r[0])
+    r = j.call(("VacationResponse/set", {"accountId": acct, "update": {"singleton": {
+        "isEnabled": False, "subject": "Away", "textBody": "Back on Monday."}}}))
+    j.set_ok(r[0], "updated")
+    return j, acct, boxes
+
+
+alice, a_acct, a_boxes = fill("alice")
+bob, b_acct, b_boxes = fill("bob")
+
+# bob shares his Projects folder with alice, read-only
+principals = bob.call(("Principal/query", {"accountId": b_acct, "filter": {"email": "[email protected]"}}))[0]["ids"]
+r = bob.call(("Mailbox/set", {"accountId": b_acct, "update": {b_boxes["Projects"]: {
+    "shareWith": {principals[0]: {"mayReadItems": True}}}}}))
+bob.set_ok(r[0], "updated")
+print("populated")

tools/import-check/snapshot.py

@@ -0,0 +1,106 @@
+#!/usr/bin/env python3
+# SPDX-FileCopyrightText: 2026 Coffey Labs LLC
+# SPDX-License-Identifier: AGPL-3.0-only
+"""Print a server's fixture data as id-free JSON, so a 0.16.27 source and an
+inbuxa target can be compared field by field."""
+import json, sys, urllib.request, os
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+from jmap import Jmap
+
+BASE = sys.argv[1]
+A = json.load(open(os.path.join(os.environ["FIXTURE"], "admin.json")))
+PW = {"alice": "alice-fixture-pass-1", "bob": "bob-fixture-pass-2", "carol": "carol-fixture-pass-3"}
+DOMAIN = {"carol": "acme.example.com"}
+REGISTRY_CAPS = ["urn:stalwart:jmap", "urn:inbuxa:jmap:registry"]
+
+
+def admin_client():
+    # The registry capability changed name; ask the session which one it has.
+    probe = Jmap(BASE, A["username"], A["secret"])
+    caps = [c for c in REGISTRY_CAPS if c in probe.session["capabilities"]]
+    return Jmap(BASE, A["username"], A["secret"], caps)
+
+
+def admin_view():
+    j = admin_client()
+    r = j.call(("x:Domain/get", {"ids": None}), ("x:Account/get", {"ids": None}),
+               ("x:Role/get", {"ids": None}), ("x:DkimSignature/get", {"ids": None}))
+    domains = {d["id"]: d["name"] for d in r[0]["list"]}
+    names = {a["id"]: a["name"] for a in r[1]["list"]}
+    roles = {x["id"]: x.get("description") for x in r[2]["list"]}
+    accounts = {}
+    for a in r[1]["list"]:
+        role = a.get("roles") or {}
+        accounts[a["name"]] = {
+            "type": a["@type"], "description": a.get("description"),
+            "domain": domains.get(a.get("domainId")), "tenant": a.get("memberTenantId") is not None,
+            "aliases": sorted(f'{x["name"]}@{domains.get(x["domainId"])}' for x in (a.get("aliases") or {}).values()),
+            "groups": sorted(names.get(g, g) for g in (a.get("memberGroupIds") or {})),
+            "quotas": a.get("quotas") or {},
+            "roles": [role.get("@type")] + sorted(roles.get(i, i) for i in (role.get("roleIds") or {})),
+        }
+    role_perms = {x["description"]: {"enabled": sorted(k for k, v in (x.get("enabledPermissions") or {}).items() if v),
+                                     "disabled": sorted(k for k, v in (x.get("disabledPermissions") or {}).items() if v),
+                                     "inherits": sorted(roles.get(i, i) for i in (x.get("roleIds") or {}))}
+                  for x in r[2]["list"]}
+    dkim = sorted(f'{domains.get(d.get("domainId"), "?")} {d["@type"]}' for d in r[3]["list"])
+    r = j.call(("x:Tenant/get", {"ids": None}), ("x:Application/get", {"ids": None}))
+    tenants = {t["id"]: t["name"] for t in r[0]["list"]}
+    apps = sorted(f'{x.get("description")} enabled={x.get("enabled")} {x.get("resourceUrl")}' for x in r[1]["list"])
+    return {"domains": sorted(domains.values()), "accounts": accounts, "roles": role_perms, "dkim": dkim,
+            "tenants": sorted(t["name"] for t in r[0]["list"]), "applications": apps}
+
+
+def user_view(name):
+    j = Jmap(BASE, f"{name}@{DOMAIN.get(name, 'example.org')}", PW[name])
+    acct = j.account_id()
+    r = j.call(("Mailbox/get", {"accountId": acct, "ids": None}))
+    boxes = {m["id"]: m for m in r[0]["list"]}
+
+    def path(mid):
+        m = boxes[mid]
+        return (path(m["parentId"]) + "/" if m.get("parentId") else "") + m["name"]
+
+    mailboxes = {path(m["id"]): {"role": m.get("role"), "total": m["totalEmails"], "unread": m["unreadEmails"]}
+                 for m in boxes.values()}
+    r = j.call(("Email/query", {"accountId": acct}),
+               ("Email/get", {"accountId": acct, "#ids": {"resultOf": "0", "name": "Email/query", "path": "/ids"},
+                              "properties": ["subject", "mailboxIds", "keywords", "threadId", "hasAttachment",
+                                             "size", "receivedAt", "messageId"]}))
+    threads = {}
+    for e in r[1]["list"]:
+        threads.setdefault(e["threadId"], []).append(e["subject"])
+    emails = sorted(({"subject": e["subject"], "mailboxes": sorted(path(b) for b in e["mailboxIds"]),
+                      "keywords": sorted(k for k in e["keywords"] if not k.startswith("$has")),
+                      "thread": sorted(threads[e["threadId"]]), "hasAttachment": e["hasAttachment"],
+                      "size": e["size"], "receivedAt": e["receivedAt"], "messageId": e["messageId"]}
+                     for e in r[1]["list"]), key=lambda e: e["subject"])
+    r = j.call(("CalendarEvent/get", {"accountId": acct, "ids": None}),
+               ("ContactCard/get", {"accountId": acct, "ids": None}),
+               ("SieveScript/get", {"accountId": acct, "ids": None}),
+               ("VacationResponse/get", {"accountId": acct, "ids": ["singleton"]}))
+    events = sorted([{"title": e.get("title"), "start": e.get("start"), "timeZone": e.get("timeZone"),
+                      "duration": e.get("duration"), "rrule": (e.get("recurrenceRule") or {}).get("frequency")}
+                     for e in r[0]["list"]], key=lambda e: e["title"] or "")
+    cards = sorted([{"name": (c.get("name") or {}).get("full"),
+                     "emails": sorted(x["address"] for x in (c.get("emails") or {}).values())}
+                    for c in r[1]["list"]], key=lambda c: c["name"] or "")
+    sieve = []
+    for s in r[2]["list"]:
+        url = j.session["downloadUrl"].replace("{accountId}", acct).replace("{blobId}", s["blobId"]) \
+            .replace("{name}", "s").replace("{type}", "application/sieve")
+        url = j.local(url)
+        req = urllib.request.Request(url, headers=j.auth)
+        sieve.append({"name": s["name"], "isActive": s["isActive"],
+                      "content": urllib.request.urlopen(req).read().decode()})
+    v = r[3]["list"][0]
+    vacation = {k: v.get(k) for k in ("isEnabled", "subject", "textBody")}
+    shared = sorted(a["name"] for k, a in j.session["accounts"].items() if k != acct)
+    return {"mailboxes": mailboxes, "emails": emails, "events": events, "contacts": cards,
+            "sieve": sorted(sieve, key=lambda s: s["name"]), "vacation": vacation, "sharedAccounts": shared}
+
+
+print(json.dumps({"admin": admin_view(), "alice": user_view("alice"), "bob": user_view("bob"),
+                  "carol": user_view("carol")},
+                 indent=2, sort_keys=True))