12ec0c3fd459ef62ceda399eaa539afff8303696
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 jcoffey-dev
|
12ec0c3fd4 |
Rename the module to the Coffey-Labs organisation
The repositories moved off LINUXexpert-org. Here that is not a documentation change: the old organisation was the module path, so it is declared in go.mod and repeated in every internal import. Leaving it would have been worse than a stale link. GitHub redirects the repository, but a go.mod whose module line disagrees with the path it was fetched from is an error rather than a redirect, so `go get` on the new address would have failed against the old declaration. go.mod, 34 files of imports, and the repository links in README and ARCHITECTURE. go mod tidy leaves go.sum untouched -- no dependency moved, only our own path. |
||
|
jcoffey-dev
|
77e80cb9f2 |
Change the copyright holder to Coffey Labs
119 SPDX-FileCopyrightText headers and the README's licence line. The distinction that matters here: LINUXexpert-org appears in this repository in two completely different roles. As a copyright holder in the SPDX headers, which is what changes, and as the GitHub organisation in the module path and 64 import statements, which does not -- the repository still lives at github.com/LINUXexpert-org/stalwart-migrator, and rewriting that would not be a licence change, it would break the build. Both replacements are anchored to their copyright forms, so an import path cannot match either. Import count is 64 before and after, and go.mod is untouched. LICENSE untouched: the FSF's copyright on the GPL text and the "<name of author>" placeholders are not ours to edit. go vet, go build and go test all clean. |
||
|
jcoffey-dev
|
ad7c2d5135 |
Cut over a container, or refuse to for a reason
A container cannot be edited in place the way a unit file can, so cutting one over means rebuilding it. That makes silent loss the default failure: a container recreated without its capabilities, its custom network or its device mappings starts cleanly and is quietly not the server it was. Section 4.5 already answers this for a unit file -- it rewrites in place rather than regenerating, because a generated unit would drop hardening options this tool has no business having an opinion about, and it refuses to edit a line it only partly understands. The same rule applies here, where the whole definition has to be rebuilt: the parts this understands are carried across, and a container using anything else is refused by name rather than rebuilt without it. The list of what it looks for is conservative and not exhaustive, which is the safe direction: docker's HostConfig has far more fields, and one this does not know about is a reason not to be recreating that container at all. The old container is renamed, not removed, and nothing here prunes the old image. Together they are the container's manual restore path -- one command starts the previous container again -- which is as close to section 4.2's preserved binary as a container gets. The inspect output is preserved as an artifact before anything is replaced, for the reason the unit file is: an operator putting a machine back by hand should not also be reconstructing the definition from memory. Recovery-mode variables are stripped from the recreated container's environment. Leaving STALWART_RECOVERY_MODE set would recovery-boot on every restart, which is the same footgun the unit rewrite exists to prevent. Run now branches, with the health check and quota recalculation shared: those ask the same question whatever started the server. The binary path moved inside an else and is otherwise untouched -- no existing test needed editing, which is the evidence for that. The container path is opt-in through Options.Container, so a Docker deployment without it is still refused exactly as before. Nothing calls it yet; wiring `run` up and lifting preflight's refusal is what remains of #3, and ARCHITECTURE.md says so in both places it previously said Docker was refused outright. |
||
|
jcoffey-dev
|
2ca9522f9a |
Fix two defects a production-clone dress rehearsal exposed
Streamed a clone of a production store into the smoke VM - 3.6 GB, 12,361 settings, 6 accounts across 9 domains - and migrated it 0.15.5 -> 0.16.14 with the tool's own phases. The migration succeeded. Two defects surfaced that no smaller instance could have shown, plus one finding worth recording. 1. Account roles broke on production-shaped names. v0.16 stores an account as a local part plus a domain reference: a v0.15 account named "[email protected]" becomes name "john" with a domainId. The generator passed the full address and the server rejected it outright ("Invalid email local part"), failing the apply. The smoke instance used bare usernames - alice, bob - and never exercised this. Fixed to use the local part. And because local parts are unique only within a domain - [email protected] and [email protected] both become "postmaster" - an ambiguous one is now refused with a warning rather than risking an upsert that grants Admin to the wrong account. Verified on the clone: the one admin came out with roles {"@type": "Admin"} and the other five accounts untouched. 2. Cutover's health check conflated liveness with credentials. A config fallback-admin does not survive the migration - v0.16's config is a store pointer, so the old [authentication.fallback-admin] block simply ceases to exist - so the credentials supplied for the pre-migration instance came back 401 on the migrated one, and the check reported the service as never having answered. It had answered; it was up and serving on all ten ports. Liveness and credentials are now separate: any response proves the service is up, and credentials that stopped working are a warning that names this cause. Also recorded: a failed apply leaves the store in bootstrap mode, where only Bootstrap objects are accessible. A half-applied plan is not a partially configured server but an unusable one. Timing, which is the other reason to rehearse: the recovery-mode conversion of that 3.6 GB store took 2 seconds. A migration window is dominated by waiting and verification, not data volume. No production data in this commit; fixtures use example.net and the shapes involved. |
||
|
jcoffey-dev
|
3e155fa42c |
Fix three defects a full VM migration exposed
Ran a complete 0.15.5 -> 0.16.14 migration of the smoke VM, driving the
phases in the order the real pipeline will. It worked - all mail intact and
readable afterwards, all ten listeners up, cutover executed for the first
time ever and checkpoint resume exercised - and it exposed three defects.
1. The converted config was installed root-owned while the service runs as
its own user. Stalwart crash-looped 28 times on "Failed to read data
store settings: Permission denied", minutes after the mistake and
nowhere near it. This is the same ownership trap that retired the
rollback implementation, in a new place: writing files as root is the
natural thing for a tool running as root to do, and it is wrong every
time the service is not root.
Cutover now installs the config itself, copying ownership and mode from
the config being replaced.
2. v0.16.14 does not serve /api - the endpoint stalwartapi assumed.
Confirmed against a fully migrated, fully configured, serving instance
rather than a sandbox: /api, /api/principal and /jmap/ all 404. The JMAP
endpoint is the one the session document advertises, which is what RFC
8620 discovery is for.
The client now discovers it, re-basing the advertised path onto the
operator's host: a real instance advertises its canonical public URL
("https://mail.smoke.test/jmap/") which frequently isn't reachable from
where this tool runs. The session is authoritative about the path; the
operator is authoritative about the host.
3. Dispatching on the urn:stalwart:jmap capability was wrong, because
NEITHER version advertises it - not 0.15.5, and not a fully migrated
0.16.14. That sent 0.16 instances down the 0.15 REST path where every
call 404s. The client probes what the instance actually serves instead.
Less elegant than a declared capability, with the advantage of being
true.
Also: a JMAP "forbidden" now explains itself. An account holding the admin
role before the migration was refused x:Account/query afterwards, and a
bare "forbidden" gives an operator nowhere to start. Whether the role
failed to carry or v0.16 wants different permissions was not isolated, and
that question is recorded as open - it gates quota recalculation and any
post-migration validation.
Verified against both live instances: the 0.15.5 reports 3 accounts and its
domain over REST, and the migrated 0.16.14 routes to JMAP, finds the right
endpoint, and returns the explained refusal.
|
||
|
jcoffey-dev
|
4b0bec8956 |
Add SPDX headers to every Go file
GPLv3's "How to Apply These Terms" asks for a notice in each source file; this is the modern two-line SPDX form of it rather than the full paragraph. 82 files, including tests. The blank line after the header is load-bearing. In Go a comment block immediately preceding `package X` becomes the package doc comment, so without the separator the SPDX lines would be absorbed into the doc for the eleven packages whose doc.go (or main.go) opens with one, and `go doc` would print them. Verified it doesn't. |
||
|
jcoffey-dev
|
7e04351b0f |
Add cutover; drop rollback in favour of operator-provided recovery
Two changes that arrived together: the cutover phase (ARCHITECTURE.md 4.5) is implemented, and the rollback phase is deleted. Recovery from a failed migration is now explicitly the operator's own snapshot or backup, and out of scope for this tool. internal/cutover implements 4.5 as seven checkpointed steps: verify the staged binary's version, install it, preserve and rewrite the service definition, reload, start, wait for a healthy JMAP session, recalculate quotas. The unit is rewritten in place rather than generated from a template. An operator's unit carries hardening options, limits and dependencies this tool has no business having an opinion about, and regenerating it would silently drop them. It repoints ExecStart (preserving systemd's -@:+! prefix characters and every argument after the executable), updates --config, and strips recovery-mode Environment lines - leaving STALWART_RECOVERY_MODE=1 set would recovery-boot the service on every restart, forever. It refuses on a unit with no ExecStart, and on an Environment line mixing a recovery variable with others: a line it only partly understands is one it must not edit. Quota recalculation is the one step allowed to fail without failing the phase. Its wire format is grounded in Stalwart's x:Task schema reference - Task/set creating one AccountMaintenance per account with maintenanceType recalculateQuota - but the upgrade guide only documents the WebUI path, so two details remain inferred and are called out in stalwartapi/task.go: whether the schema's "read-only" annotation on accountId/maintenanceType means "immutable after creation", and whether a finished task simply leaves the queue (TaskStatus documents Pending/Retry/Failed with no success state). Warning rather than failing is the honest response to that uncertainty, and stale counters are an accounting problem next to calling for a restore of a machine that is otherwise migrated and serving mail. Docker deployments are refused outright: cutting a container over means pulling an image and recreating it, not swapping a binary. On removing rollback. The implementation worked and was tested, and it was removed because restoring bytes correctly is not the hard part. It copied file contents and permissions and verified every restored file against a manifest - and did not preserve ownership. Run as root, as this tool requires, it would have produced a byte-perfect, checksum-verified, root-owned data directory that Stalwart, running as its own user, could not open, and it would have reported success. The PostgreSQL path was worse: pg_dump without --clean emits CREATE TABLE + COPY, which fails replaying into a database whose tables still exist, and the ON_ERROR_STOP=1 added so a half-applied restore couldn't be reported as success turned that into a hard failure. None of it had ever run against a real server. A filesystem snapshot has none of these failure modes, because it never lost the metadata to begin with. So cutover's gate is no longer rollback.CanRollBack but an explicit RecoveryPointConfirmed acknowledgement. That is an assertion, not a check - this tool cannot verify someone else's snapshot - and its only value is that nobody migrates a production mail server having never been asked the question. Two consequences are accepted deliberately: restoring any pre-migration recovery point discards mail delivered since, and a failed migration now stops and reports rather than undoing itself. What the tool still does to make a manual restore easier: the old binary is preserved and never deleted, the original service definition is preserved before the rewrite, the settings and principals dumps stay on disk, and every artifact path and checksum stays in the checkpoint where `status <run-id>` can print it. Also removed: the `confirm` command stub and RollbackWindowClosed, whose only purpose was closing a rollback window that no longer exists, and checkpoint.PhaseRollback. Old state.json files still load - JSON ignores the now-unknown field. Still open, and recorded in 8: cutover ignores systemd drop-ins, so an ExecStart or Environment override in stalwart.service.d/*.conf is invisible to the rewrite - including the recovery variable it exists to strip; nothing prevents concurrent runs on the same run-id; and nothing in this repo has ever run against a real Stalwart, real systemd, or a real store. |