Files
stalwart-migrator/internal/service/service_test.go
T
jcoffey-dev 7e04351b0f Add cutover; drop rollback in favour of operator-provided recovery
Two changes that arrived together: the cutover phase (ARCHITECTURE.md 4.5)
is implemented, and the rollback phase is deleted. Recovery from a failed
migration is now explicitly the operator's own snapshot or backup, and out
of scope for this tool.

internal/cutover implements 4.5 as seven checkpointed steps: verify the
staged binary's version, install it, preserve and rewrite the service
definition, reload, start, wait for a healthy JMAP session, recalculate
quotas.

The unit is rewritten in place rather than generated from a template. An
operator's unit carries hardening options, limits and dependencies this
tool has no business having an opinion about, and regenerating it would
silently drop them. It repoints ExecStart (preserving systemd's -@:+!
prefix characters and every argument after the executable), updates
--config, and strips recovery-mode Environment lines - leaving
STALWART_RECOVERY_MODE=1 set would recovery-boot the service on every
restart, forever. It refuses on a unit with no ExecStart, and on an
Environment line mixing a recovery variable with others: a line it only
partly understands is one it must not edit.

Quota recalculation is the one step allowed to fail without failing the
phase. Its wire format is grounded in Stalwart's x:Task schema reference -
Task/set creating one AccountMaintenance per account with maintenanceType
recalculateQuota - but the upgrade guide only documents the WebUI path, so
two details remain inferred and are called out in stalwartapi/task.go:
whether the schema's "read-only" annotation on accountId/maintenanceType
means "immutable after creation", and whether a finished task simply leaves
the queue (TaskStatus documents Pending/Retry/Failed with no success
state). Warning rather than failing is the honest response to that
uncertainty, and stale counters are an accounting problem next to calling
for a restore of a machine that is otherwise migrated and serving mail.

Docker deployments are refused outright: cutting a container over means
pulling an image and recreating it, not swapping a binary.

On removing rollback. The implementation worked and was tested, and it was
removed because restoring bytes correctly is not the hard part. It copied
file contents and permissions and verified every restored file against a
manifest - and did not preserve ownership. Run as root, as this tool
requires, it would have produced a byte-perfect, checksum-verified,
root-owned data directory that Stalwart, running as its own user, could not
open, and it would have reported success. The PostgreSQL path was worse:
pg_dump without --clean emits CREATE TABLE + COPY, which fails replaying
into a database whose tables still exist, and the ON_ERROR_STOP=1 added so
a half-applied restore couldn't be reported as success turned that into a
hard failure. None of it had ever run against a real server. A filesystem
snapshot has none of these failure modes, because it never lost the
metadata to begin with.

So cutover's gate is no longer rollback.CanRollBack but an explicit
RecoveryPointConfirmed acknowledgement. That is an assertion, not a check -
this tool cannot verify someone else's snapshot - and its only value is
that nobody migrates a production mail server having never been asked the
question. Two consequences are accepted deliberately: restoring any
pre-migration recovery point discards mail delivered since, and a failed
migration now stops and reports rather than undoing itself.

What the tool still does to make a manual restore easier: the old binary is
preserved and never deleted, the original service definition is preserved
before the rewrite, the settings and principals dumps stay on disk, and
every artifact path and checksum stays in the checkpoint where `status
<run-id>` can print it.

Also removed: the `confirm` command stub and RollbackWindowClosed, whose
only purpose was closing a rollback window that no longer exists, and
checkpoint.PhaseRollback. Old state.json files still load - JSON ignores
the now-unknown field.

Still open, and recorded in 8: cutover ignores systemd drop-ins, so an
ExecStart or Environment override in stalwart.service.d/*.conf is invisible
to the rewrite - including the recovery variable it exists to strip;
nothing prevents concurrent runs on the same run-id; and nothing in this
repo has ever run against a real Stalwart, real systemd, or a real store.
2026-08-23 17:52:47 -07:00

229 lines
6.7 KiB
Go

package service
import (
"context"
"fmt"
"strings"
"testing"
"time"
)
func TestNewRefusesUnknownKind(t *testing.T) {
for _, kind := range []Kind{Unknown, "", "kubernetes"} {
if _, err := New(Options{Kind: kind}); err == nil {
t.Errorf("New(%q): want error, got nil - guessing how to stop Stalwart is exactly what this must not do", kind)
}
}
}
func TestNewDefaultsTargetNames(t *testing.T) {
systemd, err := New(Options{Kind: Systemd})
if err != nil {
t.Fatal(err)
}
if got, want := systemd.Target(), "systemd unit stalwart"; got != want {
t.Errorf("Target() = %q, want %q", got, want)
}
docker, err := New(Options{Kind: Docker})
if err != nil {
t.Fatal(err)
}
if got, want := docker.Target(), "docker container stalwart"; got != want {
t.Errorf("Target() = %q, want %q", got, want)
}
}
func TestSystemdStopStartReloadInvocations(t *testing.T) {
dir := t.TempDir()
log := argsFile(t, dir)
withFakeExecutable(t, "systemctl", fakeScriptLoggingArgs(log, "exit 0"))
c, err := New(Options{Kind: Systemd, UnitName: "stalwart-mail"})
if err != nil {
t.Fatal(err)
}
ctx := context.Background()
if err := c.Stop(ctx); err != nil {
t.Fatal(err)
}
if err := c.ReloadConfig(ctx); err != nil {
t.Fatal(err)
}
if err := c.Start(ctx); err != nil {
t.Fatal(err)
}
want := "stop stalwart-mail\ndaemon-reload\nstart stalwart-mail\n"
if got := readArgsFile(t, log); got != want {
t.Errorf("systemctl invocations:\ngot: %q\nwant: %q", got, want)
}
}
func TestSystemdStopReportsCommandFailure(t *testing.T) {
withFakeExecutable(t, "systemctl", "#!/bin/sh\necho 'Failed to stop stalwart.service: Access denied' >&2\nexit 1\n")
c, err := New(Options{Kind: Systemd})
if err != nil {
t.Fatal(err)
}
err = c.Stop(context.Background())
if err == nil {
t.Fatal("Stop: want error when systemctl fails, got nil")
}
if !strings.Contains(err.Error(), "Access denied") {
t.Errorf("Stop error %q does not carry systemctl's own output, which is the only clue an operator gets", err)
}
}
// systemctl exits non-zero for every non-active state, so Active has to
// read its output rather than its exit status - otherwise "inactive", the
// answer a caller waiting for a clean stop most needs, would look like a
// failure to read the state at all.
func TestSystemdActiveReadsOutputNotExitStatus(t *testing.T) {
for _, tc := range []struct {
state string
exitCode int
want bool
}{
{"active", 0, true},
{"activating", 3, true},
{"reloading", 3, true},
{"deactivating", 3, true}, // still holding the data directory open
{"inactive", 3, false},
{"failed", 3, false},
} {
t.Run(tc.state, func(t *testing.T) {
withFakeExecutable(t, "systemctl", fmt.Sprintf("#!/bin/sh\necho %s\nexit %d\n", tc.state, tc.exitCode))
c, err := New(Options{Kind: Systemd})
if err != nil {
t.Fatal(err)
}
active, err := c.Active(context.Background())
if err != nil {
t.Fatalf("Active() for state %q: unexpected error %v", tc.state, err)
}
if active != tc.want {
t.Errorf("Active() for state %q = %v, want %v", tc.state, active, tc.want)
}
})
}
}
func TestSystemdActiveErrorsOnUnrecognizedState(t *testing.T) {
withFakeExecutable(t, "systemctl", "#!/bin/sh\necho 'command not found'\nexit 127\n")
c, err := New(Options{Kind: Systemd})
if err != nil {
t.Fatal(err)
}
if _, err := c.Active(context.Background()); err == nil {
t.Error("Active: want error for unreadable state, got nil - an unreadable state must never collapse into 'not running'")
}
}
func TestDockerStopStartInvocations(t *testing.T) {
dir := t.TempDir()
log := argsFile(t, dir)
withFakeExecutable(t, "docker", fakeScriptLoggingArgs(log, "exit 0"))
c, err := New(Options{Kind: Docker, ContainerName: "mail"})
if err != nil {
t.Fatal(err)
}
ctx := context.Background()
if err := c.Stop(ctx); err != nil {
t.Fatal(err)
}
if err := c.Start(ctx); err != nil {
t.Fatal(err)
}
if err := c.ReloadConfig(ctx); err != nil {
t.Fatalf("ReloadConfig should be a no-op for docker: %v", err)
}
want := "stop mail\nstart mail\n"
if got := readArgsFile(t, log); got != want {
t.Errorf("docker invocations:\ngot: %q\nwant: %q", got, want)
}
}
func TestDockerActive(t *testing.T) {
for _, tc := range []struct {
out string
want bool
}{{"true", true}, {"false", false}} {
t.Run(tc.out, func(t *testing.T) {
withFakeExecutable(t, "docker", fmt.Sprintf("#!/bin/sh\necho %s\n", tc.out))
c, err := New(Options{Kind: Docker})
if err != nil {
t.Fatal(err)
}
active, err := c.Active(context.Background())
if err != nil {
t.Fatal(err)
}
if active != tc.want {
t.Errorf("Active() = %v, want %v", active, tc.want)
}
})
}
}
func TestDockerActiveErrorsWhenContainerMissing(t *testing.T) {
withFakeExecutable(t, "docker", "#!/bin/sh\necho 'Error: No such object: stalwart' >&2\nexit 1\n")
c, err := New(Options{Kind: Docker})
if err != nil {
t.Fatal(err)
}
if _, err := c.Active(context.Background()); err == nil {
t.Error("Active: want error when the container doesn't exist, got nil")
}
}
// WaitFor exists because systemctl and docker both return as soon as the
// *request* succeeded: this proves it keeps polling past a still-running
// state rather than accepting the first answer.
func TestWaitForPollsUntilStateChanges(t *testing.T) {
dir := t.TempDir()
counter := dir + "/calls"
withFakeExecutable(t, "systemctl", fmt.Sprintf(
"#!/bin/sh\nn=$(cat %[1]q 2>/dev/null || echo 0)\nn=$((n+1))\necho $n > %[1]q\n"+
"if [ $n -lt 3 ]; then echo deactivating; exit 3; fi\necho inactive; exit 3\n", counter))
c, err := New(Options{Kind: Systemd})
if err != nil {
t.Fatal(err)
}
if err := WaitFor(context.Background(), c, false, 5*time.Second); err != nil {
t.Fatalf("WaitFor: %v", err)
}
}
func TestWaitForTimesOutWhileStillActive(t *testing.T) {
withFakeExecutable(t, "systemctl", "#!/bin/sh\necho active\n")
c, err := New(Options{Kind: Systemd})
if err != nil {
t.Fatal(err)
}
err = WaitFor(context.Background(), c, false, 300*time.Millisecond)
if err == nil {
t.Fatal("WaitFor: want timeout error while the unit is still active, got nil")
}
if !strings.Contains(err.Error(), "not stopped") {
t.Errorf("WaitFor error %q should say what it was waiting for", err)
}
}
func TestWaitForSurfacesLastStateReadError(t *testing.T) {
withFakeExecutable(t, "systemctl", "#!/bin/sh\necho 'no such unit' >&2\nexit 4\n")
c, err := New(Options{Kind: Systemd})
if err != nil {
t.Fatal(err)
}
err = WaitFor(context.Background(), c, false, 300*time.Millisecond)
if err == nil {
t.Fatal("WaitFor: want error when the state can't be read at all, got nil")
}
if !strings.Contains(err.Error(), "state couldn't be read") {
t.Errorf("WaitFor error %q should distinguish 'never reached the state' from 'never could tell'", err)
}
}