Streamed a clone of a production store into the smoke VM - 3.6 GB, 12,361
settings, 6 accounts across 9 domains - and migrated it 0.15.5 -> 0.16.14
with the tool's own phases. The migration succeeded. Two defects surfaced
that no smaller instance could have shown, plus one finding worth recording.
1. Account roles broke on production-shaped names. v0.16 stores an account
as a local part plus a domain reference: a v0.15 account named
"[email protected]" becomes name "john" with a domainId. The generator
passed the full address and the server rejected it outright ("Invalid
email local part"), failing the apply. The smoke instance used bare
usernames - alice, bob - and never exercised this.
Fixed to use the local part. And because local parts are unique only
within a domain - [email protected] and [email protected] both become
"postmaster" - an ambiguous one is now refused with a warning rather
than risking an upsert that grants Admin to the wrong account. Verified
on the clone: the one admin came out with roles {"@type": "Admin"} and
the other five accounts untouched.
2. Cutover's health check conflated liveness with credentials. A config
fallback-admin does not survive the migration - v0.16's config is a
store pointer, so the old [authentication.fallback-admin] block simply
ceases to exist - so the credentials supplied for the pre-migration
instance came back 401 on the migrated one, and the check reported the
service as never having answered. It had answered; it was up and serving
on all ten ports. Liveness and credentials are now separate: any
response proves the service is up, and credentials that stopped working
are a warning that names this cause.
Also recorded: a failed apply leaves the store in bootstrap mode, where
only Bootstrap objects are accessible. A half-applied plan is not a
partially configured server but an unusable one.
Timing, which is the other reason to rehearse: the recovery-mode conversion
of that 3.6 GB store took 2 seconds. A migration window is dominated by
waiting and verification, not data volume.
No production data in this commit; fixtures use example.net and the shapes
involved.
Chased down why a migrated instance had no working administrator. The
account authenticated fine and was refused every management call, and the
cause is that migrate_v016.py assigns every migrated account the User role
regardless of what it held before: an account that was `roles: ["admin"]`
in v0.15 comes out the far side as `roles: {"@type": "User"}`.
Ordinary users were never affected - User is what they had and what they
get - and their credentials, mail and mailboxes survive untouched. It is
specifically administrators who lose their privileges, which is a bad thing
to discover after cutting over.
The v0.16 shape came from the server's own schema document rather than the
published reference: GET /api/schema defines x:UserRoles as a multi-variant
type with variants User, Admin and Custom. Account is itself multi-variant,
so an upsert needs its own "@type" too - without it the server rejects the
operation outright ("upsert entry is missing `@type`").
applyplan.AccountRoleOperations restores roles from the principals dump,
emitting operations only for accounts whose role actually changes.
Rewriting every account would be a much larger blast radius for no benefit.
Where v0.15 listed several roles, admin wins - under-privileging an
administrator locks them out, which is the failure being fixed - and the
collapse is reported rather than done silently, as are roles with no known
v0.16 equivalent.
Verified end to end on the smoke VM: rehearse against the real 0.15.5 put
the role operation in the supplement, applying that supplement to a
migrated 0.16.14 whose admin was broken restored management access
(accounts=3), and alice and bob logged in over IMAPS with unchanged
credentials, read their mail, and accepted new SMTP delivery.
Also recorded: x:Account.domainId returns an internal id on v0.16, not a
domain name, so the post-migration directory comparison would read every
domain as missing. Resolving that needs an x:Domain/get call not yet
confirmed against the binary.
jcoffey-dev