Fix the domain/tenant mismatch that failed the second live migration
The second production attempt failed during recovery-mode apply, with the
mail server already stopped and the store already at schema v6:
create Account restore-13: invalidForeignKey | Object id: Domain#d
v0.16 requires a tenant-scoped Account to sit on a Domain owned by that
same tenant, for its primary domain and for every alias. v0.15 imposed no
such rule, and migrate_v016.py carries the two facts over independently:
_build_domains sets a domain's memberTenantId only for domains declared as
their own `domain` principal with a `tenant`, while _build_user sets the
account's from the account's own record. A domain that exists only inside
an email address is inferred, gets no tenant, and every tenant-scoped
account using it is then rejected.
Established by reproduction rather than inference: a synthetic v0.15
principal dump, run through the unpatched upstream converter and applied to
a real 0.16.14 in recovery mode, reproduces the error character for
character - the `#d` is the server's own object id for the offending
domain, not a plan client-id. The same harness establishes which directions
are constrained: a tenant-scoped account on a tenant-less domain or on
another tenant's domain is rejected; a global account on a tenant-owned
domain is accepted.
- applyplan.ReconcileDomainTenants repairs the plan between convert and
apply. Where a tenant-less domain is used only by accounts of one
tenant, the domain adopts that tenant - the sole assignment that both
applies and keeps every account. Where accounts genuinely disagree it
changes nothing and reports why, because forcing such a plan through
would mean dropping mailboxes.
- stalwartapi.FetchTenantLayout maps tenant membership over the 0.15 REST
API and predicts the outcome with the same rule the server enforces, so
preflight either warns about the domains that will adopt a tenant or
fails - while the service is still running.
- The plan is parsed generically rather than through the typed Operation.
A real export.json mixes shapes: `create` maps a client-id to an object,
`update` carries a flat one. The typed form failed on the first `update`
line, found by running against actual converter output. Numbers decode
as json.Number so a 10 GiB quota is not rewritten as 1.073741824e+10.
Corrects the record: the previous commit claimed the converter emits every
Account with `tenantId: null` and made preflight refuse every multi-tenant
install on that basis. The field is memberTenantId, the converter does
populate it, and the export had been inspected for a key no version of the
script ever writes. The refusal is now narrowed to what v0.16 genuinely
cannot represent.
The same fix has been prepared for migrate_v016.py upstream. The tool
downloads that script rather than vendoring it, so the repair stays here
until a released version carries it, and is a no-op on a consistent plan.
This commit is contained in:
+54
-16
@@ -829,28 +829,66 @@ happens to need them. `preflight.DeploymentKind` is a type alias for
|
||||
- `tenant-admin` has no v0.16 equivalent and is reported as unrestorable
|
||||
rather than silently dropped.
|
||||
|
||||
- **Multi-tenant installs cannot be migrated by this path, and preflight now
|
||||
refuses them.** A second live attempt on 2026-08-24 got further - preflight
|
||||
clean, binary staged, settings dumped and converted - and then failed
|
||||
during recovery-mode migration, again with the service already stopped:
|
||||
- **A domain and the accounts on it must agree about their tenant in v0.16,
|
||||
and Stalwart's converter does not make them agree.** A second live attempt
|
||||
on 2026-08-24 got further - preflight clean, binary staged, settings dumped
|
||||
and converted - and then failed during recovery-mode migration, again with
|
||||
the service already stopped:
|
||||
|
||||
created Tenant (1)
|
||||
created Domain (9)
|
||||
create Account restore-13: invalidForeignKey | Object id: Domain#d
|
||||
|
||||
`migrate_v016.py` carries the Tenant and the Domains but leaves every
|
||||
Account's `tenantId` null, so an account references a tenant-owned domain
|
||||
while belonging to no tenant and the foreign key is rejected. This is
|
||||
Stalwart's converter, not this tool, and there is no way around it from
|
||||
here - a multi-tenant install has to be migrated by hand until the
|
||||
converter handles it.
|
||||
In v0.15 a domain's tenant and a principal's tenant were independent facts.
|
||||
v0.16 requires a tenant-scoped Account to sit on a Domain owned by that
|
||||
same tenant - for its primary domain and for every alias - and answers
|
||||
`invalidForeignKey` on the Domain reference otherwise. `migrate_v016.py`
|
||||
carries the two facts over independently: `_build_domains` sets a domain's
|
||||
`memberTenantId` only when the domain appears as a declared `domain`
|
||||
principal carrying a `tenant`, while `_build_user` sets the account's from
|
||||
the account's own record. A domain that exists only inside an email address
|
||||
is inferred, gets no tenant, and any tenant-scoped account using it is then
|
||||
rejected.
|
||||
|
||||
The tool's failure was in *when* this was discovered. Tenant principals
|
||||
are one API call away and were readable while the server was running, so
|
||||
preflight now queries them (`stalwartapi.Client.TenantNames`) and fails
|
||||
before anything is touched. That is the same lesson as the stalwart-cli
|
||||
check immediately below: both were knowable in advance, and both were
|
||||
found after a production mail server had been stopped. Any future
|
||||
This was established by reproduction, not inference. A synthetic v0.15
|
||||
principal dump run through the unpatched converter and applied to a real
|
||||
0.16.14 in recovery mode reproduces `invalidForeignKey | Object id:
|
||||
Domain#d` character for character - the `#d` is the server's own object id
|
||||
for the offending domain, not a plan client-id. The same harness shows
|
||||
which directions are actually constrained:
|
||||
|
||||
| account | domain | result |
|
||||
| --- | --- | --- |
|
||||
| tenant-scoped | no tenant | **rejected** |
|
||||
| tenant-scoped | a different tenant | **rejected** |
|
||||
| tenant-scoped | its own tenant | accepted |
|
||||
| global | tenant-owned | accepted |
|
||||
|
||||
Only the first two fail, and only the first is repairable: where a
|
||||
tenant-less domain is used exclusively by accounts of one tenant, giving
|
||||
the domain that tenant is the sole assignment that both applies and keeps
|
||||
every account. `applyplan.ReconcileDomainTenants` does that to the plan
|
||||
between `convert` and `apply`, reports each adoption, and refuses - without
|
||||
modifying anything - when the accounts genuinely disagree, since forcing
|
||||
such a plan through would mean dropping mailboxes. The same fix has been
|
||||
prepared for `migrate_v016.py` upstream; the tool downloads that script
|
||||
rather than vendoring it, so the repair lives here until a released version
|
||||
carries it, and is a no-op on a plan that is already consistent.
|
||||
|
||||
An earlier version of this section claimed the converter emitted every
|
||||
Account with `tenantId: null`. That was wrong: the field is
|
||||
`memberTenantId`, the converter does populate it, and the export had been
|
||||
inspected for a key no version of the script ever writes. Preflight briefly
|
||||
refused every multi-tenant install on the strength of that misreading. The
|
||||
refusal is now narrowed to the arrangements v0.16 truly cannot represent.
|
||||
|
||||
The tool's failure was in *when* this was discovered. Tenant membership
|
||||
is readable while the server is still running, so preflight now maps it
|
||||
(`stalwartapi.Client.FetchTenantLayout`), predicts the outcome with the same
|
||||
rule the server enforces, and either warns about the domains that will adopt
|
||||
a tenant or fails - before anything is touched. That is the same lesson as
|
||||
the stalwart-cli check immediately below: both were knowable in advance, and
|
||||
both were found after a production mail server had been stopped. Any future
|
||||
dependency of the *conversion* belongs in preflight, not in the phase that
|
||||
needs it.
|
||||
|
||||
|
||||
@@ -191,7 +191,18 @@ func runRehearse(args []string) (err error) {
|
||||
}); err != nil {
|
||||
return checkpoint.StepOutcome{}, err
|
||||
}
|
||||
// The same domain/tenant repair the real run performs, so a
|
||||
// rehearsal exercises the plan that would actually be applied -
|
||||
// and so an unrepresentable multi-tenant layout is reported here,
|
||||
// with the service still running, rather than during a cutover.
|
||||
tenantFix, err := applyplan.ReconcileDomainTenantsFile(convertedExport)
|
||||
if err != nil {
|
||||
return checkpoint.StepOutcome{}, err
|
||||
}
|
||||
detail := "converted this instance's settings into a v0.16 apply plan"
|
||||
if len(tenantFix.Adoptions) > 0 {
|
||||
detail += " - " + tenantFix.String()
|
||||
}
|
||||
if report, readErr := backup.ReadUnmigratedReport(unmigratedPath); readErr == nil && report != nil {
|
||||
detail += fmt.Sprintf("; %d setting(s) will NOT carry over", report.TotalKeys)
|
||||
}
|
||||
|
||||
@@ -261,10 +261,26 @@ func runRun(args []string) (err error) {
|
||||
}); err != nil {
|
||||
return checkpoint.StepOutcome{}, err
|
||||
}
|
||||
return checkpoint.StepOutcome{Detail: "converted settings into a v0.16 apply plan"}, nil
|
||||
// Repair migrate_v016.py's domain/tenant mismatch before the
|
||||
// plan is ever applied. Left alone it surfaces as
|
||||
// "invalidForeignKey | Object id: Domain#..." partway through
|
||||
// apply - with the old service already stopped and the store
|
||||
// already at schema v6, i.e. at the one point in the run where
|
||||
// there is no way forward and no way back. See
|
||||
// applyplan/tenants.go.
|
||||
tenantFix, err := applyplan.ReconcileDomainTenantsFile(convertedExport)
|
||||
if err != nil {
|
||||
return checkpoint.StepOutcome{}, err
|
||||
}
|
||||
detail := "converted settings into a v0.16 apply plan"
|
||||
if len(tenantFix.Adoptions) > 0 {
|
||||
detail += " - " + tenantFix.String()
|
||||
}
|
||||
return checkpoint.StepOutcome{Detail: detail}, nil
|
||||
}); err != nil {
|
||||
return fmt.Errorf("convert settings: %w", err)
|
||||
}
|
||||
fmt.Println(rs.Outcome(checkpoint.PhaseStage, "convert-settings").Detail)
|
||||
unmigrated, readErr := backup.ReadUnmigratedReport(unmigratedPath)
|
||||
if readErr == nil && unmigrated != nil && unmigrated.TotalKeys > 0 {
|
||||
keptWorklist := filepath.Join(runStateDir, "unmigrated.txt")
|
||||
|
||||
@@ -0,0 +1,386 @@
|
||||
// SPDX-FileCopyrightText: 2026 LINUXexpert-org
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
package applyplan
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// This file repairs one specific defect in Stalwart's own migrate_v016.py,
|
||||
// which this tool downloads rather than vendors and therefore cannot fix at
|
||||
// the source.
|
||||
//
|
||||
// In v0.15 a domain's tenant and a principal's tenant were independent
|
||||
// facts. A tenant-scoped user could sit on a domain owned by no tenant at
|
||||
// all - which is the normal outcome whenever the domain was never declared
|
||||
// as its own `domain` principal, because migrate_v016.py then infers the
|
||||
// domain from an email address, and inferred domains carry no tenant.
|
||||
//
|
||||
// v0.16 rejects that arrangement. A tenant-scoped Account may only
|
||||
// reference a Domain owned by the same tenant, as a primary domain or as an
|
||||
// alias, and the server answers:
|
||||
//
|
||||
// invalidForeignKey | Object id: Domain#d
|
||||
//
|
||||
// on the Domain reference otherwise. This was confirmed against a real
|
||||
// 0.16.14 in recovery mode, not inferred: the reverse direction (a global,
|
||||
// tenant-less account on a tenant-owned domain) applies without complaint,
|
||||
// and only the tenant-scoped-account-to-tenant-less-domain direction fails.
|
||||
//
|
||||
// That error cost a production migration a full restore, because it
|
||||
// surfaced during apply - after the old service was already stopped, and
|
||||
// after the store had been irreversibly upgraded to schema v6. Hence both
|
||||
// halves of the fix: repair the plan here, and refuse the genuinely
|
||||
// unrepresentable cases in preflight, before anything is stopped.
|
||||
//
|
||||
// The same fix has been prepared for migrate_v016.py upstream; this stays
|
||||
// until a released migrate_v016.py carries it, and is written to be a no-op
|
||||
// against a plan that is already consistent.
|
||||
|
||||
// TenantAdoption records one domain that took on a tenant it did not
|
||||
// previously declare, so the operator is told rather than silently given a
|
||||
// different ownership model than they had.
|
||||
type TenantAdoption struct {
|
||||
Domain string // domain name, e.g. "example.com"
|
||||
Tenant string // tenant name, or the raw reference if the name is unknown
|
||||
Because string // the principal whose membership forced it
|
||||
}
|
||||
|
||||
// TenantConflict is a domain whose users disagree about which tenant owns
|
||||
// it. v0.16 cannot represent this, so it is reported rather than repaired.
|
||||
type TenantConflict struct {
|
||||
Domain string
|
||||
Detail string
|
||||
Tenants []string
|
||||
}
|
||||
|
||||
// TenantReconcileResult is what a reconciliation changed and what it could
|
||||
// not change.
|
||||
type TenantReconcileResult struct {
|
||||
Adoptions []TenantAdoption
|
||||
Conflicts []TenantConflict
|
||||
}
|
||||
|
||||
// OK reports whether the plan is now internally consistent.
|
||||
func (r TenantReconcileResult) OK() bool { return len(r.Conflicts) == 0 }
|
||||
|
||||
// String renders a short operator-facing summary.
|
||||
func (r TenantReconcileResult) String() string {
|
||||
if len(r.Adoptions) == 0 && len(r.Conflicts) == 0 {
|
||||
return "no domain/tenant mismatches"
|
||||
}
|
||||
var b strings.Builder
|
||||
for _, a := range r.Adoptions {
|
||||
fmt.Fprintf(&b, "domain %s adopted tenant %s (required by %s)\n", a.Domain, a.Tenant, a.Because)
|
||||
}
|
||||
for _, c := range r.Conflicts {
|
||||
fmt.Fprintf(&b, "domain %s: %s\n", c.Domain, c.Detail)
|
||||
}
|
||||
return strings.TrimRight(b.String(), "\n")
|
||||
}
|
||||
|
||||
// PlanOp is one line of an apply plan, kept in the generic form it was
|
||||
// read in.
|
||||
//
|
||||
// It is deliberately not the typed Operation above. A real export.json
|
||||
// mixes shapes: `create` operations map a client-id to an object, while
|
||||
// `update` operations - SystemSettings, BlobStore, SearchStore - carry a
|
||||
// flat object with no client-id layer at all. Parsing a whole plan into the
|
||||
// typed form fails on the first `update` line, and a reconciliation that
|
||||
// cannot read the plan it is meant to repair is worse than none.
|
||||
type PlanOp map[string]any
|
||||
|
||||
// ReadPlan parses an apply plan - migrate_v016.py's export.json, or one
|
||||
// this tool generated.
|
||||
//
|
||||
// Numbers are decoded as json.Number so they survive the round-trip
|
||||
// unchanged. Decoding into float64 would rewrite a quota of 10737418240 as
|
||||
// 1.073741824e+10, which is a different document than the one the
|
||||
// conversion produced.
|
||||
func ReadPlan(path string) ([]PlanOp, error) {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("applyplan: open %s: %w", path, err)
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
var ops []PlanOp
|
||||
sc := bufio.NewScanner(f)
|
||||
// Plans carry certificates and DKIM keys inline, so lines run well past
|
||||
// bufio.Scanner's default 64KiB limit.
|
||||
sc.Buffer(make([]byte, 0, 64<<10), 8<<20)
|
||||
for line := 1; sc.Scan(); line++ {
|
||||
raw := strings.TrimSpace(sc.Text())
|
||||
if raw == "" {
|
||||
continue
|
||||
}
|
||||
dec := json.NewDecoder(strings.NewReader(raw))
|
||||
dec.UseNumber()
|
||||
var op PlanOp
|
||||
if err := dec.Decode(&op); err != nil {
|
||||
return nil, fmt.Errorf("applyplan: %s line %d: %w", path, line, err)
|
||||
}
|
||||
ops = append(ops, op)
|
||||
}
|
||||
if err := sc.Err(); err != nil {
|
||||
return nil, fmt.Errorf("applyplan: read %s: %w", path, err)
|
||||
}
|
||||
return ops, nil
|
||||
}
|
||||
|
||||
// WritePlan writes plan operations atomically, so a failure part-way
|
||||
// through can never leave a truncated plan that would apply half a
|
||||
// migration.
|
||||
func WritePlan(path string, ops []PlanOp) error {
|
||||
dir := filepath.Dir(path)
|
||||
tmp, err := os.CreateTemp(dir, filepath.Base(path)+".tmp*")
|
||||
if err != nil {
|
||||
return fmt.Errorf("applyplan: create temp file in %s: %w", dir, err)
|
||||
}
|
||||
tmpName := tmp.Name()
|
||||
defer os.Remove(tmpName)
|
||||
|
||||
enc := json.NewEncoder(tmp)
|
||||
for _, op := range ops {
|
||||
if err := enc.Encode(op); err != nil {
|
||||
tmp.Close()
|
||||
return fmt.Errorf("applyplan: write %s: %w", tmpName, err)
|
||||
}
|
||||
}
|
||||
if err := tmp.Sync(); err != nil {
|
||||
tmp.Close()
|
||||
return fmt.Errorf("applyplan: sync %s: %w", tmpName, err)
|
||||
}
|
||||
if err := tmp.Close(); err != nil {
|
||||
return fmt.Errorf("applyplan: close %s: %w", tmpName, err)
|
||||
}
|
||||
if err := os.Chmod(tmpName, 0o640); err != nil {
|
||||
return fmt.Errorf("applyplan: chmod %s: %w", tmpName, err)
|
||||
}
|
||||
if err := os.Rename(tmpName, path); err != nil {
|
||||
return fmt.Errorf("applyplan: replace %s: %w", path, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// objectName is the operation's target type, e.g. "Domain".
|
||||
func (op PlanOp) objectName() string {
|
||||
s, _ := op["object"].(string)
|
||||
return s
|
||||
}
|
||||
|
||||
// createBodies returns the client-id-keyed objects a `create` operation
|
||||
// carries. Operations whose value is a flat object - every `update` - yield
|
||||
// nothing, which is what keeps them untouched.
|
||||
func (op PlanOp) createBodies() map[string]map[string]any {
|
||||
value, ok := op["value"].(map[string]any)
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
out := map[string]map[string]any{}
|
||||
for cid, body := range value {
|
||||
if b, ok := body.(map[string]any); ok {
|
||||
out[cid] = b
|
||||
}
|
||||
}
|
||||
if len(out) == 0 {
|
||||
return nil
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// refCID strips the leading "#" from a plan back-reference. Values that are
|
||||
// not back-references (an already-resolved server id, say) are returned
|
||||
// unchanged, which is what makes comparing two references safe.
|
||||
func refCID(v any) string {
|
||||
s, ok := v.(string)
|
||||
if !ok || s == "" {
|
||||
return ""
|
||||
}
|
||||
return strings.TrimPrefix(s, "#")
|
||||
}
|
||||
|
||||
// domainRefs returns every domain reference an account or mailing list
|
||||
// makes: its primary domain, plus one per alias. Aliases matter as much as
|
||||
// the primary - a tenant-scoped account with an alias on a tenant-less
|
||||
// domain is rejected exactly like one whose primary domain mismatches.
|
||||
func domainRefs(body map[string]any) []string {
|
||||
var refs []string
|
||||
if c := refCID(body["domainId"]); c != "" {
|
||||
refs = append(refs, c)
|
||||
}
|
||||
aliases, ok := body["aliases"].(map[string]any)
|
||||
if !ok {
|
||||
return refs
|
||||
}
|
||||
for _, a := range aliases {
|
||||
alias, ok := a.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if c := refCID(alias["domainId"]); c != "" {
|
||||
refs = append(refs, c)
|
||||
}
|
||||
}
|
||||
return refs
|
||||
}
|
||||
|
||||
func stringField(body map[string]any, key string) string {
|
||||
s, _ := body[key].(string)
|
||||
return s
|
||||
}
|
||||
|
||||
// ReconcileDomainTenants makes every domain's tenant agree with the
|
||||
// accounts and mailing lists that use it, editing ops in place.
|
||||
//
|
||||
// Where a domain has no tenant but every tenant-scoped principal using it
|
||||
// agrees on one, the domain adopts that tenant: it is the only assignment
|
||||
// that lets the plan apply while keeping every account, and it is safe in
|
||||
// the other direction because a global account on a tenant-owned domain is
|
||||
// accepted.
|
||||
//
|
||||
// Where the principals disagree, nothing is changed and the disagreement is
|
||||
// reported. Forcing such a plan through would mean dropping accounts, and
|
||||
// dropping accounts silently is how mailboxes get lost.
|
||||
func ReconcileDomainTenants(ops []PlanOp) TenantReconcileResult {
|
||||
var result TenantReconcileResult
|
||||
|
||||
domains := map[string]map[string]any{} // cid -> domain body
|
||||
tenantNames := map[string]string{} // cid -> tenant name
|
||||
for _, op := range ops {
|
||||
bodies := op.createBodies()
|
||||
switch op.objectName() {
|
||||
case "Domain":
|
||||
for cid, body := range bodies {
|
||||
domains[cid] = body
|
||||
}
|
||||
case "Tenant":
|
||||
for cid, body := range bodies {
|
||||
tenantNames[cid] = stringField(body, "name")
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(domains) == 0 || len(tenantNames) == 0 {
|
||||
return result // single-tenant install: nothing can mismatch
|
||||
}
|
||||
|
||||
tenantName := func(ref string) string {
|
||||
if n := tenantNames[refCID(ref)]; n != "" {
|
||||
return n
|
||||
}
|
||||
return refCID(ref)
|
||||
}
|
||||
|
||||
// domain cid -> tenant ref -> an example principal requiring it.
|
||||
required := map[string]map[string]string{}
|
||||
for _, op := range ops {
|
||||
object := op.objectName()
|
||||
if object != "Account" && object != "MailingList" {
|
||||
continue
|
||||
}
|
||||
for _, body := range op.createBodies() {
|
||||
tRef, _ := body["memberTenantId"].(string)
|
||||
if tRef == "" {
|
||||
continue // a global principal constrains nothing
|
||||
}
|
||||
label := fmt.Sprintf("%s %q", object, stringField(body, "name"))
|
||||
for _, dCID := range domainRefs(body) {
|
||||
if required[dCID] == nil {
|
||||
required[dCID] = map[string]string{}
|
||||
}
|
||||
if _, seen := required[dCID][tRef]; !seen {
|
||||
required[dCID][tRef] = label
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for _, dCID := range sortedKeys(required) {
|
||||
wanted := required[dCID]
|
||||
dom, ok := domains[dCID]
|
||||
if !ok {
|
||||
continue // reference to something this plan does not create
|
||||
}
|
||||
dName := stringField(dom, "name")
|
||||
if dName == "" {
|
||||
dName = dCID
|
||||
}
|
||||
|
||||
if len(wanted) > 1 {
|
||||
var names, parts []string
|
||||
for _, tRef := range sortedKeys(wanted) {
|
||||
names = append(names, tenantName(tRef))
|
||||
parts = append(parts, fmt.Sprintf("%s (e.g. %s)", tenantName(tRef), wanted[tRef]))
|
||||
}
|
||||
result.Conflicts = append(result.Conflicts, TenantConflict{
|
||||
Domain: dName,
|
||||
Tenants: names,
|
||||
Detail: fmt.Sprintf("used by principals from more than one tenant: %s. "+
|
||||
"v0.16 requires every tenant-scoped account to sit on a domain owned by its own "+
|
||||
"tenant, and a domain can belong to at most one tenant", strings.Join(parts, ", ")),
|
||||
})
|
||||
continue
|
||||
}
|
||||
|
||||
tRef := sortedKeys(wanted)[0]
|
||||
because := wanted[tRef]
|
||||
current, _ := dom["memberTenantId"].(string)
|
||||
if refCID(current) == refCID(tRef) {
|
||||
continue // already consistent
|
||||
}
|
||||
if current != "" {
|
||||
result.Conflicts = append(result.Conflicts, TenantConflict{
|
||||
Domain: dName,
|
||||
Tenants: []string{tenantName(current), tenantName(tRef)},
|
||||
Detail: fmt.Sprintf("belongs to tenant %s, but %s belongs to tenant %s and uses it. "+
|
||||
"v0.16 rejects an account whose tenant differs from its domain's",
|
||||
tenantName(current), because, tenantName(tRef)),
|
||||
})
|
||||
continue
|
||||
}
|
||||
|
||||
dom["memberTenantId"] = tRef
|
||||
result.Adoptions = append(result.Adoptions, TenantAdoption{
|
||||
Domain: dName, Tenant: tenantName(tRef), Because: because,
|
||||
})
|
||||
}
|
||||
|
||||
return result
|
||||
}
|
||||
|
||||
// ReconcileDomainTenantsFile reads a plan, reconciles it, and rewrites it
|
||||
// only if something actually changed and nothing conflicted.
|
||||
func ReconcileDomainTenantsFile(path string) (TenantReconcileResult, error) {
|
||||
ops, err := ReadPlan(path)
|
||||
if err != nil {
|
||||
return TenantReconcileResult{}, err
|
||||
}
|
||||
result := ReconcileDomainTenants(ops)
|
||||
if !result.OK() {
|
||||
return result, fmt.Errorf("applyplan: %s cannot be made consistent with v0.16's "+
|
||||
"tenant rules:\n%s", path, result.String())
|
||||
}
|
||||
if len(result.Adoptions) == 0 {
|
||||
return result, nil
|
||||
}
|
||||
if err := WritePlan(path, ops); err != nil {
|
||||
return result, err
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func sortedKeys[V any](m map[string]V) []string {
|
||||
out := make([]string, 0, len(m))
|
||||
for k := range m {
|
||||
out = append(out, k)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,329 @@
|
||||
// SPDX-FileCopyrightText: 2026 LINUXexpert-org
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
package applyplan
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The four cases below are the ones checked against a real Stalwart 0.16.14
|
||||
// in recovery mode, via `stalwart-cli apply`, before this code was written:
|
||||
//
|
||||
// tenant account -> tenant-less domain : invalidForeignKey (repaired here)
|
||||
// tenant account -> other tenant's dom : invalidForeignKey (unrepresentable)
|
||||
// global account -> tenant-owned domain: accepted (left alone)
|
||||
// tenant account -> own tenant's domain: accepted (left alone)
|
||||
func plan(ops ...PlanOp) []PlanOp { return ops }
|
||||
|
||||
func tenantOp(cid, name string) PlanOp {
|
||||
return PlanOp{"@type": "create", "object": "Tenant",
|
||||
"value": map[string]any{cid: map[string]any{"name": name}}}
|
||||
}
|
||||
|
||||
func domainOp(cid, name string, tenantRef string) PlanOp {
|
||||
body := map[string]any{"name": name}
|
||||
if tenantRef != "" {
|
||||
body["memberTenantId"] = tenantRef
|
||||
}
|
||||
return PlanOp{"@type": "create", "object": "Domain",
|
||||
"value": map[string]any{cid: body}}
|
||||
}
|
||||
|
||||
func accountOp(cid, name, domainRef, tenantRef string, aliasDomainRefs ...string) PlanOp {
|
||||
body := map[string]any{"@type": "User", "name": name, "domainId": domainRef}
|
||||
if tenantRef != "" {
|
||||
body["memberTenantId"] = tenantRef
|
||||
}
|
||||
if len(aliasDomainRefs) > 0 {
|
||||
aliases := map[string]any{}
|
||||
for i, r := range aliasDomainRefs {
|
||||
aliases[string(rune('0'+i))] = map[string]any{"name": name, "domainId": r}
|
||||
}
|
||||
body["aliases"] = aliases
|
||||
}
|
||||
return PlanOp{"@type": "create", "object": "Account",
|
||||
"value": map[string]any{cid: body}}
|
||||
}
|
||||
|
||||
func domainTenant(t *testing.T, ops []PlanOp, cid string) string {
|
||||
t.Helper()
|
||||
for _, op := range ops {
|
||||
if op.objectName() != "Domain" {
|
||||
continue
|
||||
}
|
||||
if body, ok := op.createBodies()[cid]; ok {
|
||||
s, _ := body["memberTenantId"].(string)
|
||||
return s
|
||||
}
|
||||
}
|
||||
t.Fatalf("no domain %q in plan", cid)
|
||||
return ""
|
||||
}
|
||||
|
||||
func TestReconcileAdoptsTenantForInferredDomain(t *testing.T) {
|
||||
ops := plan(
|
||||
tenantOp("t0", "acme"),
|
||||
domainOp("d0", "acme-corp.test", "#t0"),
|
||||
domainOp("d1", "inferred.test", ""), // never declared in v0.15
|
||||
accountOp("a0", "bob", "#d0", "#t0", "#d1"),
|
||||
)
|
||||
res := ReconcileDomainTenants(ops)
|
||||
if !res.OK() {
|
||||
t.Fatalf("unexpected conflicts: %s", res.String())
|
||||
}
|
||||
if got := domainTenant(t, ops, "d1"); got != "#t0" {
|
||||
t.Errorf("inferred domain tenant = %q, want %q", got, "#t0")
|
||||
}
|
||||
if len(res.Adoptions) != 1 || res.Adoptions[0].Domain != "inferred.test" {
|
||||
t.Errorf("adoptions = %+v, want one for inferred.test", res.Adoptions)
|
||||
}
|
||||
if res.Adoptions[0].Tenant != "acme" {
|
||||
t.Errorf("adoption tenant = %q, want the tenant name %q", res.Adoptions[0].Tenant, "acme")
|
||||
}
|
||||
}
|
||||
|
||||
func TestReconcileLeavesGlobalAccountOnTenantDomainAlone(t *testing.T) {
|
||||
// Confirmed accepted by 0.16.14: a global account may live on a
|
||||
// tenant-owned domain. Nothing should be invented for it.
|
||||
ops := plan(
|
||||
tenantOp("t0", "acme"),
|
||||
domainOp("d0", "acme-corp.test", "#t0"),
|
||||
accountOp("a0", "admin", "#d0", ""),
|
||||
)
|
||||
res := ReconcileDomainTenants(ops)
|
||||
if !res.OK() || len(res.Adoptions) != 0 {
|
||||
t.Fatalf("expected no changes, got %s", res.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestReconcileLeavesTenantlessDomainWithOnlyGlobalUsers(t *testing.T) {
|
||||
ops := plan(
|
||||
tenantOp("t0", "acme"),
|
||||
domainOp("d0", "global.test", ""),
|
||||
accountOp("a0", "carol", "#d0", ""),
|
||||
)
|
||||
res := ReconcileDomainTenants(ops)
|
||||
if got := domainTenant(t, ops, "d0"); got != "" {
|
||||
t.Errorf("global domain gained tenant %q; it has no tenant-scoped users", got)
|
||||
}
|
||||
if len(res.Adoptions) != 0 {
|
||||
t.Errorf("adoptions = %+v, want none", res.Adoptions)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReconcileReportsTwoTenantsSharingADomain(t *testing.T) {
|
||||
ops := plan(
|
||||
tenantOp("t0", "alpha"),
|
||||
tenantOp("t1", "beta"),
|
||||
domainOp("d0", "shared.test", ""),
|
||||
accountOp("a0", "u1", "#d0", "#t0"),
|
||||
accountOp("a1", "u2", "#d0", "#t1"),
|
||||
)
|
||||
res := ReconcileDomainTenants(ops)
|
||||
if res.OK() {
|
||||
t.Fatal("expected a conflict for a domain shared by two tenants")
|
||||
}
|
||||
if got := domainTenant(t, ops, "d0"); got != "" {
|
||||
t.Errorf("conflicting domain was modified to %q; it must be left untouched", got)
|
||||
}
|
||||
c := res.Conflicts[0]
|
||||
if c.Domain != "shared.test" || len(c.Tenants) != 2 {
|
||||
t.Errorf("conflict = %+v, want shared.test across two tenants", c)
|
||||
}
|
||||
for _, want := range []string{"alpha", "beta"} {
|
||||
if !strings.Contains(c.Detail, want) {
|
||||
t.Errorf("conflict detail %q does not name tenant %q", c.Detail, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestReconcileReportsAccountOnAnotherTenantsDomain(t *testing.T) {
|
||||
ops := plan(
|
||||
tenantOp("t0", "alpha"),
|
||||
tenantOp("t1", "beta"),
|
||||
domainOp("d0", "owned.test", "#t0"),
|
||||
accountOp("a0", "u2", "#d0", "#t1"),
|
||||
)
|
||||
res := ReconcileDomainTenants(ops)
|
||||
if res.OK() {
|
||||
t.Fatal("expected a conflict when an account's tenant differs from its domain's")
|
||||
}
|
||||
if got := domainTenant(t, ops, "d0"); got != "#t0" {
|
||||
t.Errorf("domain ownership changed to %q; declared ownership must not be overwritten", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReconcileIsNoOpWithoutTenants(t *testing.T) {
|
||||
ops := plan(
|
||||
domainOp("d0", "example.test", ""),
|
||||
accountOp("a0", "alice", "#d0", ""),
|
||||
)
|
||||
res := ReconcileDomainTenants(ops)
|
||||
if !res.OK() || len(res.Adoptions) != 0 {
|
||||
t.Fatalf("single-tenant plan should be untouched, got %s", res.String())
|
||||
}
|
||||
if res.String() != "no domain/tenant mismatches" {
|
||||
t.Errorf("summary = %q", res.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestReconcileFileRoundTripsAndRewritesOnlyWhenChanged(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "export.json")
|
||||
|
||||
ops := plan(
|
||||
tenantOp("t0", "acme"),
|
||||
domainOp("d0", "acme-corp.test", "#t0"),
|
||||
domainOp("d1", "inferred.test", ""),
|
||||
accountOp("a0", "bob", "#d0", "#t0", "#d1"),
|
||||
)
|
||||
if err := WritePlan(path, ops); err != nil {
|
||||
t.Fatalf("write: %v", err)
|
||||
}
|
||||
|
||||
res, err := ReconcileDomainTenantsFile(path)
|
||||
if err != nil {
|
||||
t.Fatalf("reconcile file: %v", err)
|
||||
}
|
||||
if len(res.Adoptions) != 1 {
|
||||
t.Fatalf("adoptions = %+v", res.Adoptions)
|
||||
}
|
||||
|
||||
reread, err := ReadPlan(path)
|
||||
if err != nil {
|
||||
t.Fatalf("reread: %v", err)
|
||||
}
|
||||
if got := domainTenant(t, reread, "d1"); got != "#t0" {
|
||||
t.Errorf("rewritten plan has tenant %q, want %q", got, "#t0")
|
||||
}
|
||||
|
||||
// Second pass must be a no-op: the plan is already consistent.
|
||||
before, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
res2, err := ReconcileDomainTenantsFile(path)
|
||||
if err != nil {
|
||||
t.Fatalf("second reconcile: %v", err)
|
||||
}
|
||||
if len(res2.Adoptions) != 0 {
|
||||
t.Errorf("second pass adopted %+v, want none", res2.Adoptions)
|
||||
}
|
||||
after, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if string(before) != string(after) {
|
||||
t.Error("second pass rewrote an already-consistent plan")
|
||||
}
|
||||
}
|
||||
|
||||
func TestReconcileFileRefusesConflictingPlan(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "export.json")
|
||||
ops := plan(
|
||||
tenantOp("t0", "alpha"),
|
||||
tenantOp("t1", "beta"),
|
||||
domainOp("d0", "shared.test", ""),
|
||||
accountOp("a0", "u1", "#d0", "#t0"),
|
||||
accountOp("a1", "u2", "#d0", "#t1"),
|
||||
)
|
||||
if err := WritePlan(path, ops); err != nil {
|
||||
t.Fatalf("write: %v", err)
|
||||
}
|
||||
before, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := ReconcileDomainTenantsFile(path); err == nil {
|
||||
t.Fatal("expected an error for an unrepresentable plan")
|
||||
}
|
||||
after, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if string(before) != string(after) {
|
||||
t.Error("a plan that could not be repaired was rewritten anyway")
|
||||
}
|
||||
}
|
||||
|
||||
func TestReadPlanHandlesLongLines(t *testing.T) {
|
||||
// Plans embed certificates and DKIM private keys inline, which blow
|
||||
// past bufio.Scanner's default 64KiB line limit.
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "big.json")
|
||||
huge := strings.Repeat("x", 200<<10)
|
||||
ops := plan(PlanOp{"@type": "update", "object": "Certificate",
|
||||
"value": map[string]any{"c0": map[string]any{"cert": huge}}})
|
||||
if err := WritePlan(path, ops); err != nil {
|
||||
t.Fatalf("write: %v", err)
|
||||
}
|
||||
got, err := ReadPlan(path)
|
||||
if err != nil {
|
||||
t.Fatalf("read: %v", err)
|
||||
}
|
||||
if len(got) != 1 || got[0].createBodies()["c0"]["cert"] != huge {
|
||||
t.Error("long line did not round-trip")
|
||||
}
|
||||
}
|
||||
|
||||
// A real export.json mixes `create` operations (client-id keyed) with
|
||||
// `update` operations carrying a flat object. An earlier version of this
|
||||
// code modelled every operation the first way and failed to parse a real
|
||||
// plan at the first `update` line - found by running it against actual
|
||||
// migrate_v016.py output rather than hand-built fixtures.
|
||||
func TestReconcileHandlesRealExportShapes(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "export.json")
|
||||
raw := strings.Join([]string{
|
||||
`{"@type":"create","object":"Tenant","value":{"create-0":{"name":"acme","quotas":{}}}}`,
|
||||
`{"@type":"create","object":"Domain","value":{"create-1":{"name":"acme-corp.test","memberTenantId":"#create-0"},"create-2":{"name":"inferred.test"}}}`,
|
||||
`{"@type":"create","object":"Account","value":{"restore-5":{"@type":"User","name":"bob","domainId":"#create-1","memberTenantId":"#create-0","aliases":{"0":{"name":"bob","domainId":"#create-2"}},"quotas":{"maxDiskQuota":10737418240}}}}`,
|
||||
`{"@type":"update","object":"SystemSettings","value":{"defaultDomainId":"#create-1","defaultHostname":"mail.acme-corp.test"}}`,
|
||||
`{"@type":"update","object":"BlobStore","value":{"@type":"Default"}}`,
|
||||
"",
|
||||
}, "\n")
|
||||
if err := os.WriteFile(path, []byte(raw), 0o640); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
res, err := ReconcileDomainTenantsFile(path)
|
||||
if err != nil {
|
||||
t.Fatalf("reconcile: %v", err)
|
||||
}
|
||||
if len(res.Adoptions) != 1 || res.Adoptions[0].Domain != "inferred.test" {
|
||||
t.Fatalf("adoptions = %+v", res.Adoptions)
|
||||
}
|
||||
|
||||
out, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// A large quota must survive as an integer: decoding through float64
|
||||
// would rewrite it as 1.073741824e+10 and hand the server a different
|
||||
// document than the conversion produced.
|
||||
if !strings.Contains(string(out), "10737418240") {
|
||||
t.Errorf("quota was reformatted; plan now reads:\n%s", out)
|
||||
}
|
||||
// The update operations must come back untouched.
|
||||
for _, want := range []string{`"defaultHostname":"mail.acme-corp.test"`, `"object":"BlobStore"`} {
|
||||
if !strings.Contains(strings.ReplaceAll(string(out), ", ", ","), want) {
|
||||
t.Errorf("rewritten plan lost %s:\n%s", want, out)
|
||||
}
|
||||
}
|
||||
|
||||
reread, err := ReadPlan(path)
|
||||
if err != nil {
|
||||
t.Fatalf("reread: %v", err)
|
||||
}
|
||||
if len(reread) != 5 {
|
||||
t.Errorf("operation count = %d, want 5", len(reread))
|
||||
}
|
||||
if got := domainTenant(t, reread, "create-2"); got != "#create-0" {
|
||||
t.Errorf("inferred domain tenant = %q, want %q", got, "#create-0")
|
||||
}
|
||||
}
|
||||
@@ -328,24 +328,46 @@ func (c *Checker) Run(ctx context.Context, store *checkpoint.Store, rs *checkpoi
|
||||
BaseURL: c.opts.AdminURL, Username: c.opts.AdminUser,
|
||||
Password: c.opts.AdminPassword, HTTPClient: c.opts.HTTPClient,
|
||||
}
|
||||
tenants, err := client.TenantNames(ctx)
|
||||
layout, err := client.FetchTenantLayout(ctx)
|
||||
if err != nil {
|
||||
return CheckResult{
|
||||
Status: StatusWarn,
|
||||
Detail: fmt.Sprintf("couldn't determine whether this instance is multi-tenant: %v - if it is, the migration will fail after the service is stopped", err),
|
||||
Detail: fmt.Sprintf("couldn't map this instance's tenants: %v - if it is multi-tenant, "+
|
||||
"a domain/tenant mismatch would only surface during the conversion", err),
|
||||
}, ""
|
||||
}
|
||||
if len(tenants) > 0 {
|
||||
if len(layout.Tenants) == 0 {
|
||||
return CheckResult{Status: StatusOK, Detail: "single-tenant: no tenant principals, so no account can mismatch its domain"}, ""
|
||||
}
|
||||
|
||||
plan := layout.Analyze()
|
||||
if len(plan.Problems) > 0 {
|
||||
details := make([]string, 0, len(plan.Problems))
|
||||
for _, p := range plan.Problems {
|
||||
details = append(details, fmt.Sprintf("%s: %s", p.Domain, p.Detail))
|
||||
}
|
||||
return CheckResult{
|
||||
Status: StatusFail,
|
||||
Detail: fmt.Sprintf("this instance has %d tenant(s) (%s), and Stalwart's migrate_v016.py does not carry tenant "+
|
||||
"membership onto accounts: it creates the Tenant and Domains, then every Account with a null tenantId, and the "+
|
||||
"apply is rejected with invalidForeignKey - during recovery-mode migration, with the service already stopped. "+
|
||||
"Migrate a multi-tenant install by hand, or wait for a converter that handles it",
|
||||
len(tenants), strings.Join(tenants, ", ")),
|
||||
Detail: fmt.Sprintf("this instance has %d tenant(s) (%s) in an arrangement v0.16 cannot represent - %s. "+
|
||||
"Resolve this in v0.15 first: give each tenant its own domains, or move the accounts into one tenant",
|
||||
len(layout.Tenants), strings.Join(layout.Tenants, ", "), strings.Join(details, "; ")),
|
||||
}, ""
|
||||
}
|
||||
return CheckResult{Status: StatusOK, Detail: "single-tenant: no tenant principals, so the conversion's null tenantId is harmless"}, ""
|
||||
if len(plan.Adoptions) > 0 {
|
||||
return CheckResult{
|
||||
Status: StatusWarn,
|
||||
Detail: fmt.Sprintf("this instance has %d tenant(s) (%s); %d domain(s) (%s) have no tenant of their own but are "+
|
||||
"used only by accounts of a single tenant. v0.16 requires them to match, so the conversion will assign each "+
|
||||
"domain to that tenant - the accounts migrate intact, but those domains become tenant-owned",
|
||||
len(layout.Tenants), strings.Join(layout.Tenants, ", "),
|
||||
len(plan.Adoptions), strings.Join(plan.Adoptions, ", ")),
|
||||
}, ""
|
||||
}
|
||||
return CheckResult{
|
||||
Status: StatusOK,
|
||||
Detail: fmt.Sprintf("%d tenant(s) (%s), and every account already sits on a domain of its own tenant",
|
||||
len(layout.Tenants), strings.Join(layout.Tenants, ", ")),
|
||||
}, ""
|
||||
}); err != nil {
|
||||
return report, err
|
||||
}
|
||||
|
||||
@@ -209,12 +209,15 @@ func accountKey(p restPrincipal) string {
|
||||
|
||||
// TenantNames returns the tenant principals on a v0.15.x instance.
|
||||
//
|
||||
// Multi-tenancy has to be detected before a migration starts, because
|
||||
// Stalwart's own converter does not survive it: it emits the Tenant and the
|
||||
// Domains correctly and then every Account with `tenantId: null`, so the
|
||||
// account references a tenant-owned domain while belonging to no tenant and
|
||||
// the apply is rejected with `invalidForeignKey`. Observed on a real
|
||||
// migration, at the point where the mail server was already stopped.
|
||||
// Multi-tenancy has to be established before a migration starts. v0.16
|
||||
// requires a tenant-scoped account to sit on a domain owned by that same
|
||||
// tenant; v0.15 did not, and Stalwart's converter carries the two facts
|
||||
// over independently, so an install that is valid today can convert into a
|
||||
// plan the new server rejects with `invalidForeignKey` on the Domain
|
||||
// reference - during the recovery-mode apply, with the mail server already
|
||||
// stopped and the store already at schema v6. See FetchTenantLayout, which
|
||||
// builds on this to predict that outcome, and applyplan.ReconcileDomainTenants,
|
||||
// which repairs the plan.
|
||||
func (c *Client) TenantNames(ctx context.Context) ([]string, error) {
|
||||
tenants, err := c.restPrincipals(ctx, "tenant")
|
||||
if err != nil {
|
||||
|
||||
@@ -0,0 +1,291 @@
|
||||
// SPDX-FileCopyrightText: 2026 LINUXexpert-org
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
package stalwartapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// TenantLayout is who-belongs-to-which-tenant on a v0.15.x instance, in
|
||||
// enough detail to predict whether the v0.16 conversion will produce a plan
|
||||
// the new server accepts.
|
||||
//
|
||||
// v0.16 requires a tenant-scoped account to sit on a domain owned by that
|
||||
// same tenant, for its primary domain and for every alias. v0.15 imposed no
|
||||
// such rule, so an install can be perfectly valid today and unconvertible
|
||||
// tomorrow. Establishing that here - while the server is still running and
|
||||
// nothing has been stopped - is the whole point: the alternative is finding
|
||||
// out during the recovery-mode apply, which is the one moment in the run
|
||||
// with no way forward and no way back.
|
||||
type TenantLayout struct {
|
||||
// Tenants is every tenant principal's name.
|
||||
Tenants []string
|
||||
// DomainTenant maps a declared domain name to its tenant name. Domains
|
||||
// that exist only inside an email address are absent, which mirrors the
|
||||
// converter: it infers those domains and gives them no tenant.
|
||||
DomainTenant map[string]string
|
||||
// Principals is every account, group and mailing list, with the tenant
|
||||
// it belongs to and the domains it touches.
|
||||
Principals []PrincipalTenancy
|
||||
}
|
||||
|
||||
// PrincipalTenancy is one account's tenant and the domains it references.
|
||||
type PrincipalTenancy struct {
|
||||
Name string
|
||||
Type string
|
||||
Tenant string // "" for a global principal
|
||||
Domains []string // primary plus alias domains, lowercased
|
||||
}
|
||||
|
||||
// flexString reads a field that a v0.15 instance may return either as a
|
||||
// plain string or wrapped - migrate_v016.py's own pv_string tolerates the
|
||||
// same shapes, and a preflight that only understood one of them would
|
||||
// silently see every account as global.
|
||||
func flexString(raw json.RawMessage) string {
|
||||
if len(raw) == 0 {
|
||||
return ""
|
||||
}
|
||||
var s string
|
||||
if err := json.Unmarshal(raw, &s); err == nil {
|
||||
return s
|
||||
}
|
||||
var obj map[string]json.RawMessage
|
||||
if err := json.Unmarshal(raw, &obj); err == nil {
|
||||
for _, key := range []string{"string", "name", "id"} {
|
||||
if v, ok := obj[key]; ok {
|
||||
var inner string
|
||||
if json.Unmarshal(v, &inner) == nil && inner != "" {
|
||||
return inner
|
||||
}
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
var list []json.RawMessage
|
||||
if err := json.Unmarshal(raw, &list); err == nil && len(list) > 0 {
|
||||
return flexString(list[0])
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// detailedPrincipal is the per-principal view, which carries the tenant the
|
||||
// paginated list does not reliably include.
|
||||
type detailedPrincipal struct {
|
||||
Type json.RawMessage `json:"type"`
|
||||
Name json.RawMessage `json:"name"`
|
||||
Tenant json.RawMessage `json:"tenant"`
|
||||
Emails []string `json:"emails"`
|
||||
}
|
||||
|
||||
// principalDetail fetches one principal by name. The response may or may
|
||||
// not be wrapped in a "data" envelope depending on the point release, so
|
||||
// both are accepted.
|
||||
func (c *Client) principalDetail(ctx context.Context, name string) (detailedPrincipal, error) {
|
||||
var out detailedPrincipal
|
||||
endpoint := strings.TrimRight(c.BaseURL, "/") + "/api/principal/" + url.PathEscape(name)
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
req.SetBasicAuth(c.Username, c.Password)
|
||||
resp, err := c.httpClient().Do(req)
|
||||
if err != nil {
|
||||
return out, fmt.Errorf("stalwartapi: fetch principal %q: %w", name, err)
|
||||
}
|
||||
body, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
|
||||
resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return out, fmt.Errorf("stalwartapi: GET %s returned %s", endpoint, resp.Status)
|
||||
}
|
||||
if readErr != nil {
|
||||
return out, fmt.Errorf("stalwartapi: read principal %q: %w", name, readErr)
|
||||
}
|
||||
|
||||
var envelope struct {
|
||||
Data json.RawMessage `json:"data"`
|
||||
}
|
||||
payload := body
|
||||
if json.Unmarshal(body, &envelope) == nil && len(envelope.Data) > 0 {
|
||||
payload = envelope.Data
|
||||
}
|
||||
if err := json.Unmarshal(payload, &out); err != nil {
|
||||
return out, fmt.Errorf("stalwartapi: parse principal %q: %w", name, err)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// domainsOf returns every domain a principal touches: the domain in its
|
||||
// name, if it is an address, plus one per email.
|
||||
func domainsOf(name string, emails []string) []string {
|
||||
seen := map[string]bool{}
|
||||
var out []string
|
||||
add := func(addr string) {
|
||||
at := strings.LastIndex(addr, "@")
|
||||
if at < 0 || at == len(addr)-1 {
|
||||
return
|
||||
}
|
||||
d := strings.ToLower(strings.TrimSpace(addr[at+1:]))
|
||||
if d == "" || seen[d] {
|
||||
return
|
||||
}
|
||||
seen[d] = true
|
||||
out = append(out, d)
|
||||
}
|
||||
add(name)
|
||||
for _, e := range emails {
|
||||
add(e)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
// FetchTenantLayout builds a TenantLayout from a v0.15.x instance.
|
||||
//
|
||||
// It returns an empty layout, and no error, on a single-tenant install:
|
||||
// there is nothing that can mismatch, and that is the common case.
|
||||
func (c *Client) FetchTenantLayout(ctx context.Context) (*TenantLayout, error) {
|
||||
tenants, err := c.TenantNames(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
layout := &TenantLayout{Tenants: tenants, DomainTenant: map[string]string{}}
|
||||
if len(tenants) == 0 {
|
||||
return layout, nil
|
||||
}
|
||||
|
||||
domains, err := c.restPrincipals(ctx, "domain")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, d := range domains {
|
||||
if d.Name == "" {
|
||||
continue
|
||||
}
|
||||
detail, err := c.principalDetail(ctx, d.Name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
layout.DomainTenant[strings.ToLower(d.Name)] = flexString(detail.Tenant)
|
||||
}
|
||||
|
||||
for _, pType := range []string{"individual", "group", "list"} {
|
||||
principals, err := c.restPrincipals(ctx, pType)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, p := range principals {
|
||||
if p.Name == "" {
|
||||
continue
|
||||
}
|
||||
detail, err := c.principalDetail(ctx, p.Name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
emails := p.Emails
|
||||
if len(detail.Emails) > 0 {
|
||||
emails = detail.Emails
|
||||
}
|
||||
layout.Principals = append(layout.Principals, PrincipalTenancy{
|
||||
Name: p.Name,
|
||||
Type: pType,
|
||||
Tenant: flexString(detail.Tenant),
|
||||
Domains: domainsOf(p.Name, emails),
|
||||
})
|
||||
}
|
||||
}
|
||||
return layout, nil
|
||||
}
|
||||
|
||||
// TenancyProblem is one domain whose users cannot all be represented in
|
||||
// v0.16.
|
||||
type TenancyProblem struct {
|
||||
Domain string
|
||||
Detail string
|
||||
}
|
||||
|
||||
// TenancyPlan is what the conversion will have to do to this layout.
|
||||
type TenancyPlan struct {
|
||||
// Adoptions are domains with no tenant of their own that will be given
|
||||
// one, because the only tenant-scoped accounts using them agree.
|
||||
Adoptions []string
|
||||
// Problems are the domains v0.16 cannot represent at all.
|
||||
Problems []TenancyProblem
|
||||
}
|
||||
|
||||
// Analyze predicts whether this layout converts cleanly, applying exactly
|
||||
// the rule the server enforces and the same repair applyplan performs.
|
||||
func (l *TenantLayout) Analyze() TenancyPlan {
|
||||
var plan TenancyPlan
|
||||
if len(l.Tenants) == 0 {
|
||||
return plan
|
||||
}
|
||||
|
||||
// domain -> tenant -> an example principal requiring it
|
||||
required := map[string]map[string]string{}
|
||||
for _, p := range l.Principals {
|
||||
if p.Tenant == "" {
|
||||
continue // a global principal constrains nothing
|
||||
}
|
||||
for _, d := range p.Domains {
|
||||
if required[d] == nil {
|
||||
required[d] = map[string]string{}
|
||||
}
|
||||
if _, ok := required[d][p.Tenant]; !ok {
|
||||
required[d][p.Tenant] = p.Name
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
domains := make([]string, 0, len(required))
|
||||
for d := range required {
|
||||
domains = append(domains, d)
|
||||
}
|
||||
sort.Strings(domains)
|
||||
|
||||
for _, d := range domains {
|
||||
wanted := required[d]
|
||||
names := make([]string, 0, len(wanted))
|
||||
for t := range wanted {
|
||||
names = append(names, t)
|
||||
}
|
||||
sort.Strings(names)
|
||||
|
||||
if len(names) > 1 {
|
||||
var parts []string
|
||||
for _, t := range names {
|
||||
parts = append(parts, fmt.Sprintf("%s (e.g. %s)", t, wanted[t]))
|
||||
}
|
||||
plan.Problems = append(plan.Problems, TenancyProblem{
|
||||
Domain: d,
|
||||
Detail: fmt.Sprintf("used by accounts from more than one tenant: %s - v0.16 allows a domain "+
|
||||
"to belong to at most one tenant, and requires each tenant-scoped account to sit on its "+
|
||||
"own tenant's domain", strings.Join(parts, ", ")),
|
||||
})
|
||||
continue
|
||||
}
|
||||
|
||||
want := names[0]
|
||||
switch declared, isDeclared := l.DomainTenant[d]; {
|
||||
case !isDeclared || declared == "":
|
||||
// Either inferred from an address, or declared with no tenant.
|
||||
// Either way the conversion gives it no tenant and the account
|
||||
// is rejected - this is the case that broke production.
|
||||
plan.Adoptions = append(plan.Adoptions, d)
|
||||
case declared != want:
|
||||
plan.Problems = append(plan.Problems, TenancyProblem{
|
||||
Domain: d,
|
||||
Detail: fmt.Sprintf("belongs to tenant %s, but %s (tenant %s) uses it - v0.16 rejects an "+
|
||||
"account whose tenant differs from its domain's", declared, wanted[want], want),
|
||||
})
|
||||
}
|
||||
}
|
||||
return plan
|
||||
}
|
||||
@@ -0,0 +1,138 @@
|
||||
// SPDX-FileCopyrightText: 2026 LINUXexpert-org
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
|
||||
package stalwartapi
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestFlexStringAcceptsEveryShapeAV015InstanceReturns(t *testing.T) {
|
||||
cases := map[string]string{
|
||||
`"acme"`: "acme",
|
||||
`{"string":"acme"}`: "acme",
|
||||
`{"name":"acme"}`: "acme",
|
||||
`["acme","other"]`: "acme",
|
||||
`null`: "",
|
||||
`{}`: "",
|
||||
`[]`: "",
|
||||
`{"other":"ignored"}`: "",
|
||||
}
|
||||
for raw, want := range cases {
|
||||
if got := flexString(json.RawMessage(raw)); got != want {
|
||||
t.Errorf("flexString(%s) = %q, want %q", raw, got, want)
|
||||
}
|
||||
}
|
||||
if got := flexString(nil); got != "" {
|
||||
t.Errorf("flexString(nil) = %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDomainsOfCollectsNameAndAliasDomains(t *testing.T) {
|
||||
got := domainsOf("[email protected]", []string{"[email protected]", "[email protected]", "malformed", "trailing@"})
|
||||
want := []string{"alias.net", "example.com"}
|
||||
if strings.Join(got, ",") != strings.Join(want, ",") {
|
||||
t.Errorf("domainsOf = %v, want %v", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnalyzeIsEmptyForSingleTenant(t *testing.T) {
|
||||
l := &TenantLayout{DomainTenant: map[string]string{}}
|
||||
plan := l.Analyze()
|
||||
if len(plan.Adoptions) != 0 || len(plan.Problems) != 0 {
|
||||
t.Errorf("single-tenant layout produced %+v", plan)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnalyzeFlagsUndeclaredDomainForAdoption(t *testing.T) {
|
||||
// The production failure: a tenant account with an address on a domain
|
||||
// that was never declared, so the conversion gives it no tenant.
|
||||
l := &TenantLayout{
|
||||
Tenants: []string{"acme"},
|
||||
DomainTenant: map[string]string{"acme-corp.test": "acme"},
|
||||
Principals: []PrincipalTenancy{
|
||||
{Name: "bob", Tenant: "acme", Domains: []string{"acme-corp.test", "inferred.test"}},
|
||||
},
|
||||
}
|
||||
plan := l.Analyze()
|
||||
if len(plan.Problems) != 0 {
|
||||
t.Fatalf("unexpected problems: %+v", plan.Problems)
|
||||
}
|
||||
if len(plan.Adoptions) != 1 || plan.Adoptions[0] != "inferred.test" {
|
||||
t.Errorf("adoptions = %v, want [inferred.test]", plan.Adoptions)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnalyzeIgnoresGlobalAccountsOnTenantDomains(t *testing.T) {
|
||||
// Verified against 0.16.14: this direction applies cleanly, so it must
|
||||
// not be reported as anything.
|
||||
l := &TenantLayout{
|
||||
Tenants: []string{"acme"},
|
||||
DomainTenant: map[string]string{"acme-corp.test": "acme"},
|
||||
Principals: []PrincipalTenancy{
|
||||
{Name: "admin", Tenant: "", Domains: []string{"acme-corp.test"}},
|
||||
},
|
||||
}
|
||||
plan := l.Analyze()
|
||||
if len(plan.Adoptions) != 0 || len(plan.Problems) != 0 {
|
||||
t.Errorf("global account on a tenant domain produced %+v", plan)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnalyzeReportsDomainSharedByTwoTenants(t *testing.T) {
|
||||
l := &TenantLayout{
|
||||
Tenants: []string{"alpha", "beta"},
|
||||
DomainTenant: map[string]string{},
|
||||
Principals: []PrincipalTenancy{
|
||||
{Name: "u1", Tenant: "alpha", Domains: []string{"shared.test"}},
|
||||
{Name: "u2", Tenant: "beta", Domains: []string{"shared.test"}},
|
||||
},
|
||||
}
|
||||
plan := l.Analyze()
|
||||
if len(plan.Problems) != 1 {
|
||||
t.Fatalf("problems = %+v, want one", plan.Problems)
|
||||
}
|
||||
if len(plan.Adoptions) != 0 {
|
||||
t.Errorf("a conflicting domain was also queued for adoption: %v", plan.Adoptions)
|
||||
}
|
||||
for _, want := range []string{"alpha", "beta", "u1", "u2"} {
|
||||
if !strings.Contains(plan.Problems[0].Detail, want) {
|
||||
t.Errorf("detail %q omits %q", plan.Problems[0].Detail, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnalyzeReportsAccountOnAnotherTenantsDomain(t *testing.T) {
|
||||
l := &TenantLayout{
|
||||
Tenants: []string{"alpha", "beta"},
|
||||
DomainTenant: map[string]string{"owned.test": "alpha"},
|
||||
Principals: []PrincipalTenancy{
|
||||
{Name: "u2", Tenant: "beta", Domains: []string{"owned.test"}},
|
||||
},
|
||||
}
|
||||
plan := l.Analyze()
|
||||
if len(plan.Problems) != 1 || plan.Problems[0].Domain != "owned.test" {
|
||||
t.Fatalf("problems = %+v", plan.Problems)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnalyzeAcceptsAConsistentMultiTenantLayout(t *testing.T) {
|
||||
l := &TenantLayout{
|
||||
Tenants: []string{"alpha", "beta"},
|
||||
DomainTenant: map[string]string{
|
||||
"alpha.test": "alpha",
|
||||
"beta.test": "beta",
|
||||
},
|
||||
Principals: []PrincipalTenancy{
|
||||
{Name: "u1", Tenant: "alpha", Domains: []string{"alpha.test"}},
|
||||
{Name: "u2", Tenant: "beta", Domains: []string{"beta.test"}},
|
||||
{Name: "admin", Tenant: "", Domains: []string{"alpha.test"}},
|
||||
},
|
||||
}
|
||||
plan := l.Analyze()
|
||||
if len(plan.Adoptions) != 0 || len(plan.Problems) != 0 {
|
||||
t.Errorf("a already-consistent multi-tenant layout produced %+v", plan)
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user