The runbook said nothing in it had been rehearsed. Now the sequence has
been, on data made up for the purpose: upstream 0.16.22 in a container as
the install running today, the fork beside it, both unprivileged with
CAP_NET_BIND_SERVICE. It rehearses the sequence, not the data, which is
what the compat tests are for. 27 of 27 checks passed and the rollback
took 1.5 seconds.
The question §"Open" asked about carrying a store back is answered, and
the answer is no. Upstream refuses to start on a store the fork has
opened: "Column families not opened: _". The fork adds one RocksDB column
family for masked email (SUBSPACE_INBUXA = b'_') and opens with
create_missing_column_families, so it creates it on first open; upstream
has no descriptor for it and RocksDB will not open a database holding one
it was not told about.
That makes step 3's "a copy, not a move" load-bearing in a way the step
did not say. One open by the fork is enough: pointing it at the original
even once, to check something, leaves the Enterprise install unable to
start, and there is no rollback after that. It fails loudly and before
reading anything, which is the good version of this failure, but it is
not recoverable.
Two things the rehearsal found that would have wasted time on the day:
memberTenantId does not come down from the domain and is refused on
create, so a tenant "admin" set up the obvious way is a server
administrator and the check passes while proving nothing; and IMAP's
INBOX is not JMAP's account, because mail from an unauthenticated sender
is filed as spam, so the two counts differ before and after alike.
What the rehearsal does not cover is in its README and in §"Open":
systemd and `systemctl disable stalwart` above all, ACME renewal, load,
the front ends, and INBUXA's own data.
Rehearsed the run against a real RocksDB store, and it died at startup
before checking anything: the harness inserts listeners of its own, the
registry keys them by name, and a real server already has a "jmap" and an
"imap". The message was "Primary key conflict on property name with
existing object NetworkListener", which says nothing about what to do.
Under NO_INSERT the harness now calls its listeners compat-jmap and so on,
and the same run gets through to the test's own checks.
run-compat.sh copies the store for each test and removes the copy after,
because several of these write to what they open: monitoring_compat purges
the history it reads and undelete_compat restores what it finds. The source
stays untouched, which matters when it is the only copy of a production
store anyone took that day.
INBUXA runs RocksDB, so a copy is a directory copy. The SQL backends would
need more than this: the harness builds its own container and connects to
fixed local credentials, so it cannot open a dump in place.
"You are not an owner of account X" is not a permission the server is
withholding; it is how far that identity can see. Answering it with "needs
sysMaskedEmailGet" sends you off to grant something that changes nothing.
A refusal that mentions ownership now says so, and says which account the
run wants: the administrator with the run of the server, with tenant
administrators passed as --tenant-admin.
A recording runs against a server that may not be up again soon, so an
administrator missing one permission shouldn't throw away the whole pass.
Each of the three is recorded on its own now: what the server allows is
written, what it refuses is named at the end with the permission it wants,
and the exit is still non-zero so an incomplete recording can't pass for a
finished one.
A refusal used to print the raw JMAP error. It now reads, for example,
"[email protected] may not x:Tenant/get: You are not authorized to perform
this action (needs sysTenantGet)".
Checked both ways: the refusal path against a stubbed client, where the
other two sections still record; the whole thing against a live test
server, which recorded 3 tenants and 53 masked addresses and exited 0.
A wrong tenant-administrator password failed only once the script had
already enumerated every account, on a server that may not be up for long.
All the identities are now checked against the session endpoint first, and
a refusal names each one that failed, with the two things that usually
explain it: basic authentication wants the account's name rather than its
email address, and an account with two-factor or OAuth-only sign-in needs
an app password. It also gives the curl line to test one on its own.
The unreachable message no longer suggests --insecure for a refused
connection; that hint is now only for a certificate it couldn't verify.
Three of the eight compat tests check INBUXA's data against a recording of
how the Enterprise server read it, and that recording can only be made
while that server is still up. SPEC §7 gives it 45 days from the notice, so
the capture shouldn't wait on the cutover being scheduled.
record-compat.py writes all three files: the tenants with their quotas and
members and what each tenant administrator sees, every masked address and
its state, and every archived item whole, since undelete_compat compares
every property it recorded. It only reads, and refuses to send a method
that isn't /get or /query, because it is the one tool here that runs
against the live server. Queries follow their pages, so a server that caps
one doesn't leave a short recording behind.
Exercised against the fork's own test server, which answers the same JMAP:
3 tenants with members, 8 masked addresses and 3 archived items, each in
the shape its test reads.
A few of upstream's dual-licensed files carry code from other projects
under MIT or BSD terms. The fork redistributes it, so their licenses
require the notices to travel with it. THIRD-PARTY.md reproduces them.
strip.py now reads the stripped tree's comments for another copyright
holder, another license, or a note that code came from somewhere else, and
names any file THIRD-PARTY.md doesn't cover. It reports, never fails: the
notice goes in with the merge that brings the release in.
On v0.16.22 it finds 14 files, all of them covered. The rest of the report
is byte-for-byte what the committed one says, so the scan disturbs nothing
it already did.
Upstream's Dockerfiles and CI pass --features "... enterprise" on the cargo
command line, which the manifest edits don't reach. 11 occurrences at
v0.16.22. Dockerfile.build and Dockerfile.fdb are now scanned too, whatever
their extension.
jcoffey-dev