jcoffey-dev is traveling from Thursday 1 October through Sunday 4 October. Issues and pull requests are welcome, and will get an answer after that. Thanks for your patience.
Personal-data catalog spec, §6 (Phase 3c).
inbuxa:DataInventory/get evaluates the catalog against the server's
live settings and says what this server holds: for each source and each
object that can hold personal data, its categories and whose data it
is, whether it is collected here at all, what bounds its retention (the
live value of the setting that does, or unbounded), whether it leaves
the host and to which endpoints, and a summary. Every host that
receives something is listed once as a candidate processor with what it
receives. Inside a tenant it answers with the tenant's slice and none
of the server's processors. Read-only, with sysComplianceGet.
inbuxa:InventorySnapshot/get is the history: a dated copy of the
evaluated inventory, recorded when it changes -- after a registry write
to an object the inventory reads, after inbuxa's log, audit or AI
settings change, and on the daily clean-up -- and kept as long as the
audit log's records. ids: null lists every snapshot, newest first; the
full inventory only when asked for.
The catalog is embedded and parsed at start (new dependency: toml,
MIT/Apache); the evaluation is a pure function of it and the live
facts, so each configuration is tested without a server. Loopback
endpoints stay on the host; any other configured endpoint leaves it.
Tested: unit tests for the evaluation (a new install's defaults, an
external blob store, a hosted AI endpoint, telemetry off, a tenant's
slice, hosts from URLs, loopback), snapshots, and the fact gathering's
store and duration rules; the compliance system test, extended (the
officer reads the inventory, a plain user is refused, a tenant's
officer sees its slice and no processors, a webhook to another host
becomes a processor and a snapshot names x:WebHook, a retention change
reads through); the system, audit, legal hold and account lock suites;
fork checks. The system suite failed once of three runs with an email
import's blob not found, in antispam.rs; the same happened once in
purge.rs on the previous branch. Nothing here touches uploads; noted
for a separate look.
Personal-data catalog spec, §7 (settled 2026-09-28).
sysComplianceGet (673) sees the data inventory and compliance
overview: superusers and, for their tenant's slice, tenant
administrators, by default and through the one-time grants on servers
that already have their roles stored.
A Compliance Officer role at server level holds it with reading and
exporting the audit log, placing, widening, releasing and exporting
legal holds, seeing account locks, and reading accounts, lists,
domains, tenants and roles. It changes no server setting, creates or
deletes no account, and can't shorten audit retention.
A tenant's accounts can hold only roles of their own tenant (MT-3), so
the tenant role is one "Compliance Officer" role per tenant, without
holds (LH-13): made once for every tenant a server has, and whenever a
tenant is created. While nobody holds it, it is removed with its tenant
so it doesn't block the delete, and put back if the delete is refused
for another reason. Both roles carry a user's own permissions too,
since roles given to a person replace the default user role, which a
tenant's accounts can't hold anyway.
Every server makes these once, new or existing -- the built-in roles
are only made on a server with none -- and records each under P c, so a
role an administrator deletes stays deleted.
Tested: unit tests (neither role changes a setting beyond a user's
own; holds for the server's officer only; per-place records); a new
compliance system test (one server-level role; an officer reads the
audit log, places and releases a hold, and is refused a setting, an
account and audit retention; a tenant gets its role, whose holder reads
the tenant's audit log and no holds; a tenant with an unused role is
deleted and the role goes with it); the system, audit, legal hold,
account lock and SCIM suites; fork checks. The directory suite needs
its LDAP container and wasn't run here.
A 3-node rehearsal found that saving an MtaDeliverySchedule left it
unknown to the queue ("Queue strategy not found") until someone ran
x:Action ReloadSettings; only Directory and Authentication writes
reloaded (DIR-17). The admin UI has to remember a separate reload after
every save, and a script or API client that doesn't gets a server
running stale settings.
x:<Object>/set now reloads the running settings when it created,
updated or destroyed an object they are built from, and broadcasts the
same RegistryChange::Reload over the coordinator as ReloadSettings, so
every node applies it:
- Settings objects (MTA, spam filter, listeners, tracers, Sieve system
scripts, cluster roles, directories, ...: the object types the core,
telemetry, listener and directory builders read) get a full reload.
- Certificates, lookup stores and blocked/allowed IPs get their own
targeted reloads.
- Accounts, domains, roles and other data read as needed, stores (they
take a restart) and applications (their own reload action) get none.
Full reloads are coalesced: a write waits for a reload that started
after it was stored and joins one if it can, so a burst of writes, or
a request with many objects, costs one or two reloads, not one each.
The write itself is never undone. When the reload is refused (build
errors in objects that were working, the rule from the previous
commit), the set response says so in a new x:settingsReload field,
{"applied": false, "description": "Saved, but the running settings
were not reloaded. <object>: <error>"}; {"applied": true} otherwise.
The field is absent when the write needs no reload. The description
helper is shared with ReloadSettings' refusal.
Each reload sends the queue a ReloadSettings event, so the SMTP test
harness's read_event, try_read_event and assert_no_events now pass over
those; expect_reload_settings still waits for one.
system::auto_reload::settings_reload_tests (new): an MtaVirtualQueue
and an MtaDeliverySchedule created over JMAP are in the running
settings with no ReloadSettings, and gone once destroyed; eight
concurrent creates all land; a write whose reload fails is stored and
reported applied: false with the error; a domain write carries no
x:settingsReload. On main the new schedule is missing. The cluster
broadcast test (three nodes, PostgreSQL + NATS) now checks that every
node has a schedule created on node 0 without a reload.
A 3-node rehearsal found every settings reload refused, cluster-wide,
because one node couldn't resolve the Pyzor server:
- PyzorConfig::parse resolved the host while building the settings and
made a failed lookup a build error. It now keeps the host and port and
resolves when a message is checked (an IP address is used as is, a
name is reused for five minutes, the lookup counts against the Pyzor
timeout). A failure there is a Pyzor error for that message.
- A milter's hostname was resolved the same way, with a blocking
to_socket_addrs in async code. An IP address is kept; a name is now
resolved on each connection.
Other build-time I/O is already non-fatal: directories that can't
connect become unavailable with a warning (DIR-21), and the AI model
locality check only warns.
reload_registry swapped the core only when the whole build was free of
errors, while boot runs with whatever built. One failing object thus
refused every later reload, and the running settings went stale. Now a
reload is refused only for errors in objects that built when the
running settings were built (at boot or by the last applied reload):
applying it would lose those. Objects that already failed then are
missing from the running settings anyway, as at boot, so their errors
are logged and returned as known_errors but don't hold the reload back.
Refusing on new errors keeps a bad edit from taking a working object
out of service; the admin gets the error instead.
ReloadSettings now says "Settings were not reloaded." and names the
object and its error ("Tracer with id ...: Only one console tracer is
allowed"), with a count of any further errors. A refused reload after a
directory change logs its errors too.
system::reload::reload_tests (new): with Pyzor enabled on an
unresolvable host, ReloadSettings succeeds (on main it fails with
"Invalid address: failed to lookup address information"); an IP host
needs no lookup; a new build error refuses the reload, names the object
and leaves the running settings unchanged; the same error, once known
from the running settings' build, no longer blocks; once fixed, a new
error there blocks again. smtp::inbound::milter's session test now
names its milter "localhost", so the connect-time lookup is exercised.
Deleting a tenant now also removes its stored inbuxa:TenantProtocolPolicy,
in the same place the registry's other per-type clean-ups run. Without it
the row outlived the tenant, and a tenant that later came to have the same
id would have started with legacy protocols off.
The e2e deletes a tenant whose switch a server administrator had turned
off, and would check that a new tenant with the same id starts with them
on. On this build the registry hands out a fresh id instead ("d" after
"c"), so the reuse -- and with it the removal -- isn't observable over
JMAP; the test says so rather than passing silently. The risk it guards
was therefore smaller than feared, and the change is mostly about not
leaving an orphaned row behind. All 72 checks pass.
While legacy mail protocols are off, x:NetworkListener/set refuses to
create a listener the switch would close, and refuses an update that would
turn an existing one into such a listener -- otherwise changing a
listener's protocol would walk straight past the check. The refusal is
invalidProperties on protocol (or on bind, for a submission listener once
SMTP is unlocked, since its port is what makes it one), and its description
names inbuxa:ProtocolPolicy and says to turn legacy protocols back on first.
The rule is the switch's own, listeners::closes, so what can't be added is
exactly what the switch would close: locked protocols (SMTP, LMTP, HTTP)
and the inbound port are never refused. Putting saved listeners back
(LP-5) writes through the registry, not /set, so it is unaffected.
The e2e changes with it. LP-6's check that an IMAP listener "created by
mistake" refuses sign-in can't be set up any more -- LP-4 is what stops
that listener existing -- so that step now proves test 4 instead: creating
an IMAP listener is refused, naming the policy; an SMTP listener can still
be created; and updating it to IMAP is refused. LP-6 stays proven live over
submission, and its IMAP wording by unit tests. All checks pass.
The AGPL asks a modified version to carry prominent notices saying it was
modified, and giving a date. Publishing the source is the conveyance that
asks for it, so it wants doing before the repository is public rather than
at the release.
Every upstream file the fork changed now says so in its header, beneath the
notice it came with: 164 files, found by diffing against the upstream
snapshot branch rather than by guessing, so the list is what actually
differs. Files the fork wrote itself already carry their own copyright and
need nothing. Upstream's notices are untouched, which its licence requires
and which was already true.
The README says the same thing in prose, since the obligation is on the
work as a whole and not only its Rust files.
Builds unchanged: the server and the test binary both compile.
Every one of the 14 was `#[cfg(not(feature = "enterprise"))]` on the arm the
fork always compiles: the Enterprise arms went with the import, and nothing
turns the feature on. Removing the attribute leaves the same code, now
unconditional, in 11 files.
Two of them looked like behavior worth checking before touching: the
`validate_tenant_quota` stub that always passes, and the refusal to cancel a
pending DestroyAccount task. The stub is vestigial — the rebuilt
multi-tenancy enforces quotas in `crates/features/src/tenancy/quota.rs` for
those objects and more — and the refusal is undelete's open question, which
this change leaves exactly as it was.
The binary builds with no new warnings, and `system_tests` and `jmap_tests`,
which cover the touched registry, task-manager and auth paths, both pass.
The feature definitions stay in the manifests, inert: taking them out would
widen every sync's diff for nothing.
A write to x:Directory or Authentication reloads the directories at once,
here and across the cluster. A directory that fails to open is logged as
a warning against its id and becomes unavailable; before, it was a build
error, and any build error stopped every later reload from applying.
per_domain_directory_tests covers acceptance tests 1, 3, 4, 6, 7, 9, 11,
15 and 19 over SQL directories on SQLite files: each domain against its
own directory and the default, no fallback to internal passwords, a
directory answering for another directory's domain, aliases and groups
dropped, recipients through the directory and a 4xx while it's down, app
passwords while it's down, password changes refused and then allowed
after a move to the internal directory, linked directories, tenant
foreign keys, and changes taking effect without a reload.
Every minute the node that calculates metrics evaluates each enabled
x:Alert, read from the registry, with metric('name') or the underscore
name. An alert fires when its condition turns true: it emits
telemetry.alert-event with the rendered message, and queues one email per
recipient through the outbound queue, placeholders filled and
Auto-Submitted set. An alert naming an unknown metric is refused on save.
The shared alerts suite runs; every telemetry suite is un-gated.
A lossy collector subscriber keeps each inbound SMTP session that reached
MAIL FROM and each delivery attempt, info level and above and never raw
I/O, at most 1,000 events with strings cut at 4 KiB, and writes it when
the span closes as an x:Trace under the telemetry key class, scheduling
its indexing. The index task builds a document of event types, queue ids
and keywords when indexTelemetry is on. x:Trace/get derives timestamp,
from, to and size; /query filters by opening event, text, queueId and
time; /set destroys only. The data purge honours holdTracesFor. The shared
tracing and webhook suites run.
The classifier sends only the subject and text, between unforgeable markers
after the operator's prompt, to an OpenAI-compatible endpoint the operator
configured; nothing is preset. Its answer maps to an LLM_ tag whose score is
clamped (+5.0, -1.0 by default) and can never discard or reject on its own;
X-Spam-LLM is sanitized, encoded and folded, and a planted one is removed.
Failures, timeouts past the ceiling, a full slot or a paused model leave
mail flowing untagged. llm_prompt answers trusted scripts, and accounts
holding interactAi within an hourly limit. Redirects aren't followed and no
content or secret is logged. The limits live in inbuxa:AiLimits.
Acceptance tests 1 and 3 to 21; test 2 as the re-enabled shared llm case,
whose setup no longer waits on a rules file from a developer's own path;
test 22 written as the ignored ai_compat.
Logos resolve domain, then tenant, then server-wide, then the built-in, with
subdomains finding their domain. GET /logo serves a data-URL image, redirects
to a URL logo without fetching it, sandboxes SVG, and answers 404 when no
custom logo applies. Emails embed the first PNG, JPEG or GIF logo. Logo and
template writes are checked; stored templates are read at send time, always
escaped, and fall back to the built-in with a build warning when they don't
parse. The RSVP page is served byte for byte with a CSP and no-referrer. The
sign-in and RSVP pages load the logo through an image element. MT-22's
session logo follows the chain to the server-wide logo.
Acceptance tests 1 to 17; test 18 written as the ignored branding_compat.
With archiveDeletedAccountsFor set, a destroyed account's record is kept in
the fork subspace with its id, its DestroyAccount task is due at the end of
the period, and its shares are suspended both ways. Its addresses can't be
taken by new accounts, aliases, lists or masks. inbuxa:DeletedAccount/get
lists kept accounts to server and tenant administrators; /set restores one
with a new password (same id, task cancelled, shares reinstated) or destroys
it now. The destroy task also clears undelete's own records.
Acceptance test 14; test 16 written as the ignored undelete_compat.
Every way of deleting mail for good (JMAP, IMAP expunge, POP3, Trash
emptying, mailbox removal) notes the message's mailboxes and keywords while
archiving is on, fixing its deadline then; when its data is finally removed
it becomes an x:ArchivedItem record, written as upstream writes them, with
its copy held until the deadline. Retention is read at deletion time, so a
change applies at once. Restore puts a message back in the mailboxes it was
in (Trash only if that was all), with its keywords, and removes the record;
over quota it stays archived. x:ArchivedItem/get returns status and
accountId; query filters on type, archivedAt and text; set requests a
restore once or destroys; /changes is a fork addition. Expired items go in
the data purge. The shared account-access rule moves to jmap::inbuxa::access.
system_tests now calls undelete::test, and the archiving gate is gone.
x:MaskedEmail is no longer refused as unbuilt. Creates generate the address
on an allowed domain and check the prefix, maxMaskedAddresses and the create
rate; updates keep server-set fields; enabled reads and writes map to the
shared state; query filters on enabled, forDomain and text; a tenant
administrator reaches its tenant's accounts' masks.
Inside a tenant, server-level object types are forbidden and x:Tenant reads
return only the caller's own tenant, which it can't change. Registry writes
refuse links across tenant boundaries in both directions, give a new
principal its domain's tenant, move a domain's principals and DKIM keys with
it into a tenant, refuse moves out of a tenant while its people remain, and
refuse creates past a tenant's count limits with overQuota and
limit.tenant-quota.
Upstream commit: 474dd0229cb20cf513036619781ed97bd8073c3f
Enterprise-only files removed or emptied: 63
Enterprise-only snippets removed: 117 in 50 files
Dangling module declarations removed: 5
Cargo edits turning enterprise off: 14
Verification: clean
Enterprise feature gates left for rebuilt features: 19 in 18 files
Produced by tools/fork/strip.py. The full report is in docs/fork/strip-reports/ on main.
jcoffey-dev