jcoffey-dev is traveling from Thursday 1 October through Sunday 4 October. Issues and pull requests are welcome, and will get an answer after that. Thanks for your patience.
Phase 4 of the journaling spec.
- inbuxa:JournalEntry/query and /get (sysJournalSearch): filter by time,
sender, recipient, either, direction, subject words, Message-ID and
journal, newest first; the whole report only when asked for.
- inbuxa:JournalExport/set (sysJournalExport): a reason is required; a
ZIP of the matching reports with manifest.csv, exceptions.csv and
manifest.sha256, up to 10,000 reports and 1 GB.
- inbuxa:JournalVerification/set (sysJournalGet): chains and reports
rechecked.
- Every search, listing, read, export and check is written to the audit
log before anything is returned, with existing actions only.
- Catalog entries for the three objects; spec as-built notes.
journal_tests: administrators can't search; a Compliance Officer searches,
lists, reads a report, exports (reason required) and checks the chain;
the officer can't change journals; each of those is in the audit log.
Phase 3 of the journaling spec.
- A journal's destination: builtIn (true for journals stored before) and
archiveAddress, at least one. Reports to an archive are queued from the
empty sender, one per address, flagged so they're never journaled.
- A pending record per report. When the queue lets go of one without
delivering it (refused, expired, deleted), it becomes its own entry in
the built-in journal under the sending journals' retention, the
journal's archiveFailures (count, last time, reason) goes up, and the
audit log records it; if that can't be written it stays queued.
- Journal it: a rule action naming a journal, on mail flow rules and
beside a DLP rule's block, warn or hold. A journal whose scope chooses
nobody takes only what rules send it.
- The report lists recipients a rule added or redirected to under
"Added by rule", by rule name.
- A rule's route is cleared between messages in one SMTP session, with the
new journal marks; a second message used to keep the first one's route.
tests/src/system/journal.rs: destination validation, a rule-only journal
fed by a rule that also adds a recipient, an unreachable archive's report
kept in the built-in journal with the failure counted, a report delivered
to an archive here and not journaled itself.
Phase 2 of the journaling spec.
- A copy of each message is taken in MessageWrapper::queue, after DLP and
transport rules, for every enabled journal that takes it (direction and
scope: everyone, or accounts, groups, domains, tenants). If the copy
can't be taken the message isn't queued (temporary failure).
- The journal report: the envelope one field a line (sender, To, Cc, Bcc
from the envelope, list members from their ORCPT, direction, held for
review), then the queued message byte for byte as message/rfc822.
- The built-in journal under J in the inbuxa subspace: one chain per node
whose links name each entry by SHA-256, so entries can expire out of
chain order; purge leaves a marker, and verify catches an entry changed
or removed early and a report that doesn't match.
- Retention per journal (30 to 3650 days); an entry keeps what it was
written with. The daily maintenance purges what's due, keeping entries
whose people a legal hold covers (deleted accounts a hold keeps too),
and records the counts in the audit log.
- inbuxa:Journal get/set, audited by the request layer. Permissions
680-683: administrators see and change journals; the Compliance Officer
sees, searches and exports. Whoever changes journals may grant search and
export without holding them, so officers can still be appointed.
- Catalog entries (inbuxa:Journal, source "journal"); spec as-built notes.
tests/src/system/journal.rs: validation, internal mail with a Bcc,
outgoing into two journals, incoming over LMTP, the report and its
original, tamper and early removal caught, hold-aware purge, retention
changes leave entries alone, disabled and removed journals take nothing.
senderGroup, senderTenant and recipientGroup conditions now read and write group and tenant ids as JMAP ids ("b", "c"…), like legal hold scopes and the rest of the API, so the console can use its object pickers; plain numbers are still read. Held as numbers for matching. Unit test for both forms and a bad id; mail_rules_tests round-trips a tenant condition over JMAP.
inbuxa:DlpSettings (singleton, urn:inbuxa:jmap): keepHeldDays, 1 to 90,
7 by default (settled answer 5 made it a setting). sysDlpPolicyGet reads
it, sysDlpPolicyUpdate changes it, server-level, audited by the request
layer. Each held message keeps the days it was given, and the sender's
notices say that number. Privacy catalog entry; spec §2.6 updated.
mail_rules_tests: 7 by default, 0 refused, 3 set and a message held
afterwards expires 3 days after it was held, the expiry notice says 3.
An EmailSubmission create's response carries inbuxa:held (dlp-and-mail-flow-rules spec, §2.6, §4): true when the message is held for review, false otherwise, so the webmail can say so at once. A sender can't read the review queue, and a held message's sendAt is its real send time, not the century-off release, so this is how the sender learns. mail_rules_tests checks both values.
The hold action now holds (dlp-and-mail-flow-rules spec, §2.6), where
until now it blocked.
- At DATA a hold decision queues the message with its release a century
off (the queue's future-release mechanism, so the stored format is
unchanged and an older node just never sends it), transport rules
still applied, and replies 250 Held for review. A review record under
R/h + queue id keeps the sender, recipients, subject, size, rules and
detector counts. The sender is told when the rule asks.
- smtp/queue/held.rs: release (each recipient due now, its next notice
as far off as it was, its lifetime counted from the release), reject
(removed from the queue, the sender told, with the reviewer's note),
and expiry: the daily clean-up rejects what nobody reviewed in 7 days,
recorded as the server's doing.
- inbuxa:HeldMessage get/set: the review queue, sysDlpReviewGet to list
and read (preview, 64 KB of text, only when asked for and recorded as
blobAccess), sysDlpReviewUpdate to release or reject, a reason
required and audited by the request layer; no create or destroy;
server-level only.
- Guards: Emails > Queue refuses to change or delete held mail; the
sender can't unsend it.
- Privacy catalog entry for inbuxa:HeldMessage; spec §2.6 as built.
Tests: mail_rules_tests gains the whole flow (held and listed with
counts, sender notified and nothing delivered, queue and unsend
refused, preview recorded, reject needs a reason and tells the sender
the note, release delivers, expiry returns it, decisions audited with
reasons). smtp inbound, system_tests (after one BlobNotFound in
antispam, the known flake, then clean), features and common unit tests.
Phase 2g of the DLP and mail flow rules spec: transport rules now act,
on outgoing and incoming mail.
- features/mailflow/rewrite.rs: add or remove a header, prefix or set the
subject (an RFC 2047 word when not ASCII), add a disclaimer. A
disclaimer edits the message's main text and HTML bodies only, each
decoded, changed and written back as UTF-8 quoted-printable with its
other headers kept, top or bottom (after <body> or before </body> in
HTML); attachments and attached messages are left alone, and a
disclaimer already present isn't added again.
- smtp/inbound/mailflow.rs: the check runs for incoming mail too
(transport rules only; DLP stays outgoing). After DLP passes, each
matched transport rule's actions run in order: message edits,
add-recipient and redirect (envelope changes DATA applies), route (a
per-message queue ahead of the queue strategy), refuse (550 5.7.1
with the rule's text). The override tag is stripped with the same
subject writer, so a non-ASCII subject stays valid.
- Audit: refusals and changes to where mail goes are recorded (sender,
or system:mail-flow for incoming mail); wording and header changes
aren't, or a banner rule would record every message (spec §2.7).
Tests: rewrite unit tests (headers, encoded subjects, disclaimers on a
single part and on multipart/alternative with an attachment, once
only); mail_rules_tests gains the actions end to end: disclaimer,
header and subject prefix on a delivered message, a redirect, a
refusal, a banner on incoming LMTP mail that outgoing rules leave
alone, and which of those are audited.
Phase 2f of the DLP and mail flow rules spec: the rules now run on mail
an authenticated sender submits, after the DATA system script and
before headers and DKIM signing (§2.1).
- smtp/inbound/mailflow.rs: builds what the rules look at from the
message (subject, the text version of each body, one level of attached
messages, attachment text via the extractor, 10 MB of text at most)
and the envelope (sender's groups and tenant; each recipient local or
not, and its groups). Skipped entirely when no enabled rule applies to
outgoing mail. Rules that can't be loaded refuse with a 451: nothing
unchecked leaves.
- Block: 550 5.7.1 with the rule's notice. Warn: 550 5.7.1 with the
notice and how to override: "[override: reason]" at the start of the
subject, taken out before the message goes on (settled answer 1).
Until phase 3, a hold rule blocks rather than let mail through.
- JMAP: EmailSubmission takes inbuxa:dlpOverride {reason}; a refusal
comes back as inbuxa:dlpWarning or inbuxa:dlpBlocked with each rule's
name and notice (description too, for older clients).
- Audit: one record per DLP match, the sender as actor, action create,
target a message: the recipient domains, each rule with its detectors'
counts, the outcome, an override's reason. Never the matched text. No
new audit action: an older node that meets one fails its daily
clean-up, which would make rolling back unsafe (spec §2.7 updated).
Tests: mail_rules_tests gains the DLP flow over JMAP (no rules, warning
with rule and notice, local recipient not warned, override with a
reason, block that no reason passes, the subject tag stripped from the
delivered message, audit records with no card or key text). smtp
inbound tests pass; system_tests passed twice after one timeout in the
email delivery tests that didn't recur.
Phase 2e of the DLP and mail flow rules spec, the API half.
- inbuxa:MailRule/get and /set under urn:inbuxa:jmap. Rules convert
through serde, so what a client sends is the stored format. A create
or change is validated whole (Rule::validate) and refused with the
property at fault; id, createdBy, createdAt and updatedAt are the
server's. Every change goes through the request layer's audit record.
- Six permissions, ids 674-679 (enum and schema labels): mail flow rules
(sysMailRuleGet/Update), DLP rules (sysDlpPolicyGet/Update) and held
mail (sysDlpReviewGet/Update, for phase 3). Either kind's permission
gets through the gate; the handler shows and changes each rule only
with its own kind's. All server-level: a tenant is refused (settled
answer 3).
- Administrators get all six; the server-level Compliance Officer gets
DLP rules to see and held mail to review (settled answer 4), added
once to an existing server's officer role by the grant mechanism,
which gains an officer audience.
- Privacy catalog entry for inbuxa:MailRule.
tests/src/system/mail_rules.rs: create, list in order, validation,
server-set properties refused, update, kind-separated permissions for
an officer, destroy, audit records.
Personal-data catalog spec, default D5 (settled 2026-09-28; built after
the v0.16.24 import's spam-rules loader landed). msbl.org's EBL is sent
a SHA-1 of every email address it's asked about. A new install's first
boot now leaves a note, and the rules update, once the bundled rules
are in, switches STWT_MSBL_EBL_EMAIL off and forgets the note, so it
happens once; the loader keeps that switch through later updates. An
existing server has no note and keeps every blocklist as it is.
Also fixes the data inventory's DNSBL endpoints: a zone is an
expression (`ip_reverse + '.zen.spamhaus.org'`, conditional branches,
`hash(email, 'sha1') + '.ebl.msbl.org'`), and the zone names are now
the quoted literals that start with a dot, from every branch, rather
than the expression's text.
Tested: unit test for the zone rule; the compliance system test (no
note, no change; the inventory lists ebl.msbl.org, not a hash; with the
note the blocklist goes off; the note works once); the system suite;
fork checks.
Personal-data catalog spec, §6 (Phase 3c).
inbuxa:DataInventory/get evaluates the catalog against the server's
live settings and says what this server holds: for each source and each
object that can hold personal data, its categories and whose data it
is, whether it is collected here at all, what bounds its retention (the
live value of the setting that does, or unbounded), whether it leaves
the host and to which endpoints, and a summary. Every host that
receives something is listed once as a candidate processor with what it
receives. Inside a tenant it answers with the tenant's slice and none
of the server's processors. Read-only, with sysComplianceGet.
inbuxa:InventorySnapshot/get is the history: a dated copy of the
evaluated inventory, recorded when it changes -- after a registry write
to an object the inventory reads, after inbuxa's log, audit or AI
settings change, and on the daily clean-up -- and kept as long as the
audit log's records. ids: null lists every snapshot, newest first; the
full inventory only when asked for.
The catalog is embedded and parsed at start (new dependency: toml,
MIT/Apache); the evaluation is a pure function of it and the live
facts, so each configuration is tested without a server. Loopback
endpoints stay on the host; any other configured endpoint leaves it.
Tested: unit tests for the evaluation (a new install's defaults, an
external blob store, a hosted AI endpoint, telemetry off, a tenant's
slice, hosts from URLs, loopback), snapshots, and the fact gathering's
store and duration rules; the compliance system test, extended (the
officer reads the inventory, a plain user is refused, a tenant's
officer sees its slice and no processors, a webhook to another host
becomes a processor and a snapshot names x:WebHook, a retention change
reads through); the system, audit, legal hold and account lock suites;
fork checks. The system suite failed once of three runs with an email
import's blob not found, in antispam.rs; the same happened once in
purge.rs on the previous branch. Nothing here touches uploads; noted
for a separate look.
The schema conflicted as a binary file: taken from main and the one
edit here re-applied (sysComplianceGet after sysLegalHoldExport). The
import kept the permission count at 673, so the new id stays 673.
Retested on the merged tree in its own target directory: the
compliance and system suites pass. One earlier system run failed in
purge.rs (an imported blob not found) and didn't recur.
Personal-data catalog spec, §7 (settled 2026-09-28).
sysComplianceGet (673) sees the data inventory and compliance
overview: superusers and, for their tenant's slice, tenant
administrators, by default and through the one-time grants on servers
that already have their roles stored.
A Compliance Officer role at server level holds it with reading and
exporting the audit log, placing, widening, releasing and exporting
legal holds, seeing account locks, and reading accounts, lists,
domains, tenants and roles. It changes no server setting, creates or
deletes no account, and can't shorten audit retention.
A tenant's accounts can hold only roles of their own tenant (MT-3), so
the tenant role is one "Compliance Officer" role per tenant, without
holds (LH-13): made once for every tenant a server has, and whenever a
tenant is created. While nobody holds it, it is removed with its tenant
so it doesn't block the delete, and put back if the delete is refused
for another reason. Both roles carry a user's own permissions too,
since roles given to a person replace the default user role, which a
tenant's accounts can't hold anyway.
Every server makes these once, new or existing -- the built-in roles
are only made on a server with none -- and records each under P c, so a
role an administrator deletes stays deleted.
Tested: unit tests (neither role changes a setting beyond a user's
own; holds for the server's officer only; per-place records); a new
compliance system test (one server-level role; an officer reads the
audit log, places and releases a hold, and is refused a setting, an
account and audit retention; a tenant gets its role, whose holder reads
the tenant's audit log and no holds; a tenant with an unused role is
deleted and the role goes with it); the system, audit, legal hold,
account lock and SCIM suites; fork checks. The directory suite needs
its LDAP container and wasn't run here.
Personal-data catalog spec, default D1 (settled 2026-09-28): log files
were never deleted. inbuxa:LogSettings.keepForDays says how many days
rotated log files are kept; unset (null) keeps every file, as before,
and a new install sets 30 days.
It is a fork-owned setting, stored under T + l as audit retention is,
not a field on x:TracerLog: that object is also stored inside
x:Bootstrap with a field after it, so a new field would change
x:Bootstrap's stored format. Server-level, with the tracers'
permissions (sysTracerGet, sysTracerUpdate); changes are in the audit
log, before and after.
Log files are local, so every node deletes its own: hourly, and at once
when the setting changes on that node. Only regular files named
<prefix>.<something> in each enabled log tracer's directory, last
changed more than the limit ago, are removed; the file being written is
never that old, and nothing else in the directory is touched. Minimum
one day. The catalog classifies inbuxa:LogSettings and points the log
file's retention at it.
Tested: unit tests for the file rule (only this log's old files; the
current file, other files and directories stay) and a purge on disk;
the system suite, which reads, sets, refuses zero, restores null and
checks the audit records; fork checks.
The schema, which both sides changed, merged as JSON with no conflicts.
The personal-data catalog (#83) gains upstream's new x:DnsServerPowerDns:
nothing personal but its API key, like the other DNS providers.
Personal-data catalog spec, defaults D2, D3, D4, D6 and D7 (settled
2026-09-28, new installs only):
- D2: automatic IP bans expire after 30 days instead of never; D3:
spam training samples, whole messages, are kept 90 days instead of
180; D4: Pyzor, which sends a digest of each message's text to a
public server, is off; D6: delivery history is kept 14 days instead
of 30. Written on the first boot of a new install only -- one with no
roles yet, the same test the built-in roles use -- by reading each
singleton, setting these fields and writing it back whole. A server
with roles keeps its settings, saved or default.
- D7: a webhook created from now on starts with the include policy and
no events, so it sends nothing until events are chosen (Rust default
and schema default, marked). The registry stores every field, so
existing webhooks keep their policy.
- Expired bans are also removed by the daily data clean-up. They
already stopped blocking and were deleted when settings next loaded;
a server that seldom reloads kept them.
D1 (log retention) and D5 (the hashed-address blocklist off) are held,
and the spec says why: x:TracerLog is stored inside x:Bootstrap with a
field after it, so adding one changes that object's stored format; and
the spam-rules loader D5 touches is being reworked by the v0.16.24
import. The spec also corrects finding 3: expired bans were deleted on
settings load; bans were permanent only because no period is set.
Tested: unit tests for the new-install values and that everything else
in each singleton stays; the system suite, whose security test now
purges an expired ban and checks its record is gone; the telemetry
test; common's unit tests; fork checks.
Eight conflicted files resolved, plus the lock file and the schema:
- crates/services/src/task_manager/spam_classifier.rs: upstream's rules
update now replaces existing rules, DNSBL servers, lookups and file
extensions, keeping only whether each is on. Taken, with one difference:
an object an admin edited is kept as it is. Every object an update writes
is fingerprinted (content without `enable`, SHA-256, stored under
SUBSPACE_INBUXA "Sf"), and only one that still matches is replaced.
Scores are never replaced, as upstream has it. The AU-1.10 summary record
now names what was added, replaced and kept, and the bundled rules are
marked applied only when the update fully succeeded, so a failure runs
again on the next start. The marker becomes "3.0.2+2", which runs the
update once on upgrade to fingerprint every rule still as bundled.
- crates/common/src/network/autoconfig/autodiscover.rs: upstream's rewrite
(implicit TLS first, labeled SSL), with the per-protocol switches (LP-7,
LP-14a) passed in as a filter.
- crates/store/src/backend/mysql/{search,write}.rs: upstream's chunked
deletes (no unbounded first DELETE, stop on a short chunk, halve the
chunk on the new chunk-too-large errors) inside the fork's query timeout.
- crates/smtp/src/lib.rs: the fork's queue spawn kept. It already fixed the
stall upstream fixes here (a node without outboundMta stops accepting
mail at about 1024 queued messages), and follows role changes live.
- crates/jmap/src/registry/mapping/bootstrap.rs: the log path stays
/var/log/inbuxa/; upstream's PowerDNS mapping taken.
- crates/main/Cargo.toml: the AGPL-only license kept, version 0.16.24.
- tests/src/jmap/principal/get.rs: the fork's capabilities kept.
- resources/schema/schema.json.gz: merged as JSON; upstream relabeled the
vendor Sieve extensions "(Stalwart)", kept as "(vnd.inbuxa)".
- Cargo.lock: upstream's, with the fork's crates added by Cargo.
Also:
- tests/src/smtp/inbound/spam_rules_kept.rs: an edited rule survives an
update, an unedited one is updated, rules from before fingerprints are
handled, and the audit summary says so. Upstream's own spam_rules test
passes unchanged.
- tests/src/smtp/reporting/reschedule.rs moves to port 19058; upstream's
new spam_rules test took 19057.
- tools/fork/renames.py renames the "(Stalwart)" labels and the default
log path, so neither conflicts again.
- tools/fork/notice-check.py compares against the newest snapshot in the
checked-out history instead of the upstream branch head, so moving the
branch no longer fails other open pull requests.
- tests/src/directory/issuer.rs (since v0.16.23) stays out, and is on the
build check's known list: it tests issuer-based directory routing, which
the fork doesn't have (DIR-2).
- Strip report: docs/fork/strip-reports/v0.16.24.{md,json}.
Upstream commit: af37a234981722493b74623a983581691d2b70b6
Enterprise-only files removed or emptied: 63
Enterprise-only snippets removed: 118 in 50 files
Dangling module declarations removed: 5
Edits turning enterprise off: 25
Third-party code: 14 files, 0 not in THIRD-PARTY.md
Renamed identifiers: 62 in 18 files
Verification: clean
The same Enterprise footprint as v0.16.23. The build check fails only on
tests/src/directory/issuer.rs, unchanged since v0.16.23: it calls a helper
from upstream's Enterprise-only OIDC test, and tests issuer-based directory
routing, an Enterprise feature. main has never carried it.
An item the hold covers whose stored record or content can't be read
goes in exceptions.csv with the path it would have had and the reason,
rather than being left out silently. The file is always in the ZIP, so a
header-only one shows nothing was missed, and manifest.sha256 carries
its hash beside the manifest's.
inbuxa:HoldExport/set takes a hold, optionally some of the accounts it
covers, and a reason; the collection runs in the background and get
says when it's ready. The ZIP has, per account, mail as .eml under its
folders, calendars as .ics, contacts as .vcf, files as stored, and the
archived items the hold keeps under archived/; a manifest.csv gives each
entry's account, kind, folder, date, whether it was archived, size and
SHA-256, and manifest.sha256 hashes the manifest. Accounts the hold
doesn't cover are left out, and items outside its date range are too:
live mail by arrival, events by start, and archived items the same way,
so an export doesn't carry deleted items that only another hold keeps.
The finished file is a blob of whoever started the export, so only they
download it, and it lasts as long as any upload (uploadTtl). Exports
are records under the hold (SUBSPACE_INBUXA H/e): never changed or
destroyed, each with its status, counts, size and checksum. Starting
one needs sysLegalHoldExport, an active hold and a reason, and is
recorded in the audit log like the audit log's own export.
The build is in memory and capped at 2 GB; bigger holds fail with a
message saying so, and are split by picking accounts.
Tested: unit tests for safe ZIP names and the manifest and its hash;
the legal_hold system test, on RocksDB, PostgreSQL and MySQL, exports a
hold end to end (live and archived mail, the manifest's hash, an asked-
for account the hold doesn't cover left out) and checks the refusals
(no reason, a user without the permission, a released hold) and the
audit record; and by hand from the console on a local server. Not
covered by a test: the archived-item date range with two holds of
different ranges over one account.
An account's or mailing list's name is only its local part, so the log
said "Account ken.gosling" where two domains could each have one; it
now says [email protected]. A change to a legal hold was
recorded under its id; the hold's current state is now read first, so
the record carries its case name and each change reads before/after.
inbuxa:LegalHold/get answers accountsCovered, itemsHeld and sizeHeld
when asked: the accounts a hold reaches now (deleted ones it keeps
included) and the archived items it keeps, with their size. Worked out
in one pass over accounts and archive, only for requests that name them.
Held items stay out of the user's quota, as all archived copies do
(LH-9).
Destroying a held account removes the login, as offboarding needs, but
keeps its data as a deleted account with no expiry, whether or not
undelete keeps accounts; its addresses stay reserved and its holds name
it from then on. Destroy-now refuses it, and its DestroyAccount task
defers itself while it's held or its time hasn't come. Holds placed or
released later freeze or free kept accounts in the same settle pass,
with 30 days' grace after the last release (LH-8, LH-10).
Placing or widening a hold freezes what's already archived in its scope
and range, its old deadline noted; releasing one gives each item no other
hold covers that deadline back, or release plus 30 days if later. One
pass over the archive does both and changes nothing twice (LH-6, LH-10,
LH-11). A held archived item can't be destroyed; restoring still can,
and the hold is named only to callers who may see holds (LH-7). Audit
records about a held account survive the purge (AU-7).
Fixes the daily clean-up of expired archived items (UD-13), which never
found any: the registry's unfiltered query reads an all-ids index that
archived items aren't in. Items are now walked account by account, kept
deleted accounts included. Expired items were still removed whenever
their account's archive was read.
Every way of deleting mail (JMAP, IMAP EXPUNGE, POP3, mailbox removal,
Trash emptying) and Sieve scripts, events, contacts and files now asks
how the account's deletions are kept: a hold keeps them with no expiry
(archivedUntil 9999-12-31), even with undelete off; otherwise undelete's
period applies as before (LH-4).
A hold's date range decides by the item's own date (LH-3). Mail is noted
as held at deletion and settled when it's archived, once its received
date is known; outside the range it gets undelete's deadline or isn't
kept. Events go by their start, with a day's slack for time zones;
recurring events, contacts, files and scripts are held whole.
A groupware item's note now stays until its archive succeeds, and a
failure retries the task instead of being logged and lost (LH-5).
A hold reaches an account by name, through any of its addresses'
domains, its groups or its tenant, as they are now, so an account added
to a held domain later is held too. An account that leaves a held
domain, group or tenant stays held: the registry write hook adds it to
the hold by name on every account change, whoever makes it (LH-2).
Server::holds_on answers for the deletion paths, from the store each
time so a hold binds every node at once.
inbuxa:LegalHold get/set places a hold on accounts, groups, domains,
tenants or the whole server, with an optional date range. A hold's range
and scope can only widen, a released hold is read-only, and none is ever
deleted. Placing, changing and releasing each need a reason and are
audited (LH-1, LH-3, LH-10, AU-12).
Permissions 669-672 (see, place, widen or release, export held data)
go to server administrators only; the tenant ceiling always strips them,
as it does Impersonate (LH-13). Schema: Compliance > Legal Holds.
What a hold keeps comes next, through the undelete hooks.
Also moves the lock expiry helpers below the lock module's imports.
A shared account refuses top-level folders, so an organize or full
delegate couldn't add anything to a locked account with no folders. A
delegate who may write now can, as the owner could; the reconcile after
the create grants it the new folder. Read delegates still can't (AL-6,
AL-7).
A delegate's token listed the locked account only for kinds of data it
held grants on, so one with no files (or no calendar) was refused to the
delegate outright: "You do not have access to account". The token now
lists the locked account for mail, calendars, contacts and files alike,
so an empty kind reads as empty. What the delegate may see or change is
still each container's grant (AL-7).
A delegation with an end date dropped out of the delegate's token then,
but its folder grants stayed until the daily sweep, so the delegate kept
the account as an ordinary share for up to a day. Each node now sleeps
until the soonest end date, woken early by any lock write and at least
hourly, and re-applies that lock under a cluster-wide claim.
The sweep also had a second-run bug: a delegation past its date gave the
delegate back its earlier share, then dropped the note, so the next sweep
removed that share entirely. The note is now kept while the delegate is
still listed.
A locked account can't sign in (it fails as a wrong password does), its
sessions end on every node, refresh tokens stop working, and its Sieve
scripts forward and reply to nothing. Mail keeps arriving.
Delegates get real ACL grants on the account's mailboxes, calendars,
address books and files at read, organize or full, with the rights they
replaced restored on unlock. Folders made later are granted after the
create and in a daily sweep. Organize delegates can't destroy; send-as
needs organize or full. The JMAP session marks delegated accounts in
urn:inbuxa:jmap.
New inbuxa:AccountLock object with get/set, permissions 665-668, and a
Compliance > Locked Accounts entry in the schema. Lock, unlock and
delegate changes need a reason and are audited; delegate access and
writes are audited too (audit-hold-lock spec AL-1 to AL-12).
What administrators and the server itself do to the control plane is now
recorded, from inbuxa-drafts/specs/audit-hold-lock.md (AU-1 to AU-12):
settings, accounts, domains, roles and every other registry change, with
each field's before and after (secrets only as "changed"); the fork's own
settings objects; administrator sign-ins (and failed ones to administrator
accounts), master-user and recovery-admin sign-ins, once an hour per
account, method and address; access to another account's data through
impersonation or FetchAnyBlob, once an hour; exports and tamper checks;
and registry writes the server makes on its own, named by subsystem
(system:AcmeRenewal, system:auto-ban, system:directory-sync, ...), with a
spam rules update as one summary record.
No change without its record (AU-3): before a set method changes anything,
a pending record per requested create, update and destroy is written; if
that fails, the method is refused with serverFail. Its outcome follows as
a later entry. A change interrupted by a crash stays "unfinished".
Records live in the fork's subspace under L, as one SHA-256 hash chain per
node. The chain's head is stored, never cached, and every append asserts
it, so two writers can't take the same place. Nothing can edit or delete
a record; the daily purge removes the oldest past the retention (default
730 days, minimum 90) and records where the chain now starts, so
verification still passes. security.audit-recorded (647) copies each
record to webhooks, OpenTelemetry and the log; security.audit-write-failed
(648) reports a failed write.
New JMAP objects under urn:inbuxa:jmap: inbuxa:AuditEvent/get and /query
(filters: time, actor, action, target, account, tenant, outcome, address,
text), inbuxa:AuditSettings, inbuxa:AuditExport (CSV or JSON Lines built
on the server, each line with its chain hash, ending in a manifest; the
created object names the blob and its SHA-256) and
inbuxa:AuditVerification. New permissions sysAuditGet, sysAuditExport and
sysAuditSettingsUpdate: the Administrator role gets all three, the Tenant
Administrator role gets read and export, once, on existing installs too.
A tenant administrator sees records whose actor or target is in its
tenant, including a server administrator's changes there.
Sign-in method on the session: access tokens now remember how they signed
in (password, app password, API key, OAuth client, directory, master user,
recovery admin), including across the HTTP credential cache. New OAuth
access tokens carry their client id in the sealed claims; older ones show
as client "unknown" until they expire.
The schema gains the permissions, the two events and a Management >
Compliance > Audit Log link.
Stack: the request layer boxes every inner future where it's made. Without
that, a debug build overflowed the default 2 MB worker stack on a registry
set; measured with the same request, the branch and main now overflow at
the same stack size (between 1856 and 1920 KiB, debug), so the layer adds
nothing measurable.
Tests: unit tests in inbuxa-features and jmap; system::audit::audit_log_tests
(run with --ignored) passes on RocksDB, SQLite, PostgreSQL, PostgreSQL with a
read replica, MySQL, MySQL with a replica and FoundationDB. The system, JMAP
and SCIM suites pass. authorization.rs skipped fork permissions that guard
no registry object; the audit suite checks a plain user is refused instead.
A singleton such as x:SpamSettings has no stored object until someone
saves it; /get shows its defaults instead. Explain looked only for the
stored object, so every setting still at its defaults answered "No
such x:SpamSettings." It now falls back to the defaults the same way.
A new method, inbuxa:Explanation/set, asks the node's local model for a
short plain-words reading of one thing an administrator is looking at:
a failed recipient in the queue, a Classify verdict, a log line or trace
event, or one setting with its saved value. The server builds the prompt
itself from stored data and the registry schema, never from text the
console sends, and grounds SMTP replies in RFC 3463 and RFC 5321.
What the model is never shown: secrets (including ones nested inside a
setting, like an AI model's HTTP auth), raw protocol events, and the
contents of any other event. A tag name that doesn't have a tag's shape is
refused before a model is asked.
Calls share the AI gate with spam classification, but mail always keeps
its slot, and Explain has its own hourly count per account and its own
on/off switch in inbuxa:AiLimits. The permission is sysAiExplain,
superuser only; tenant administrators can't use it. The session carries
an aiExplain flag so a console knows when to offer the button.
An install whose roles were stored before the permission existed gets it
added once, at start-up, to the roles that are administrators' alone,
not the User role their defaults share with every account. An operator
who removes it later isn't overruled.
Tests: unit tests in inbuxa-features and jmap, and ai_explain_tests
(run with --ignored) covering the acceptance tests and the upgrade.
A cluster rehearsal sent ten x:<Object>/set requests at once and got
ten full reloads on every node. #39's coalescing only joined writes
that queued behind a running reload, but the requests reached the
server about 33 ms apart and a reload takes tens of milliseconds, so
none overlapped one.
A full reload after a registry write now waits for writes to settle:
75 ms after the last one, and at most 250 ms after the first it
covers, so a steady stream still reloads at least four times a
second. 75 ms is a little over twice the gap the rehearsal saw between
requests. A single write pays it once: in the tests a settings write
takes about 140 ms instead of 60. The reload runs in a task of its
own, so a request that goes away doesn't cancel it for the others.
Each write takes the result of the first reload that started after it
was stored (the gate keeps the last 64 results), so applied true or
false still describes the reload that covered that write.
The 33 ms gap was a queue on the server, not password hashing: Basic
credentials are cached per Authorization header, so they are checked
once. Every authenticated HTTP request counted itself against the
account's rate limit by incrementing one counter per account in the
in-memory store, so parallel requests from one account queued on that
key: a row lock on PostgreSQL (a few round trips to the database
each) and conflict retries with a 50-300 ms backoff on RocksDB. An
account with the unlimitedRequests permission (administrators, by
default) passes the rate and concurrency limits anyway, so its
requests are no longer counted. Ten parallel Core/echo calls as the
admin now finish in 1-4 ms; before, they finished one after another
over 20 ms on a local PostgreSQL and 300-450 ms on RocksDB. Other
accounts still count every request.
system::auto_reload::settings_reload_tests: ten concurrent writes now
take one reload (the gate counts them; at most two allowed), all are
applied: true and in the running settings, and a single write takes
exactly one reload. RocksDB and PostgreSQL, 1 reload in 141-196 ms.
With the old behavior (no wait, requests counted) the same writes
took 5 reloads; without the wait but with the rate fix, 2.
cluster::broadcast (3 nodes, PostgreSQL + NATS) and system::reload
still pass.
A cluster rehearsal moved a Log tracer to another directory: the write
was reported x:settingsReload applied:true, but the tracer kept writing
to the old file until a restart. Telemetry::update only refreshed each
running tracer's events, level and lossiness; a tracer's own settings
(path, prefix, rotation, format, endpoint, headers, ...) stayed as built.
Each tracer now carries a hash of the registry object it was built
from, less the fields that change in place. The reload compares it with
the running tracer's: unchanged ones are updated in place as before,
changed ones are started over, new ones started and removed ones
stopped. Only tracers this server started are removed; upstream removed
every subscriber not in the settings, which also cut off live-tracing
streams on each reload.
Starting over is a swap in the collector, so no event is lost or
written twice: a subscriber registered under a running one's id
replaces it between two collection passes. The old one's batch is sent
first (what its full channel can't take moves to the new one), and
dropping it closes its channel, so its task writes what is queued and
ends. Per tracer kind:
- Log: a tracer started over on the same files (rotation or format
changed) waits for the old one to finish, so lines don't interleave.
- Webhook: the task held a sender of its own channel for retries, so
it never ended; retries now use a weak sender, and pending events are
posted when the channel closes.
- OpenTelemetry: pending logs and spans are exported when the channel
closes instead of dropped, and a span that was open across the swap
is exported by the new tracer with the events it saw.
- Console and journal: nothing kept between batches.
- Trace history: built from the tracing store, which takes a restart,
so it is never started over.
No kind needs a restart, so x:settingsReload doesn't gain one.
system::tracer_reload::tracer_reload_tests (new): a Log tracer created
over JMAP writes to its directory; its path is changed over JMAP while
2000 numbered events are emitted; after the reload, events land in the
new file and not the old one, each numbered event is in exactly one of
the two files, and a destroyed tracer writes nothing. On main the new
file never appears.
write_reload_target sent AllowedIp writes to the blocked-IP reload, but
that reload rebuilds only BlockedIps. Allowed IPs are parsed into the
core's security settings (Security::parse), which only a full reload
rebuilds, so an AllowedIp write reported x:settingsReload applied: true
while the change wasn't live until the next full reload.
AllowedIp now maps to the full reload, like the other settings objects;
BlockedIp keeps its targeted reload.
system::auto_reload::settings_reload_tests now creates an allowed IP
over JMAP and checks that is_ip_allowed sees it with no ReloadSettings,
and that destroying it takes it out again. On main it fails ("allowed
IP not in the running settings").
A 3-node rehearsal found that saving an MtaDeliverySchedule left it
unknown to the queue ("Queue strategy not found") until someone ran
x:Action ReloadSettings; only Directory and Authentication writes
reloaded (DIR-17). The admin UI has to remember a separate reload after
every save, and a script or API client that doesn't gets a server
running stale settings.
x:<Object>/set now reloads the running settings when it created,
updated or destroyed an object they are built from, and broadcasts the
same RegistryChange::Reload over the coordinator as ReloadSettings, so
every node applies it:
- Settings objects (MTA, spam filter, listeners, tracers, Sieve system
scripts, cluster roles, directories, ...: the object types the core,
telemetry, listener and directory builders read) get a full reload.
- Certificates, lookup stores and blocked/allowed IPs get their own
targeted reloads.
- Accounts, domains, roles and other data read as needed, stores (they
take a restart) and applications (their own reload action) get none.
Full reloads are coalesced: a write waits for a reload that started
after it was stored and joins one if it can, so a burst of writes, or
a request with many objects, costs one or two reloads, not one each.
The write itself is never undone. When the reload is refused (build
errors in objects that were working, the rule from the previous
commit), the set response says so in a new x:settingsReload field,
{"applied": false, "description": "Saved, but the running settings
were not reloaded. <object>: <error>"}; {"applied": true} otherwise.
The field is absent when the write needs no reload. The description
helper is shared with ReloadSettings' refusal.
Each reload sends the queue a ReloadSettings event, so the SMTP test
harness's read_event, try_read_event and assert_no_events now pass over
those; expect_reload_settings still waits for one.
system::auto_reload::settings_reload_tests (new): an MtaVirtualQueue
and an MtaDeliverySchedule created over JMAP are in the running
settings with no ReloadSettings, and gone once destroyed; eight
concurrent creates all land; a write whose reload fails is stored and
reported applied: false with the error; a domain write carries no
x:settingsReload. On main the new schedule is missing. The cluster
broadcast test (three nodes, PostgreSQL + NATS) now checks that every
node has a schedule created on node 0 without a reload.
A 3-node rehearsal found every settings reload refused, cluster-wide,
because one node couldn't resolve the Pyzor server:
- PyzorConfig::parse resolved the host while building the settings and
made a failed lookup a build error. It now keeps the host and port and
resolves when a message is checked (an IP address is used as is, a
name is reused for five minutes, the lookup counts against the Pyzor
timeout). A failure there is a Pyzor error for that message.
- A milter's hostname was resolved the same way, with a blocking
to_socket_addrs in async code. An IP address is kept; a name is now
resolved on each connection.
Other build-time I/O is already non-fatal: directories that can't
connect become unavailable with a warning (DIR-21), and the AI model
locality check only warns.
reload_registry swapped the core only when the whole build was free of
errors, while boot runs with whatever built. One failing object thus
refused every later reload, and the running settings went stale. Now a
reload is refused only for errors in objects that built when the
running settings were built (at boot or by the last applied reload):
applying it would lose those. Objects that already failed then are
missing from the running settings anyway, as at boot, so their errors
are logged and returned as known_errors but don't hold the reload back.
Refusing on new errors keeps a bad edit from taking a working object
out of service; the admin gets the error instead.
ReloadSettings now says "Settings were not reloaded." and names the
object and its error ("Tracer with id ...: Only one console tracer is
allowed"), with a count of any further errors. A refused reload after a
directory change logs its errors too.
system::reload::reload_tests (new): with Pyzor enabled on an
unresolvable host, ReloadSettings succeeds (on main it fails with
"Invalid address: failed to lookup address information"); an IP host
needs no lookup; a new build error refuses the reload, names the object
and leaves the running settings unchanged; the same error, once known
from the running settings' build, no longer blocks; once fixed, a new
error there blocks again. smtp::inbound::milter's session test now
names its milter "localhost", so the connect-time lookup is exercised.
The trace index task wrote the event type (its name) and the queue id as
text, but the tracing search index types both as integers on every
backend: BIGINT on PostgreSQL and MySQL, long on Elasticsearch. On
PostgreSQL every batch holding a trace document failed with "cannot
convert between the Rust type String and the Postgres type int8", and
since a batch writes trace and email documents together, email indexing
stalled behind it.
The document is now built by trace_search_document(), which writes:
- the event type as the opening event's numeric id, the event
x:Trace/query's event filter already matches on;
- the queue id as an integer, the first one the trace names;
- every queue id into the keywords as well, since the column holds one
value and an SMTP session can queue several messages.
index_keyword() replaced the field on every call, so before this only the
last event type and queue id survived anyway.
x:Trace/query's queueId filter parses the id (a string, or now a number)
and matches the column or the keywords, so a session is found by any of
its queue ids on every backend. The monitoring spec says what is indexed.
Traces indexed before this on the built-in index keep their text values;
the reindexTelemetry maintenance task rebuilds them.
Tests: the search store suite builds trace documents with the index
task's code, indexes them and finds them by queue id, event type and
keyword (Sqlite, PostgreSQL, MySQL); the monitoring suite finds a real
trace by queueId through x:Trace/query.
The name is inbuxa, lowercase, like the wordmark; INBUXA reads as an
acronym. The admin and webmail already changed. Here that's everything
the server shows people: the brand macro behind the protocol greetings,
the HTTP and SCIM realms, the startup banner and the calendar and contact
PRODID; the first-party OAuth client descriptions; the legacy-protocol
refusals; the default calendar and address book names and the SMTP
greeting default, in the code and the schema served to the admin
(checksum regenerated); startup and shutdown events; the User-Agent;
the sign-in and RSVP pages; the service units; the OpenAPI realm; the
crate descriptions and the README, where it's set in bold.
Identifiers that are uppercase for their own reasons stay: INBUXA_*
settings, SUBSPACE_INBUXA. So do code comments and the AGPL 5(a) notice
lines.
Tests follow: the IMAP ID name, the default collection names, the PRODID
in the iTIP fixtures and the CalDAV free-busy expectations, and the e2e
legacy-protocol refusals. The webdav, imap and jmap suites pass, so do
the unit tests of every crate touched, and 73 of 75 SMTP tests; of the
other two, antispam fails on main too, and queue_retry is a timing flake
that passes on its own.
Everything clients, users and operators meet now carries the fork's name,
with no aliases (SPEC.md §2.4, changed here from "protocol identifiers
stay"):
- JMAP: upstream's registry capability is urn:inbuxa:jmap:registry, beside
the fork's own urn:inbuxa:jmap.
- WebDAV lock and sync tokens are urn:inbuxa:dav*; clients resync once.
- Sieve: vnd.inbuxa.while and vnd.inbuxa.expressions. sieve-rs spells these
into its compiler, so it's vendored (vendor/sieve-rs, 0.7.3) and patched in;
a unit test fails if Cargo.lock ever moves past the vendored copy. The
trusted runtime now names itself too, rather than answering sieve-rs's
default.
- The web interface's OAuth client is inbuxa-webui. On every start the old
stalwart-webui client is removed and any application naming it is moved
over.
- The spam filter's blobs are INBUXA_SPAM_*; every start moves any left
under the old keys, so a trained model survives.
- SQL stores and log files default to inbuxa, in the code and in the
schema served to the admin (checksum regenerated).
- Settings are INBUXA_* only. A STALWART_* variable that's set where its
INBUXA_* one isn't stops the server at startup, naming it.
- The version-upgrade messages link docs.inbuxa.org's migration page, and
the OpenAPI description, smtp crate metadata and web-push test fixtures
lose the name.
Kept on purpose, allowlisted with reasons: the OAuth key-derivation
contexts (renaming them would end every session and invalidate every
sealed client id) and the hashed application prefix.
Also fixes a latent start-up failure: ensure_client updated an existing
first-party client with a revision of 0, which the registry's assertion
never matches, so adding a redirect URI or changing the webmail secret
failed start-up. And the principal session test now expects
legacyProtocols (C-1, added 2026-09-21), which it had missed.
Tested: the server builds without warnings; common's 106 unit tests,
including the vendoring check; a new integration test for the two
start-up migrations; and the webdav, jmap, imap and SMTP Sieve suites.
Five conflicts, resolved:
- crates/common/src/auth/authentication.rs: upstream's get_directory_for_token
and JwtClaims replace extract_jwt_domain; the per-domain directory code
(DIR-1, DIR-5 to DIR-7) is kept, and the token lookup routes through it.
The release's one new Enterprise snippet was the body of
get_directory_for_issuer, which stays returning None: a token naming no
address gets the server default, as DIR-2 specifies and as v0.16.22 did.
- crates/common/src/manager/application.rs: upstream's rewrite of the tests,
with the temp directory names renamed again, and the 5(a) notice the
name-purge change should have added.
- crates/common/src/network/mta.rs: both sides' imports.
- crates/main/Cargo.toml: the AGPL-only license kept, version 0.16.23.
- Cargo.lock: upstream's, with the fork's crates added by Cargo.
jcoffey-dev