jcoffey-dev is traveling from Thursday 1 October through Sunday 4 October. Issues and pull requests are welcome, and will get an answer after that. Thanks for your patience.
Phase 4 of the journaling spec.
- inbuxa:JournalEntry/query and /get (sysJournalSearch): filter by time,
sender, recipient, either, direction, subject words, Message-ID and
journal, newest first; the whole report only when asked for.
- inbuxa:JournalExport/set (sysJournalExport): a reason is required; a
ZIP of the matching reports with manifest.csv, exceptions.csv and
manifest.sha256, up to 10,000 reports and 1 GB.
- inbuxa:JournalVerification/set (sysJournalGet): chains and reports
rechecked.
- Every search, listing, read, export and check is written to the audit
log before anything is returned, with existing actions only.
- Catalog entries for the three objects; spec as-built notes.
journal_tests: administrators can't search; a Compliance Officer searches,
lists, reads a report, exports (reason required) and checks the chain;
the officer can't change journals; each of those is in the audit log.
Phase 3 of the journaling spec.
- A journal's destination: builtIn (true for journals stored before) and
archiveAddress, at least one. Reports to an archive are queued from the
empty sender, one per address, flagged so they're never journaled.
- A pending record per report. When the queue lets go of one without
delivering it (refused, expired, deleted), it becomes its own entry in
the built-in journal under the sending journals' retention, the
journal's archiveFailures (count, last time, reason) goes up, and the
audit log records it; if that can't be written it stays queued.
- Journal it: a rule action naming a journal, on mail flow rules and
beside a DLP rule's block, warn or hold. A journal whose scope chooses
nobody takes only what rules send it.
- The report lists recipients a rule added or redirected to under
"Added by rule", by rule name.
- A rule's route is cleared between messages in one SMTP session, with the
new journal marks; a second message used to keep the first one's route.
tests/src/system/journal.rs: destination validation, a rule-only journal
fed by a rule that also adds a recipient, an unreachable archive's report
kept in the built-in journal with the failure counted, a report delivered
to an archive here and not journaled itself.
Phase 2 of the journaling spec.
- A copy of each message is taken in MessageWrapper::queue, after DLP and
transport rules, for every enabled journal that takes it (direction and
scope: everyone, or accounts, groups, domains, tenants). If the copy
can't be taken the message isn't queued (temporary failure).
- The journal report: the envelope one field a line (sender, To, Cc, Bcc
from the envelope, list members from their ORCPT, direction, held for
review), then the queued message byte for byte as message/rfc822.
- The built-in journal under J in the inbuxa subspace: one chain per node
whose links name each entry by SHA-256, so entries can expire out of
chain order; purge leaves a marker, and verify catches an entry changed
or removed early and a report that doesn't match.
- Retention per journal (30 to 3650 days); an entry keeps what it was
written with. The daily maintenance purges what's due, keeping entries
whose people a legal hold covers (deleted accounts a hold keeps too),
and records the counts in the audit log.
- inbuxa:Journal get/set, audited by the request layer. Permissions
680-683: administrators see and change journals; the Compliance Officer
sees, searches and exports. Whoever changes journals may grant search and
export without holding them, so officers can still be appointed.
- Catalog entries (inbuxa:Journal, source "journal"); spec as-built notes.
tests/src/system/journal.rs: validation, internal mail with a Bcc,
outgoing into two journals, incoming over LMTP, the report and its
original, tamper and early removal caught, hold-aware purge, retention
changes leave entries alone, disabled and removed journals take nothing.
senderGroup, senderTenant and recipientGroup conditions now read and write group and tenant ids as JMAP ids ("b", "c"…), like legal hold scopes and the rest of the API, so the console can use its object pickers; plain numbers are still read. Held as numbers for matching. Unit test for both forms and a bad id; mail_rules_tests round-trips a tenant condition over JMAP.
inbuxa:DlpSettings (singleton, urn:inbuxa:jmap): keepHeldDays, 1 to 90,
7 by default (settled answer 5 made it a setting). sysDlpPolicyGet reads
it, sysDlpPolicyUpdate changes it, server-level, audited by the request
layer. Each held message keeps the days it was given, and the sender's
notices say that number. Privacy catalog entry; spec §2.6 updated.
mail_rules_tests: 7 by default, 0 refused, 3 set and a message held
afterwards expires 3 days after it was held, the expiry notice says 3.
The hold action now holds (dlp-and-mail-flow-rules spec, §2.6), where
until now it blocked.
- At DATA a hold decision queues the message with its release a century
off (the queue's future-release mechanism, so the stored format is
unchanged and an older node just never sends it), transport rules
still applied, and replies 250 Held for review. A review record under
R/h + queue id keeps the sender, recipients, subject, size, rules and
detector counts. The sender is told when the rule asks.
- smtp/queue/held.rs: release (each recipient due now, its next notice
as far off as it was, its lifetime counted from the release), reject
(removed from the queue, the sender told, with the reviewer's note),
and expiry: the daily clean-up rejects what nobody reviewed in 7 days,
recorded as the server's doing.
- inbuxa:HeldMessage get/set: the review queue, sysDlpReviewGet to list
and read (preview, 64 KB of text, only when asked for and recorded as
blobAccess), sysDlpReviewUpdate to release or reject, a reason
required and audited by the request layer; no create or destroy;
server-level only.
- Guards: Emails > Queue refuses to change or delete held mail; the
sender can't unsend it.
- Privacy catalog entry for inbuxa:HeldMessage; spec §2.6 as built.
Tests: mail_rules_tests gains the whole flow (held and listed with
counts, sender notified and nothing delivered, queue and unsend
refused, preview recorded, reject needs a reason and tells the sender
the note, release delivers, expiry returns it, decisions audited with
reasons). smtp inbound, system_tests (after one BlobNotFound in
antispam, the known flake, then clean), features and common unit tests.
Phase 2g of the DLP and mail flow rules spec: transport rules now act,
on outgoing and incoming mail.
- features/mailflow/rewrite.rs: add or remove a header, prefix or set the
subject (an RFC 2047 word when not ASCII), add a disclaimer. A
disclaimer edits the message's main text and HTML bodies only, each
decoded, changed and written back as UTF-8 quoted-printable with its
other headers kept, top or bottom (after <body> or before </body> in
HTML); attachments and attached messages are left alone, and a
disclaimer already present isn't added again.
- smtp/inbound/mailflow.rs: the check runs for incoming mail too
(transport rules only; DLP stays outgoing). After DLP passes, each
matched transport rule's actions run in order: message edits,
add-recipient and redirect (envelope changes DATA applies), route (a
per-message queue ahead of the queue strategy), refuse (550 5.7.1
with the rule's text). The override tag is stripped with the same
subject writer, so a non-ASCII subject stays valid.
- Audit: refusals and changes to where mail goes are recorded (sender,
or system:mail-flow for incoming mail); wording and header changes
aren't, or a banner rule would record every message (spec §2.7).
Tests: rewrite unit tests (headers, encoded subjects, disclaimers on a
single part and on multipart/alternative with an attachment, once
only); mail_rules_tests gains the actions end to end: disclaimer,
header and subject prefix on a delivered message, a redirect, a
refusal, a banner on incoming LMTP mail that outgoing rules leave
alone, and which of those are audited.
Phase 2f of the DLP and mail flow rules spec: the rules now run on mail
an authenticated sender submits, after the DATA system script and
before headers and DKIM signing (§2.1).
- smtp/inbound/mailflow.rs: builds what the rules look at from the
message (subject, the text version of each body, one level of attached
messages, attachment text via the extractor, 10 MB of text at most)
and the envelope (sender's groups and tenant; each recipient local or
not, and its groups). Skipped entirely when no enabled rule applies to
outgoing mail. Rules that can't be loaded refuse with a 451: nothing
unchecked leaves.
- Block: 550 5.7.1 with the rule's notice. Warn: 550 5.7.1 with the
notice and how to override: "[override: reason]" at the start of the
subject, taken out before the message goes on (settled answer 1).
Until phase 3, a hold rule blocks rather than let mail through.
- JMAP: EmailSubmission takes inbuxa:dlpOverride {reason}; a refusal
comes back as inbuxa:dlpWarning or inbuxa:dlpBlocked with each rule's
name and notice (description too, for older clients).
- Audit: one record per DLP match, the sender as actor, action create,
target a message: the recipient domains, each rule with its detectors'
counts, the outcome, an override's reason. Never the matched text. No
new audit action: an older node that meets one fails its daily
clean-up, which would make rolling back unsafe (spec §2.7 updated).
Tests: mail_rules_tests gains the DLP flow over JMAP (no rules, warning
with rule and notice, local recipient not warned, override with a
reason, block that no reason passes, the subject tag stripped from the
delivered message, audit records with no card or key text). smtp
inbound tests pass; system_tests passed twice after one timeout in the
email delivery tests that didn't recur.
Phase 2e of the DLP and mail flow rules spec, in the features crate.
- rules.rs: a rule (§2.2) with its conditions (§2.3) and actions (§2.4),
as JSON under R/r in the fork's subspace. validate() enforces the
spec's shape: DLP rules check outgoing mail and have exactly one of
block, warn or hold; transport rules have neither those nor
detectors; lists, header names, header values (one line), addresses,
texts, word lists, patterns and detector ids are checked.
- engine.rs: rules compiled once (word lists to automata, patterns to
size-limited regexes) and run in priority order with exceptions and
stop processing. Each detector runs at most once per message and
only when a rule asks for it. The outcome lists what matched with
each detector's count, and decides DLP strictest first: block, hold,
warn; an override answers warnings only (§2.5).
- cache.rs: each node's compiled copy, refreshed after 30 seconds or at
once when this node changes a rule.
Nothing calls this yet: the JMAP object and the check at DATA follow.
55 unit tests in mailflow.
Phase 2b of the DLP and mail flow rules spec: every identifier in the
§2.3 catalog, each implemented from its issuer's published rules and
tested against published examples.
US (SSN, ITIN, EIN, ABA routing, driver's licenses, MBI, NPI, DEA), UK
(NI number, NHS number, UTR), Canada (SIN), Australia (TFN, Medicare),
the EU (Germany's tax ID and ID card, France's NIR, Spain's DNI/NIE,
Italy's codice fiscale, the Dutch BSN, Belgium's national number,
Poland's PESEL, Sweden's personnummer, Denmark's CPR, Finland's HETU,
Ireland's PPS, Portugal's NIF, Austria's SVNR), Norway, Switzerland,
India (Aadhaar, PAN), China, Japan, Singapore, South Korea, Brazil (CPF,
CNPJ), Mexico (CURP) and South Africa. 49 detectors in all, plus seven
templates named for what they find.
An identifier that is only digits and whose check about one random
number in ten passes counts alone only in its written form
(536-22-1234, 943 476 5919) and as bare digits only beside a word; ABA
routing numbers and NPIs always need one. Spec §2.3 records this.
A test runs every detector over an ordinary business email (order,
invoice and tracking numbers, dates, amounts, an address) and requires
nothing to fire but the contact detectors. 47 unit tests.
Phase 2a of the DLP and mail flow rules spec: pure functions in
crates/features/src/mailflow, nothing wired into the mail path yet.
- Detectors report distinct values found, each either checked by its
published check digit or counted only beside a corroborating word
within 50 characters. This PR adds the region-free ones: payment
cards (issuer prefixes, Luhn), IBAN (registry lengths, mod 97),
SWIFT/BIC, email addresses and phone numbers in bulk, dates of birth,
passport numbers, private keys and published service-token formats.
Regional identifiers follow, a region per PR.
- Word lists (Aho-Corasick, whole words, any case) and patterns (regex
with a compiled-size limit) count occurrences.
- Attachment text: text files with or without a UTF-16 mark, HTML,
DOCX/XLSX/PPTX, ODT/ODS/ODP and ZIP archives one level deep, read
with the zip and quick-xml crates the workspace already has.
Encrypted files, PDF, legacy binary Office files, nested archives
and anything past the limits come back as not inspectable, with why.
21 unit tests, against the networks' test card numbers and the IBAN
registry's own examples among others.
Personal-data catalog spec, §6 (Phase 3c).
inbuxa:DataInventory/get evaluates the catalog against the server's
live settings and says what this server holds: for each source and each
object that can hold personal data, its categories and whose data it
is, whether it is collected here at all, what bounds its retention (the
live value of the setting that does, or unbounded), whether it leaves
the host and to which endpoints, and a summary. Every host that
receives something is listed once as a candidate processor with what it
receives. Inside a tenant it answers with the tenant's slice and none
of the server's processors. Read-only, with sysComplianceGet.
inbuxa:InventorySnapshot/get is the history: a dated copy of the
evaluated inventory, recorded when it changes -- after a registry write
to an object the inventory reads, after inbuxa's log, audit or AI
settings change, and on the daily clean-up -- and kept as long as the
audit log's records. ids: null lists every snapshot, newest first; the
full inventory only when asked for.
The catalog is embedded and parsed at start (new dependency: toml,
MIT/Apache); the evaluation is a pure function of it and the live
facts, so each configuration is tested without a server. Loopback
endpoints stay on the host; any other configured endpoint leaves it.
Tested: unit tests for the evaluation (a new install's defaults, an
external blob store, a hosted AI endpoint, telemetry off, a tenant's
slice, hosts from URLs, loopback), snapshots, and the fact gathering's
store and duration rules; the compliance system test, extended (the
officer reads the inventory, a plain user is refused, a tenant's
officer sees its slice and no processors, a webhook to another host
becomes a processor and a snapshot names x:WebHook, a retention change
reads through); the system, audit, legal hold and account lock suites;
fork checks. The system suite failed once of three runs with an email
import's blob not found, in antispam.rs; the same happened once in
purge.rs on the previous branch. Nothing here touches uploads; noted
for a separate look.
Personal-data catalog spec, default D1 (settled 2026-09-28): log files
were never deleted. inbuxa:LogSettings.keepForDays says how many days
rotated log files are kept; unset (null) keeps every file, as before,
and a new install sets 30 days.
It is a fork-owned setting, stored under T + l as audit retention is,
not a field on x:TracerLog: that object is also stored inside
x:Bootstrap with a field after it, so a new field would change
x:Bootstrap's stored format. Server-level, with the tracers'
permissions (sysTracerGet, sysTracerUpdate); changes are in the audit
log, before and after.
Log files are local, so every node deletes its own: hourly, and at once
when the setting changes on that node. Only regular files named
<prefix>.<something> in each enabled log tracer's directory, last
changed more than the limit ago, are removed; the file being written is
never that old, and nothing else in the directory is touched. Minimum
one day. The catalog classifies inbuxa:LogSettings and points the log
file's retention at it.
Tested: unit tests for the file rule (only this log's old files; the
current file, other files and directories stay) and a purge on disk;
the system suite, which reads, sets, refuses zero, restores null and
checks the audit records; fork checks.
The legacy-protocols switch was all or nothing. An operator can now stop
POP3 and keep IMAP: each of IMAP, POP3 and ManageSieve has its own
switch, server-wide on inbuxa:ProtocolPolicy and per tenant on
inbuxa:TenantProtocolPolicy (properties imap, pop3, manageSieve).
legacyProtocols stays as the kill-all: setting it sets all three, and it
reads "disabled" exactly when all three are off. A policy stored before
this has only legacyProtocols and reads as all three at that value, so
existing servers and tenants carry over unchanged. In one /set, a
protocol named beside legacyProtocols overrides it.
SMTP submission keeps no switch of its own: sign-in over it is refused
only when all three are off, as the single switch did (LP-6), so
turning one protocol off never stops a mail app sending. For a tenant,
the server's switches and the tenant's count together.
Server-wide, a change closes the listeners of whatever is now off and
puts back the saved listeners of whatever is on again, both in one
change if asked; listeners of a protocol still off stay saved. Sign-in,
autoconfig, autodiscover, PACC (now prepared once per combination) and
the suggested DNS records all follow each protocol separately. A tenant
may turn a protocol on only while the server has it on (LP-9), and the
refusal names which. The JMAP session adds legacyAllowed, the protocols
still allowed for the account; legacyProtocols there keeps its meaning
for older webmail builds. Events name the switches ("pop3 disabled"),
and audit before/after reads every switch even from an older policy.
Tested: unit tests for the switches, the old-policy reading, the
server/tenant combination, the tenant refusal and listener refusal; and
tests/e2e/legacy_protocols.py against a running server, all 100 checks,
including new ones: POP3 alone off closes only its port and refuses
only its sign-in while IMAP and sending go on; only POP3 stops being
advertised; one change closes IMAP and reopens POP3; a tenant turns
POP3 off for itself, and can't turn IMAP on while the server has it off.
inbuxa:HoldExport/set takes a hold, optionally some of the accounts it
covers, and a reason; the collection runs in the background and get
says when it's ready. The ZIP has, per account, mail as .eml under its
folders, calendars as .ics, contacts as .vcf, files as stored, and the
archived items the hold keeps under archived/; a manifest.csv gives each
entry's account, kind, folder, date, whether it was archived, size and
SHA-256, and manifest.sha256 hashes the manifest. Accounts the hold
doesn't cover are left out, and items outside its date range are too:
live mail by arrival, events by start, and archived items the same way,
so an export doesn't carry deleted items that only another hold keeps.
The finished file is a blob of whoever started the export, so only they
download it, and it lasts as long as any upload (uploadTtl). Exports
are records under the hold (SUBSPACE_INBUXA H/e): never changed or
destroyed, each with its status, counts, size and checksum. Starting
one needs sysLegalHoldExport, an active hold and a reason, and is
recorded in the audit log like the audit log's own export.
The build is in memory and capped at 2 GB; bigger holds fail with a
message saying so, and are split by picking accounts.
Tested: unit tests for safe ZIP names and the manifest and its hash;
the legal_hold system test, on RocksDB, PostgreSQL and MySQL, exports a
hold end to end (live and archived mail, the manifest's hash, an asked-
for account the hold doesn't cover left out) and checks the refusals
(no reason, a user without the permission, a released hold) and the
audit record; and by hand from the console on a local server. Not
covered by a test: the archived-item date range with two holds of
different ranges over one account.
Destroying a held account removes the login, as offboarding needs, but
keeps its data as a deleted account with no expiry, whether or not
undelete keeps accounts; its addresses stay reserved and its holds name
it from then on. Destroy-now refuses it, and its DestroyAccount task
defers itself while it's held or its time hasn't come. Holds placed or
released later freeze or free kept accounts in the same settle pass,
with 30 days' grace after the last release (LH-8, LH-10).
Placing or widening a hold freezes what's already archived in its scope
and range, its old deadline noted; releasing one gives each item no other
hold covers that deadline back, or release plus 30 days if later. One
pass over the archive does both and changes nothing twice (LH-6, LH-10,
LH-11). A held archived item can't be destroyed; restoring still can,
and the hold is named only to callers who may see holds (LH-7). Audit
records about a held account survive the purge (AU-7).
Fixes the daily clean-up of expired archived items (UD-13), which never
found any: the registry's unfiltered query reads an all-ids index that
archived items aren't in. Items are now walked account by account, kept
deleted accounts included. Expired items were still removed whenever
their account's archive was read.
Every way of deleting mail (JMAP, IMAP EXPUNGE, POP3, mailbox removal,
Trash emptying) and Sieve scripts, events, contacts and files now asks
how the account's deletions are kept: a hold keeps them with no expiry
(archivedUntil 9999-12-31), even with undelete off; otherwise undelete's
period applies as before (LH-4).
A hold's date range decides by the item's own date (LH-3). Mail is noted
as held at deletion and settled when it's archived, once its received
date is known; outside the range it gets undelete's deadline or isn't
kept. Events go by their start, with a day's slack for time zones;
recurring events, contacts, files and scripts are held whole.
A groupware item's note now stays until its archive succeeds, and a
failure retries the task instead of being logged and lost (LH-5).
A hold reaches an account by name, through any of its addresses'
domains, its groups or its tenant, as they are now, so an account added
to a held domain later is held too. An account that leaves a held
domain, group or tenant stays held: the registry write hook adds it to
the hold by name on every account change, whoever makes it (LH-2).
Server::holds_on answers for the deletion paths, from the store each
time so a hold binds every node at once.
inbuxa:LegalHold get/set places a hold on accounts, groups, domains,
tenants or the whole server, with an optional date range. A hold's range
and scope can only widen, a released hold is read-only, and none is ever
deleted. Placing, changing and releasing each need a reason and are
audited (LH-1, LH-3, LH-10, AU-12).
Permissions 669-672 (see, place, widen or release, export held data)
go to server administrators only; the tenant ceiling always strips them,
as it does Impersonate (LH-13). Schema: Compliance > Legal Holds.
What a hold keeps comes next, through the undelete hooks.
Also moves the lock expiry helpers below the lock module's imports.
A delegation with an end date dropped out of the delegate's token then,
but its folder grants stayed until the daily sweep, so the delegate kept
the account as an ordinary share for up to a day. Each node now sleeps
until the soonest end date, woken early by any lock write and at least
hourly, and re-applies that lock under a cluster-wide claim.
The sweep also had a second-run bug: a delegation past its date gave the
delegate back its earlier share, then dropped the note, so the next sweep
removed that share entirely. The note is now kept while the delegate is
still listed.
A locked account can't sign in (it fails as a wrong password does), its
sessions end on every node, refresh tokens stop working, and its Sieve
scripts forward and reply to nothing. Mail keeps arriving.
Delegates get real ACL grants on the account's mailboxes, calendars,
address books and files at read, organize or full, with the rights they
replaced restored on unlock. Folders made later are granted after the
create and in a daily sweep. Organize delegates can't destroy; send-as
needs organize or full. The JMAP session marks delegated accounts in
urn:inbuxa:jmap.
New inbuxa:AccountLock object with get/set, permissions 665-668, and a
Compliance > Locked Accounts entry in the schema. Lock, unlock and
delegate changes need a reason and are audited; delegate access and
writes are audited too (audit-hold-lock spec AL-1 to AL-12).
What administrators and the server itself do to the control plane is now
recorded, from inbuxa-drafts/specs/audit-hold-lock.md (AU-1 to AU-12):
settings, accounts, domains, roles and every other registry change, with
each field's before and after (secrets only as "changed"); the fork's own
settings objects; administrator sign-ins (and failed ones to administrator
accounts), master-user and recovery-admin sign-ins, once an hour per
account, method and address; access to another account's data through
impersonation or FetchAnyBlob, once an hour; exports and tamper checks;
and registry writes the server makes on its own, named by subsystem
(system:AcmeRenewal, system:auto-ban, system:directory-sync, ...), with a
spam rules update as one summary record.
No change without its record (AU-3): before a set method changes anything,
a pending record per requested create, update and destroy is written; if
that fails, the method is refused with serverFail. Its outcome follows as
a later entry. A change interrupted by a crash stays "unfinished".
Records live in the fork's subspace under L, as one SHA-256 hash chain per
node. The chain's head is stored, never cached, and every append asserts
it, so two writers can't take the same place. Nothing can edit or delete
a record; the daily purge removes the oldest past the retention (default
730 days, minimum 90) and records where the chain now starts, so
verification still passes. security.audit-recorded (647) copies each
record to webhooks, OpenTelemetry and the log; security.audit-write-failed
(648) reports a failed write.
New JMAP objects under urn:inbuxa:jmap: inbuxa:AuditEvent/get and /query
(filters: time, actor, action, target, account, tenant, outcome, address,
text), inbuxa:AuditSettings, inbuxa:AuditExport (CSV or JSON Lines built
on the server, each line with its chain hash, ending in a manifest; the
created object names the blob and its SHA-256) and
inbuxa:AuditVerification. New permissions sysAuditGet, sysAuditExport and
sysAuditSettingsUpdate: the Administrator role gets all three, the Tenant
Administrator role gets read and export, once, on existing installs too.
A tenant administrator sees records whose actor or target is in its
tenant, including a server administrator's changes there.
Sign-in method on the session: access tokens now remember how they signed
in (password, app password, API key, OAuth client, directory, master user,
recovery admin), including across the HTTP credential cache. New OAuth
access tokens carry their client id in the sealed claims; older ones show
as client "unknown" until they expire.
The schema gains the permissions, the two events and a Management >
Compliance > Audit Log link.
Stack: the request layer boxes every inner future where it's made. Without
that, a debug build overflowed the default 2 MB worker stack on a registry
set; measured with the same request, the branch and main now overflow at
the same stack size (between 1856 and 1920 KiB, debug), so the layer adds
nothing measurable.
Tests: unit tests in inbuxa-features and jmap; system::audit::audit_log_tests
(run with --ignored) passes on RocksDB, SQLite, PostgreSQL, PostgreSQL with a
read replica, MySQL, MySQL with a replica and FoundationDB. The system, JMAP
and SCIM suites pass. authorization.rs skipped fork permissions that guard
no registry object; the audit suite checks a plain user is refused instead.
ai-explain spec, amendment 1 (EX-22 to EX-28):
- answers are three or four sentences, max_tokens 160, cut at 700 chars;
- POST /api/explain streams the answer as server-sent events;
- each node remembers answers in memory (1,000, 24 h), keyed by the facts,
prompt version and model, shared by server-level administrators;
- resources/explain/settings.json.gz ships answers for settings at their
defaults, generated with prepare_setting_explanations (717 for 2026.9.27);
- the system prompt no longer carries the per-request marker, so a model
server can reuse it;
- inbuxa:Explanation gains source, answeredAt and preparedFor.
A new method, inbuxa:Explanation/set, asks the node's local model for a
short plain-words reading of one thing an administrator is looking at:
a failed recipient in the queue, a Classify verdict, a log line or trace
event, or one setting with its saved value. The server builds the prompt
itself from stored data and the registry schema, never from text the
console sends, and grounds SMTP replies in RFC 3463 and RFC 5321.
What the model is never shown: secrets (including ones nested inside a
setting, like an AI model's HTTP auth), raw protocol events, and the
contents of any other event. A tag name that doesn't have a tag's shape is
refused before a model is asked.
Calls share the AI gate with spam classification, but mail always keeps
its slot, and Explain has its own hourly count per account and its own
on/off switch in inbuxa:AiLimits. The permission is sysAiExplain,
superuser only; tenant administrators can't use it. The session carries
an aiExplain flag so a console knows when to offer the button.
An install whose roles were stored before the permission existed gets it
added once, at start-up, to the roles that are administrators' alone,
not the User role their defaults share with every account. An operator
who removes it later isn't overruled.
Tests: unit tests in inbuxa-features and jmap, and ai_explain_tests
(run with --ignored) covering the acceptance tests and the upgrade.
Deleting a tenant now also removes its stored inbuxa:TenantProtocolPolicy,
in the same place the registry's other per-type clean-ups run. Without it
the row outlived the tenant, and a tenant that later came to have the same
id would have started with legacy protocols off.
The e2e deletes a tenant whose switch a server administrator had turned
off, and would check that a new tenant with the same id starts with them
on. On this build the registry hands out a fresh id instead ("d" after
"c"), so the reuse -- and with it the removal -- isn't observable over
JMAP; the test says so rather than passing silently. The risk it guards
was therefore smaller than feared, and the change is mostly about not
leaving an orphaned row behind. All 72 checks pass.
The impact panel's data. Every successful sign-in over IMAP, POP3,
ManageSieve or SMTP AUTH records, per account and per protocol, one
timestamp -- nothing else: no address, no IP, no client. It is written at
most once an hour per account and protocol, so a mail app polling every
minute costs a read per sign-in and a write an hour. A record that can't be
written is logged and the sign-in goes ahead.
Both switches serve it as a read-only property, recentLegacyUse, as
wouldClose serves the confirmation: a list of {accountId, name, protocol,
lastUsedAt} for sign-ins in the last 30 days, most recent first.
inbuxa:ProtocolPolicy lists every account; inbuxa:TenantProtocolPolicy
lists only its tenant's own (MT-1). Accounts since deleted are left out. It
is computed only when the property is asked for.
The recording sits where the tenant check already runs once the account is
known, which becomes admit_legacy_session: refuse if the account's tenant
has legacy protocols off, otherwise record. A refused sign-in is never
recorded.
The spec leaves the interface to the implementation; a property on each
switch keeps the panel's data behind the same permission as the switch
itself, with no new object.
Unit tests hold the 30-day window to acceptance test 11 (three days ago
listed, forty not), the hourly throttle and the keys. The e2e proves on a
running server that the admin's IMAP and submission sign-ins are listed
with their time, that a second sign-in within the hour isn't written again,
and that a tenant's list holds its own user and nobody outside the tenant.
All 70 checks pass.
The tenant switch. A tenant's administrator turns legacy mail protocols
off for its own tenant, and from then on sign-in over IMAP, POP3,
ManageSieve and SMTP AUTH is refused for every address on the tenant's
domains, while every other domain on the server carries on. No port
closes, since other tenants share them (LP-13): it is one stored fact per
tenant, read at sign-in and when client configuration is answered.
inbuxa:TenantProtocolPolicy/get and /set, one per tenant, id the tenant's:
- Inside a tenant, a principal reaches only its own tenant's switch
(MT-1): /get with no ids answers with it, another tenant's is notFound
and can't be changed. At server level /get with no ids lists every
tenant's.
- Turning it off is always allowed. Turning it back on is refused with
forbidden, naming inbuxa:ProtocolPolicy, while the server has legacy
protocols off (LP-9).
- A change raises security.legacy-protocols-changed with policy = tenant,
the tenant's id, the new value and who made it (LP-14).
- It takes sysDomainGet and sysDomainUpdate, not the two new permissions
the spec names. The switch governs sign-in on the tenant's domains, so
whoever manages those domains may turn it -- and the default Tenant
Administrator role already holds both, where new permissions would reach
no role already stored on a server (MT-12's note), leaving today's
tenant administrators without the switch until someone edited their
role by hand. The same trade inbuxa:AiLimits and inbuxa:ProtocolPolicy
made. /query is not built yet; /get with no ids covers listing.
Sign-in (LP-10 to LP-12). Before the credentials are looked at, the name
given is resolved to its domain and the domain to its tenant, so a real
account and a made-up address on the domain get the same refusal, with a
right password or a wrong one, counted as no failed sign-in (LP-11). The
words are the spec's: "Your organization allows only INBUXA webmail and
JMAP apps...", in each protocol's form. A bearer token needn't name an
account, so after authentication the account's own tenant is checked too;
a token that named nobody can't slip past.
The refusal carries policy = tenant and the domain, not the tenant's id:
IMAP answers a command's tag from the Id key, so an error holding one was
sent under the wrong tag and the mail app hung waiting for its reply. The
first live run found that; a unit test now holds the refusal to it.
Client configuration (LP-14a). Autoconfig, autodiscover, PACC and the
suggested DNS records now ask whether legacy services are off for the
domain being answered for -- the server's switch, or the domain's
tenant's -- so a tenant's domains stop offering IMAP, POP3 and
submission while others still do.
tests/e2e/legacy_protocols.py builds a tenant with its own domain, a user
and a tenant administrator, and a second tenant, and proves on a running
server: the admin sees and changes only its own tenant's switch (test 10);
turning it off is an event (test 14); the tenant's user is refused over
IMAP with the right password and a wrong one, a made-up address on the
domain the same (tests 6, 7); POP3 and submission refuse in their own
forms and JMAP still works (test 8); an account on another domain signs in
normally (test 6); autoconfig drops IMAP for the tenant's domain only; with
the server off, the tenant can't turn it back on (test 9); and once back
on, the user signs in again. All 62 checks pass.
John, 2026-09-20: "SMTP and JMAP should be shown with the selector locked,
we want to prevent those two protocols from being shutdown for now."
The selector lists every mail protocol the server speaks, so the operator
sees the whole surface at once; SMTP and JMAP sit in it named and visibly
not switchable. JMAP was never closeable -- closing it locks everyone out
of their mail and the operator out of INBUXA Admin, with no way back but
the host -- and is now visibly so. SMTP is locked whole. LP-3 already
spared inbound on 25; this extends that to submission on 465 and 587,
which LP-1 would otherwise have closed by default.
So closeSubmission has no effect while the lock stands, and is forced to
false. A client that asks for true is not refused: the value is recorded,
overruled, and the overrule reported, because the field is specified and
the lock is meant to be temporary. is_locked() is consulted before
anything else in closes(), so no phrasing of a request reaches past it.
This costs the feature nothing. Submission's ports stay open and sign-in
over them is still refused once LP-6 lands, which is the case acceptance
test 2 already described: a mail app reaching 465 is told it cannot sign
in rather than finding nothing listening. The operator also keeps a port
they may well be forwarding, which is the LP-20 problem in miniature.
The locked set is a server constant the front ends read, not a list they
carry, so unlocking later is a server change and no admin release. The
LP-3 tests stay as they are, to keep it covered if the lock is lifted.
Recorded as LP-21, with acceptance tests 17 and 18.
LP-1 and LP-5, the registry half. close() removes every listener object
the policy closes, saving each one whole first; reopen() puts them back.
The switch removes the listener objects, not just their sockets. A stopped
socket returns on the next restart, which would reopen every port the
operator had just closed, and the operator would have no way to know. A
removed object stays removed, and a server that boots with the switch on
never spawns those listeners at all -- so there is no boot-time special
case to write or to forget.
LP-3 is decided here, on the object rather than the running socket, and
sees every address a listener binds: a submission listener that also binds
25 is inbound and stays. lmtp and http are never candidates.
A listener that cannot be put back does not stop the others; it comes back
with its reason and stays saved for another try (LP-5). A delete the
registry declines is reported as not removed, so the policy never claims a
port is closed while it is still accepting.
Stopping the running socket is still a separate step in common, which owns
the listener registry. Nothing calls any of this yet.
The server-wide legacy-protocols policy: the switch, whether submission
closes with it, the listeners taken away to honour it, and who last
changed it. Stored like inbuxa:AiLimits, as JSON in the fork's subspace,
so an unset field reads as its default and an old record still loads.
closes() is where LP-3 lives. imap, pop3 and manageSieve are named
outright; smtp is not, because an SMTP listener is inbound or submission
depending on its port and nothing else can tell them apart. A listener
bound to 25 is inbound whatever it is called, including one that also
binds 465, so it stays. http and lmtp are never candidates at all.
savedListeners keeps each listener's registry object whole rather than a
few fields of it. LP-5 promises the listeners come back exactly as they
were, and a listener carries proxy networks, TLS timeouts and socket
options that no one should have to re-derive -- a field this code has
never heard of has to survive the round trip too, and a test holds that.
The module is under security/ rather than beside the rebuilt features,
because this one is not a rebuild: upstream has nothing like it.
Still only a fact. Nothing reads this policy yet, so no port closes and
no sign-in is refused; the acting code needs the listener registry and
the config store, which live above this crate.
The calibration harness (tests/src/system/ai_calibration.rs, ignored) sends
what the classifier sends to a real local model and scores its answers with
the classifier's parser.
The classifier sends only the subject and text, between unforgeable markers
after the operator's prompt, to an OpenAI-compatible endpoint the operator
configured; nothing is preset. Its answer maps to an LLM_ tag whose score is
clamped (+5.0, -1.0 by default) and can never discard or reject on its own;
X-Spam-LLM is sanitized, encoded and folded, and a planted one is removed.
Failures, timeouts past the ceiling, a full slot or a paused model leave
mail flowing untagged. llm_prompt answers trusted scripts, and accounts
holding interactAi within an hourly limit. Redirects aren't followed and no
content or secret is logged. The limits live in inbuxa:AiLimits.
Acceptance tests 1 and 3 to 21; test 2 as the re-enabled shared llm case,
whose setup no longer waits on a rules file from a developer's own path;
test 22 written as the ignored ai_compat.
Logos resolve domain, then tenant, then server-wide, then the built-in, with
subdomains finding their domain. GET /logo serves a data-URL image, redirects
to a URL logo without fetching it, sandboxes SVG, and answers 404 when no
custom logo applies. Emails embed the first PNG, JPEG or GIF logo. Logo and
template writes are checked; stored templates are read at send time, always
escaped, and fall back to the built-in with a build warning when they don't
parse. The RSVP page is served byte for byte with a CSP and no-referrer. The
sign-in and RSVP pages load the logo through an image element. MT-22's
session logo follows the chain to the server-wide logo.
Acceptance tests 1 to 17; test 18 written as the ignored branding_compat.
With archiveDeletedAccountsFor set, a destroyed account's record is kept in
the fork subspace with its id, its DestroyAccount task is due at the end of
the period, and its shares are suspended both ways. Its addresses can't be
taken by new accounts, aliases, lists or masks. inbuxa:DeletedAccount/get
lists kept accounts to server and tenant administrators; /set restores one
with a new password (same id, task cancelled, shares reinstated) or destroys
it now. The destroy task also clears undelete's own records.
Acceptance test 14; test 16 written as the ignored undelete_compat.
Files, events and contacts are noted when deleted for good and archived by
the unindex task when retention is on; Sieve scripts are archived at
deletion. A restore goes back to its folder, calendar or address book if it
still exists, takes a free " (restored)" name, comes back inactive for
scripts, and is refused over quota with the item left archived.
Acceptance tests 6, 8 and 12.
Every way of deleting mail for good (JMAP, IMAP expunge, POP3, Trash
emptying, mailbox removal) notes the message's mailboxes and keywords while
archiving is on, fixing its deadline then; when its data is finally removed
it becomes an x:ArchivedItem record, written as upstream writes them, with
its copy held until the deadline. Retention is read at deletion time, so a
change applies at once. Restore puts a message back in the mailboxes it was
in (Trash only if that was all), with its keywords, and removes the record;
over quota it stays archived. x:ArchivedItem/get returns status and
accountId; query filters on type, archivedAt and text; set requests a
restore once or destroys; /changes is a fork addition. Expired items go in
the data purge. The shared account-access rule moves to jmap::inbuxa::access.
system_tests now calls undelete::test, and the archiving gate is gone.
Found by running system_tests, which masked email no longer stops:
- rcpt_resolve rewrites a live mask to its owner's address, so
Delivered-To names the account; delivery recognizes the mask from the
original recipient when it belongs to that account.
- x:MaskedEmail/set create responses carry the server-set email.
- x:MaskedEmail/query returns every mask to a server-level impersonate
holder, and filters on accountId.
- The refusal for an unlinked emailDomain uses upstream's wording.
- The shared delivery test checks the fork's address format (ME-13).
- The masked email test's tenant domain uses manual DKIM, so its cleanup
leaves nothing behind.
The fork's per-account change log answers /changes, collapsing a mask
created and destroyed in the window. /changes on a registry type needs that
type's get permission.
A live mask accepts mail at RCPT TO and delivers to its owner, with an
X-Masked-Email header naming it. A disabled mask files straight to Trash,
past the owner's Sieve script. Deleted and expired masks are refused like
unknown addresses, without being cached as unknown. Mail moves lastMessageAt
and turns a pending mask enabled. Sub-addresses on a mask work.
Subspace _ holds the fork's own data, with its own SQL table and RocksDB
column family, and is part of backup. The masked_email module keeps each
mask's state, last mail and pending deadline beside upstream's record, an
index from address to mask with tombstones, and a per-account change log;
and generates addresses in the fork's format.
New crate crates/features (inbuxa-features), AGPL-3.0-only, holding the
tenancy rules. Hooks in common: a tenant's roles and permission lists cap
its people's permissions; a change to a tenant, or to a role a tenant holds,
drops its members' cached permissions; delivery and every other write check
the tenant's maxDiskQuota; usedDiskQuota reads the tenant usage counter.
The default Tenant Administrator role gains sysTenantGet and sysTenantQuery.
jcoffey-dev