jcoffey-dev is traveling from Thursday 1 October through Sunday 4 October. Issues and pull requests are welcome, and will get an answer after that. Thanks for your patience.
Personal-data catalog spec, default D5 (settled 2026-09-28; built after
the v0.16.24 import's spam-rules loader landed). msbl.org's EBL is sent
a SHA-1 of every email address it's asked about. A new install's first
boot now leaves a note, and the rules update, once the bundled rules
are in, switches STWT_MSBL_EBL_EMAIL off and forgets the note, so it
happens once; the loader keeps that switch through later updates. An
existing server has no note and keeps every blocklist as it is.
Also fixes the data inventory's DNSBL endpoints: a zone is an
expression (`ip_reverse + '.zen.spamhaus.org'`, conditional branches,
`hash(email, 'sha1') + '.ebl.msbl.org'`), and the zone names are now
the quoted literals that start with a dot, from every branch, rather
than the expression's text.
Tested: unit test for the zone rule; the compliance system test (no
note, no change; the inventory lists ebl.msbl.org, not a hash; with the
note the blocklist goes off; the note works once); the system suite;
fork checks.
Personal-data catalog spec, §6 (Phase 3c).
inbuxa:DataInventory/get evaluates the catalog against the server's
live settings and says what this server holds: for each source and each
object that can hold personal data, its categories and whose data it
is, whether it is collected here at all, what bounds its retention (the
live value of the setting that does, or unbounded), whether it leaves
the host and to which endpoints, and a summary. Every host that
receives something is listed once as a candidate processor with what it
receives. Inside a tenant it answers with the tenant's slice and none
of the server's processors. Read-only, with sysComplianceGet.
inbuxa:InventorySnapshot/get is the history: a dated copy of the
evaluated inventory, recorded when it changes -- after a registry write
to an object the inventory reads, after inbuxa's log, audit or AI
settings change, and on the daily clean-up -- and kept as long as the
audit log's records. ids: null lists every snapshot, newest first; the
full inventory only when asked for.
The catalog is embedded and parsed at start (new dependency: toml,
MIT/Apache); the evaluation is a pure function of it and the live
facts, so each configuration is tested without a server. Loopback
endpoints stay on the host; any other configured endpoint leaves it.
Tested: unit tests for the evaluation (a new install's defaults, an
external blob store, a hosted AI endpoint, telemetry off, a tenant's
slice, hosts from URLs, loopback), snapshots, and the fact gathering's
store and duration rules; the compliance system test, extended (the
officer reads the inventory, a plain user is refused, a tenant's
officer sees its slice and no processors, a webhook to another host
becomes a processor and a snapshot names x:WebHook, a retention change
reads through); the system, audit, legal hold and account lock suites;
fork checks. The system suite failed once of three runs with an email
import's blob not found, in antispam.rs; the same happened once in
purge.rs on the previous branch. Nothing here touches uploads; noted
for a separate look.
Personal-data catalog spec, default D1 (settled 2026-09-28): log files
were never deleted. inbuxa:LogSettings.keepForDays says how many days
rotated log files are kept; unset (null) keeps every file, as before,
and a new install sets 30 days.
It is a fork-owned setting, stored under T + l as audit retention is,
not a field on x:TracerLog: that object is also stored inside
x:Bootstrap with a field after it, so a new field would change
x:Bootstrap's stored format. Server-level, with the tracers'
permissions (sysTracerGet, sysTracerUpdate); changes are in the audit
log, before and after.
Log files are local, so every node deletes its own: hourly, and at once
when the setting changes on that node. Only regular files named
<prefix>.<something> in each enabled log tracer's directory, last
changed more than the limit ago, are removed; the file being written is
never that old, and nothing else in the directory is touched. Minimum
one day. The catalog classifies inbuxa:LogSettings and points the log
file's retention at it.
Tested: unit tests for the file rule (only this log's old files; the
current file, other files and directories stay) and a purge on disk;
the system suite, which reads, sets, refuses zero, restores null and
checks the audit records; fork checks.
The schema, which both sides changed, merged as JSON with no conflicts.
The personal-data catalog (#83) gains upstream's new x:DnsServerPowerDns:
nothing personal but its API key, like the other DNS providers.
Personal-data catalog spec, defaults D2, D3, D4, D6 and D7 (settled
2026-09-28, new installs only):
- D2: automatic IP bans expire after 30 days instead of never; D3:
spam training samples, whole messages, are kept 90 days instead of
180; D4: Pyzor, which sends a digest of each message's text to a
public server, is off; D6: delivery history is kept 14 days instead
of 30. Written on the first boot of a new install only -- one with no
roles yet, the same test the built-in roles use -- by reading each
singleton, setting these fields and writing it back whole. A server
with roles keeps its settings, saved or default.
- D7: a webhook created from now on starts with the include policy and
no events, so it sends nothing until events are chosen (Rust default
and schema default, marked). The registry stores every field, so
existing webhooks keep their policy.
- Expired bans are also removed by the daily data clean-up. They
already stopped blocking and were deleted when settings next loaded;
a server that seldom reloads kept them.
D1 (log retention) and D5 (the hashed-address blocklist off) are held,
and the spec says why: x:TracerLog is stored inside x:Bootstrap with a
field after it, so adding one changes that object's stored format; and
the spam-rules loader D5 touches is being reworked by the v0.16.24
import. The spec also corrects finding 3: expired bans were deleted on
settings load; bans were permanent only because no period is set.
Tested: unit tests for the new-install values and that everything else
in each singleton stays; the system suite, whose security test now
purges an expired ban and checks its record is gone; the telemetry
test; common's unit tests; fork checks.
Eight conflicted files resolved, plus the lock file and the schema:
- crates/services/src/task_manager/spam_classifier.rs: upstream's rules
update now replaces existing rules, DNSBL servers, lookups and file
extensions, keeping only whether each is on. Taken, with one difference:
an object an admin edited is kept as it is. Every object an update writes
is fingerprinted (content without `enable`, SHA-256, stored under
SUBSPACE_INBUXA "Sf"), and only one that still matches is replaced.
Scores are never replaced, as upstream has it. The AU-1.10 summary record
now names what was added, replaced and kept, and the bundled rules are
marked applied only when the update fully succeeded, so a failure runs
again on the next start. The marker becomes "3.0.2+2", which runs the
update once on upgrade to fingerprint every rule still as bundled.
- crates/common/src/network/autoconfig/autodiscover.rs: upstream's rewrite
(implicit TLS first, labeled SSL), with the per-protocol switches (LP-7,
LP-14a) passed in as a filter.
- crates/store/src/backend/mysql/{search,write}.rs: upstream's chunked
deletes (no unbounded first DELETE, stop on a short chunk, halve the
chunk on the new chunk-too-large errors) inside the fork's query timeout.
- crates/smtp/src/lib.rs: the fork's queue spawn kept. It already fixed the
stall upstream fixes here (a node without outboundMta stops accepting
mail at about 1024 queued messages), and follows role changes live.
- crates/jmap/src/registry/mapping/bootstrap.rs: the log path stays
/var/log/inbuxa/; upstream's PowerDNS mapping taken.
- crates/main/Cargo.toml: the AGPL-only license kept, version 0.16.24.
- tests/src/jmap/principal/get.rs: the fork's capabilities kept.
- resources/schema/schema.json.gz: merged as JSON; upstream relabeled the
vendor Sieve extensions "(Stalwart)", kept as "(vnd.inbuxa)".
- Cargo.lock: upstream's, with the fork's crates added by Cargo.
Also:
- tests/src/smtp/inbound/spam_rules_kept.rs: an edited rule survives an
update, an unedited one is updated, rules from before fingerprints are
handled, and the audit summary says so. Upstream's own spam_rules test
passes unchanged.
- tests/src/smtp/reporting/reschedule.rs moves to port 19058; upstream's
new spam_rules test took 19057.
- tools/fork/renames.py renames the "(Stalwart)" labels and the default
log path, so neither conflicts again.
- tools/fork/notice-check.py compares against the newest snapshot in the
checked-out history instead of the upstream branch head, so moving the
branch no longer fails other open pull requests.
- tests/src/directory/issuer.rs (since v0.16.23) stays out, and is on the
build check's known list: it tests issuer-based directory routing, which
the fork doesn't have (DIR-2).
- Strip report: docs/fork/strip-reports/v0.16.24.{md,json}.
Upstream commit: af37a234981722493b74623a983581691d2b70b6
Enterprise-only files removed or emptied: 63
Enterprise-only snippets removed: 118 in 50 files
Dangling module declarations removed: 5
Edits turning enterprise off: 25
Third-party code: 14 files, 0 not in THIRD-PARTY.md
Renamed identifiers: 62 in 18 files
Verification: clean
The same Enterprise footprint as v0.16.23. The build check fails only on
tests/src/directory/issuer.rs, unchanged since v0.16.23: it calls a helper
from upstream's Enterprise-only OIDC test, and tests issuer-based directory
routing, an Enterprise feature. main has never carried it.
Destroying a held account removes the login, as offboarding needs, but
keeps its data as a deleted account with no expiry, whether or not
undelete keeps accounts; its addresses stay reserved and its holds name
it from then on. Destroy-now refuses it, and its DestroyAccount task
defers itself while it's held or its time hasn't come. Holds placed or
released later freeze or free kept accounts in the same settle pass,
with 30 days' grace after the last release (LH-8, LH-10).
Every way of deleting mail (JMAP, IMAP EXPUNGE, POP3, mailbox removal,
Trash emptying) and Sieve scripts, events, contacts and files now asks
how the account's deletions are kept: a hold keeps them with no expiry
(archivedUntil 9999-12-31), even with undelete off; otherwise undelete's
period applies as before (LH-4).
A hold's date range decides by the item's own date (LH-3). Mail is noted
as held at deletion and settled when it's archived, once its received
date is known; outside the range it gets undelete's deadline or isn't
kept. Events go by their start, with a day's slack for time zones;
recurring events, contacts, files and scripts are held whole.
A groupware item's note now stays until its archive succeeds, and a
failure retries the task instead of being logged and lost (LH-5).
A delegation with an end date dropped out of the delegate's token then,
but its folder grants stayed until the daily sweep, so the delegate kept
the account as an ordinary share for up to a day. Each node now sleeps
until the soonest end date, woken early by any lock write and at least
hourly, and re-applies that lock under a cluster-wide claim.
The sweep also had a second-run bug: a delegation past its date gave the
delegate back its earlier share, then dropped the note, so the next sweep
removed that share entirely. The note is now kept while the delegate is
still listed.
A locked account can't sign in (it fails as a wrong password does), its
sessions end on every node, refresh tokens stop working, and its Sieve
scripts forward and reply to nothing. Mail keeps arriving.
Delegates get real ACL grants on the account's mailboxes, calendars,
address books and files at read, organize or full, with the rights they
replaced restored on unlock. Folders made later are granted after the
create and in a daily sweep. Organize delegates can't destroy; send-as
needs organize or full. The JMAP session marks delegated accounts in
urn:inbuxa:jmap.
New inbuxa:AccountLock object with get/set, permissions 665-668, and a
Compliance > Locked Accounts entry in the schema. Lock, unlock and
delegate changes need a reason and are audited; delegate access and
writes are audited too (audit-hold-lock spec AL-1 to AL-12).
What administrators and the server itself do to the control plane is now
recorded, from inbuxa-drafts/specs/audit-hold-lock.md (AU-1 to AU-12):
settings, accounts, domains, roles and every other registry change, with
each field's before and after (secrets only as "changed"); the fork's own
settings objects; administrator sign-ins (and failed ones to administrator
accounts), master-user and recovery-admin sign-ins, once an hour per
account, method and address; access to another account's data through
impersonation or FetchAnyBlob, once an hour; exports and tamper checks;
and registry writes the server makes on its own, named by subsystem
(system:AcmeRenewal, system:auto-ban, system:directory-sync, ...), with a
spam rules update as one summary record.
No change without its record (AU-3): before a set method changes anything,
a pending record per requested create, update and destroy is written; if
that fails, the method is refused with serverFail. Its outcome follows as
a later entry. A change interrupted by a crash stays "unfinished".
Records live in the fork's subspace under L, as one SHA-256 hash chain per
node. The chain's head is stored, never cached, and every append asserts
it, so two writers can't take the same place. Nothing can edit or delete
a record; the daily purge removes the oldest past the retention (default
730 days, minimum 90) and records where the chain now starts, so
verification still passes. security.audit-recorded (647) copies each
record to webhooks, OpenTelemetry and the log; security.audit-write-failed
(648) reports a failed write.
New JMAP objects under urn:inbuxa:jmap: inbuxa:AuditEvent/get and /query
(filters: time, actor, action, target, account, tenant, outcome, address,
text), inbuxa:AuditSettings, inbuxa:AuditExport (CSV or JSON Lines built
on the server, each line with its chain hash, ending in a manifest; the
created object names the blob and its SHA-256) and
inbuxa:AuditVerification. New permissions sysAuditGet, sysAuditExport and
sysAuditSettingsUpdate: the Administrator role gets all three, the Tenant
Administrator role gets read and export, once, on existing installs too.
A tenant administrator sees records whose actor or target is in its
tenant, including a server administrator's changes there.
Sign-in method on the session: access tokens now remember how they signed
in (password, app password, API key, OAuth client, directory, master user,
recovery admin), including across the HTTP credential cache. New OAuth
access tokens carry their client id in the sealed claims; older ones show
as client "unknown" until they expire.
The schema gains the permissions, the two events and a Management >
Compliance > Audit Log link.
Stack: the request layer boxes every inner future where it's made. Without
that, a debug build overflowed the default 2 MB worker stack on a registry
set; measured with the same request, the branch and main now overflow at
the same stack size (between 1856 and 1920 KiB, debug), so the layer adds
nothing measurable.
Tests: unit tests in inbuxa-features and jmap; system::audit::audit_log_tests
(run with --ignored) passes on RocksDB, SQLite, PostgreSQL, PostgreSQL with a
read replica, MySQL, MySQL with a replica and FoundationDB. The system, JMAP
and SCIM suites pass. authorization.rs skipped fork permissions that guard
no registry object; the audit suite checks a plain user is refused instead.
Setting deliverAt on an internal DMARC or TLS report wrote the new task
queue row with the report's object type (0x21, 0x6e) instead of the task
type (7, 8), and left the task row at its old due. The task manager's scan
failed on that row with store.data-corruption ("Failed to iterate over task
queue"), and because the error ended the whole scan, every task due after
the row stopped running on every node.
- reschedule_ops writes the new queue row through schedule_task_with_id, so
it carries the task type and the task row gets the new due. It removes
the row the task is actually queued under (the task's due, which differs
from deliverAt once the task has been retried) and any row an earlier
reschedule left at deliverAt.
- x:DmarcInternalReport/set and x:TlsInternalReport/set lock the report's
task while they move it, as x:Task/set does, refuse while the report is
being sent, release the locks however the request ends, and wake the task
manager.
- The task manager logs a queue row it can't read (id, due, key, value) and
skips it instead of ending the scan. It then repairs the row from its task:
the row is rewritten with the task's type, and a row with no task behind
it is removed. A row holding a report's object type for a report task is
what the old reschedule wrote: the task is moved to that row's time, as
the reschedule intended, and its old queue row is removed. Stores that
already hold such a row recover on their own once it comes due.
- x:Task/query with a type filter skips an unreadable row instead of
failing.
Test: smtp::reporting::reschedule (RocksDB and PostgreSQL). It fails on
main: x:Task/get shows the old due, and with that check removed, neither
report nor a later task ever runs.
In cluster rehearsal 3, turning outboundMta off on node1's role was
reported applied (x:settingsReload applied: true), yet node1 kept
delivering mail, a report message included, until it was restarted.
The queue and report managers were started at boot only when the
node's role included outboundMta (crates/smtp/src/lib.rs), and the task
manager only when the role had some task type (spawn_task_manager).
After that nothing looked at the role again: a queue manager that was
running kept claiming and delivering, and one that wasn't never
started.
They now start on every node (outside recovery mode) and follow the
role live:
- Queue manager: before each scan it reads the role from the running
settings. Without outboundMta it claims nothing new; deliveries
already running finish and report back as usual, which releases
their locks. When the role comes back (a reload wakes the manager
with ReloadSettings, and it looks again every 30 s regardless) it
logs queue.started and scans the whole queue at once.
- Report scheduler: DMARC and TLS report events are handled only while
the role has outboundMta, as at boot; events arriving without it are
dropped, as they were on a node started without the role.
- Task manager: task_enabled already read the current role on every
scan. It now also runs on nodes whose role has no task type (the
scan returns at once until one is added), a job claimed before a
role change is handed back at once rather than run or held until
its lease lapses, and a settings reload wakes the manager so a role
that gained task types starts claiming them straight away.
Starting the queue manager on every node also drains the queue channel
on nodes without outboundMta. Upstream left that channel unread, so
each message queued there parked a refresh in it, and by the code,
queueing would block once 1024 had piled up (not reproduced here).
A role object edit reaches the nodes that name that role in
INBUXA_ROLE. Moving a node to another role still means changing its
environment, and so a restart. Listener changes in a role still need a
restart too (listeners bind at boot); this change is about tasks and
delivery.
cluster::live_roles::live_role_tests (new; PostgreSQL, two nodes over
one store):
1. A node started with outboundMta delivers and runs a TLS report
task; after its role loses outboundMta and the settings reload, a
new message isn't attempted and a new report task stays pending;
with the role back, both are taken up.
2. A node started with no task type at all gains outboundMta: a
waiting message is attempted and a report task runs.
On main the test fails at step 1 ("delivery attempted without
outboundMta"); with step 1 bypassed, step 2 fails (nothing picked the
message up in 20 s).
A 3-node rehearsal (PostgreSQL + NATS + Garage) found two ways a crash
leaves work stuck:
Pool hangs. The PostgreSQL pool (deadpool) was built with no timeouts,
so a request waited for a free connection, and for one to be opened or
recycled, for as long as it took: forever when the server stopped
answering. MySQL's pool (mysql_async) has no wait timeout at all.
- PostgreSQL: wait 30 s (or the store's timeout if longer), create the
store's timeout or 15 s (it bounds the whole handshake, where
tokio-postgres's connect_timeout covers only the TCP connect), recycle
10 s. The pool config is now always set, not only with
poolMaxConnections.
- MySQL: every connection is taken through MysqlStore::conn(), which
gives up after 30 s.
- Both: TCP keepalive after 60 s idle, so a server that vanished
without closing the connection is noticed in minutes rather than the
two-hour system default.
The DataStore schema has no pool timeout settings, so these are fixed
defaults; the store's own timeout bounds connecting on PostgreSQL.
Task locks. A task lock lasted an hour, so after a hard crash the dead
node's tasks waited up to an hour and five minutes. The lock is now a
five-minute lease: while this node runs a task, the task manager renews
its lock every third of the lifetime (InMemoryStore::renew_lock, a
compare-and-set on the store backends and SET XX EX on Redis, which
leaves a lock that already expired alone). A killed node's tasks run
elsewhere within about five minutes plus the claim recheck. A task this
node holds isn't handed to a worker again by the scan.
store::pool_timeout (new): a local listener that accepts connections
and never answers plays a hung server; a PostgreSQL store with a 2 s
timeout returns an error in about 4 s, and a MySQL store in 30 s.
Without the timeouts both wait for good. store::task_locks gains a
task held for 1.5 lock lifetimes: its lease is still held, and released
when the task ends.
A 3-node rehearsal found taskQueueProcessing didn't filter anything:
roles.task_manager only decided whether the task manager started, and
report, ACME, DKIM, DNS, calendar, thread-merge and restore tasks ran on
any node with a task manager (manager.rs returned true for them). A node
whose role left taskQueueProcessing off still ran them if it indexed or
did maintenance.
Every task type now answers to one ClusterTaskType (task_enabled):
- IndexDocument, UnindexDocument, IndexTrace: searchIndexing
- AccountMaintenance, TenantMaintenance, DestroyAccount:
accountMaintenance
- StoreMaintenance: storeMaintenance
- SpamFilterMaintenance: spamClassifierTraining
- DmarcReport, TlsReport: outboundMta. They build and send reports to
other domains (TLS reports can go straight to an HTTPS endpoint),
which is the outbound MTA's business.
- CalendarAlarmEmail, CalendarAlarmNotification, CalendarItipMessage,
MergeThreads, RestoreArchivedItem, AcmeRenewal, DkimManagement,
DnsManagement: taskQueueProcessing, the role for queue tasks with no
role of their own.
A node that may not run a task leaves it unclaimed (no lock), so a node
that may picks it up. The task manager also starts on a node whose only
task role is outboundMta, so reports still run there.
cluster::task_roles::task_role_tests (new, two task managers over one
PostgreSQL store): node A (taskQueueProcessing only) runs a DNS task and
leaves an unindex task and a TLS report pending; node B (searchIndexing
and outboundMta) comes up and runs those two; a DNS task scheduled next
stays pending on B and runs on A. On main node A runs the TLS report.
A cluster rehearsal (PostgreSQL + NATS) left index tasks pending well
past the one-hour task lock after the node that claimed them was stopped
or killed. The exact cause there isn't confirmed; this closes every path
found in the task manager that stretches a takeover past the lock, or
keeps a task claimed without running it:
- A graceful stop never released the locks it held, so every task the
node had claimed stayed blocked for an hour. The server now tracks the
locks it holds (common::ipc::TaskLocks) and, once the shutdown signal
arrives, stops claiming and releases them before exiting.
- A node that failed to claim a task (another node held it) set its own
local hold for a full lock lifetime from that scan. If the holder
claimed it just after the scan began, or ran on a clock ahead, that
hold ran out a moment before the lock did and was set for another
hour: two hours in all. Such claims are now tried again every five
minutes (a twelfth of the lock lifetime), and the task manager wakes
up for them: before, a node without a coordinator could sleep up to
five minutes past the recheck, or until something else woke it.
- A worker that panicked took its task type down on that node for good,
while the scan kept claiming that type's tasks and failing to hand them
over, re-taking each lock as it expired and so starving every other
node of them. Each batch now runs on a task of its own; a panic is
logged, the batch's locks are released and the worker carries on. A
failed hand-over releases the lock too.
- A claimed task the worker couldn't read, or found gone, kept its lock
for the hour. It is released.
- An IndexDocument task for a file (not indexed) returned no result,
which shifted every later result in the batch onto the wrong task in
update_tasks. It returns Ignored. Nothing queues such a task today.
The lock lifetime stays one hour; it now lives per server so the tests
can shorten it.
store::task_locks::task_lock_tests plays a second node by writing its
locks straight into the in-memory store: tasks it claimed and abandoned
run here once its locks expire, including locks that outlive this node's
view of them, and a graceful stop hands this node's locks back at once
and claims nothing more. It passes on RocksDB, SQLite and PostgreSQL.
With the old recheck it fails.
The trace index task wrote the event type (its name) and the queue id as
text, but the tracing search index types both as integers on every
backend: BIGINT on PostgreSQL and MySQL, long on Elasticsearch. On
PostgreSQL every batch holding a trace document failed with "cannot
convert between the Rust type String and the Postgres type int8", and
since a batch writes trace and email documents together, email indexing
stalled behind it.
The document is now built by trace_search_document(), which writes:
- the event type as the opening event's numeric id, the event
x:Trace/query's event filter already matches on;
- the queue id as an integer, the first one the trace names;
- every queue id into the keywords as well, since the column holds one
value and an SMTP session can queue several messages.
index_keyword() replaced the field on every call, so before this only the
last event type and queue id survived anyway.
x:Trace/query's queueId filter parses the id (a string, or now a number)
and matches the column or the keywords, so a session is found by any of
its queue ids on every backend. The monitoring spec says what is indexed.
Traces indexed before this on the built-in index keep their text values;
the reindexTelemetry maintenance task rebuilds them.
Tests: the search store suite builds trace documents with the index
task's code, indexes them and finds them by queue id, event type and
keyword (Sqlite, PostgreSQL, MySQL); the monitoring suite finds a real
trace by queueId through x:Trace/query.
The broadcast subscriber waited 1 << retry_count.max(6) seconds between
failed subscribe attempts. max(6) turns the cap into a floor: the first
retry waited 64 s instead of 1 s, and each later one doubled without a
bound (and would overflow the shift after enough failures).
The delay now comes from subscribe_retry_delay(), 1 s, 2 s, 4 s ... capped
at 64 s, and the retry counter saturates. A unit test pins the schedule
and the top of the range.
The server fetched upstream's latest published rules from GitHub at run
time: a version nobody here tested, code-like expressions from an account
we don't control, and the upstream name as a default in the admin form.
The published rules of spam-filter v3.0.2 are now embedded
(resources/spam-filter/, MIT, in THIRD-PARTY.md) and used whenever no other
source is configured. An empty setting and upstream's old default both mean
the bundled rules, so existing installs switch without a settings change;
the URL stays an operator override (https:// or file://). The schema default
is dropped and its description says what empty means, and the strip's
rename pass does the same to each import.
Rules load on first boot as before, and again whenever the bundled version
differs from the last one loaded, which only adds missing rules and tags.
That brings the AI classifier's LLM_* scores to installs that predate them:
production has none today.
upstream-watch now also opens an issue when spam-filter publishes a newer
release; resources/spam-filter/README.md says how to take it.
The antispam test now runs on the bundled rules, the path production
takes; SPAM_RULES_URL tests another set. Unit tests cover the URL handling
and that the bundled rules parse and score the AI tags as the AI spec says.
Five conflicts, resolved:
- crates/common/src/auth/authentication.rs: upstream's get_directory_for_token
and JwtClaims replace extract_jwt_domain; the per-domain directory code
(DIR-1, DIR-5 to DIR-7) is kept, and the token lookup routes through it.
The release's one new Enterprise snippet was the body of
get_directory_for_issuer, which stays returning None: a token naming no
address gets the server default, as DIR-2 specifies and as v0.16.22 did.
- crates/common/src/manager/application.rs: upstream's rewrite of the tests,
with the temp directory names renamed again, and the 5(a) notice the
name-purge change should have added.
- crates/common/src/network/mta.rs: both sides' imports.
- crates/main/Cargo.toml: the AGPL-only license kept, version 0.16.23.
- Cargo.lock: upstream's, with the fork's crates added by Cargo.
The AGPL asks a modified version to carry prominent notices saying it was
modified, and giving a date. Publishing the source is the conveyance that
asks for it, so it wants doing before the repository is public rather than
at the release.
Every upstream file the fork changed now says so in its header, beneath the
notice it came with: 164 files, found by diffing against the upstream
snapshot branch rather than by guessing, so the list is what actually
differs. Files the fork wrote itself already carry their own copyright and
need nothing. Upstream's notices are untouched, which its licence requires
and which was already true.
The README says the same thing in prose, since the obligation is on the
work as a whole and not only its Rust files.
Builds unchanged: the server and the test binary both compile.
Every one of the 14 was `#[cfg(not(feature = "enterprise"))]` on the arm the
fork always compiles: the Enterprise arms went with the import, and nothing
turns the feature on. Removing the attribute leaves the same code, now
unconditional, in 11 files.
Two of them looked like behavior worth checking before touching: the
`validate_tenant_quota` stub that always passes, and the refusal to cancel a
pending DestroyAccount task. The stub is vestigial — the rebuilt
multi-tenancy enforces quotas in `crates/features/src/tenancy/quota.rs` for
those objects and more — and the refusal is undelete's open question, which
this change leaves exactly as it was.
The binary builds with no new warnings, and `system_tests` and `jmap_tests`,
which cover the touched registry, task-manager and auth paths, both pass.
The feature definitions stay in the manifests, inert: taking them out would
widen every sync's diff for nothing.
A FETCH with CHANGEDSINCE presents its mod-sequence to the read scope, so
a replica must have that change before it answers, as a JMAP sinceState
already did. replica_cluster_tests covers test 11: with the replica's
replay paused, a write on one node is read back through a second store
with its own marks, sharing through Redis.
Composite stores nest store futures deeply enough to pass rustc's default
query depth once both postgres and redis are compiled in, so the server
crates raise their recursion limit.
A data store with readReplicas becomes a replicated store. Writes,
operator-written SQL and everything outside a read scope go to the
primary. JMAP reads before a request's first write, IMAP LIST, STATUS,
SEARCH, SORT and FETCH, POP3 RETR and TOP, DAV GET, PROPFIND and REPORT,
and blob downloads run in a read scope. Only account data (properties,
indexes, change logs, counters, ACLs, blobs, the search index) is read
from a replica; the registry, in-memory values, the task queue and the
rest stay on the primary.
In a scope, the first read picks a replica round-robin among those up
and under the lag limit, and only if it has every change this node has
written or heard of for the scope's accounts: marks come from write
results, the cluster's state-change broadcasts, a sinceState the client
presents, and, with more than one node, Redis. A write inside the scope
sends the rest of it to the primary. A miss on a replica is looked up on
the primary, and a replica error retries the read there and marks the
replica down.
Each node samples lag every second (WAL positions on PostgreSQL; GTID
sets or Seconds_Behind_Source on MySQL), stops reading from a replica
over 5 s and starts again under 2.5 s, and probes a down replica every
10 s. At startup a replica is left out if it's the primary, isn't
read-only, applies out of commit order, or doesn't show a marker written
to the primary within six tries.
replica_tests (postgres, STORE=PostgreSqlReplicated) runs a primary and a
streaming hot standby in containers: tests 9, 10, 12, 13, 14 and 15 pass.
The push router records which account each subscription belongs to, and
a new Revoke event drops every subscription the account holds, so its
IMAP IDLE, JMAP event streams and WebSockets on this node end. Other
users' subscriptions to its shared mailboxes stay. Test 28 now checks an
open IDLE is ended, and that the account's password over HTTP and its
own API key, both already cached, are refused on the next request.
Every minute the node that calculates metrics evaluates each enabled
x:Alert, read from the registry, with metric('name') or the underscore
name. An alert fires when its condition turns true: it emits
telemetry.alert-event with the rendered message, and queues one email per
recipient through the outbound queue, placeholders filled and
Auto-Submitted set. An alert naming an unknown metric is refused on save.
The shared alerts suite runs; every telemetry suite is un-gated.
A lossy collector subscriber keeps each inbound SMTP session that reached
MAIL FROM and each delivery attempt, info level and above and never raw
I/O, at most 1,000 events with strings cut at 4 KiB, and writes it when
the span closes as an x:Trace under the telemetry key class, scheduling
its indexing. The index task builds a document of event types, queue ids
and keywords when indexTelemetry is on. x:Trace/get derives timestamp,
from, to and size; /query filters by opening event, text, queueId and
time; /set destroys only. The data purge honours holdTracesFor. The shared
tracing and webhook suites run.
Every node writes a sample per metric on metricsCollectionInterval:
counters as the increase since its last sample, gauges always, histograms
as totals when changed. Samples are x:Metric in the registry's encoding
under the telemetry key class, ids time-ordered. x:Metric/get and /query
read them with metric and timestamp filters and full paging, hide what's
past holdMetricsFor, and the data purge deletes it. The is_enterprise
split is gone, so every gauge and histogram is collected and exported, and
queue.count is set from the queue itself. The shared metrics suite runs.
Logos resolve domain, then tenant, then server-wide, then the built-in, with
subdomains finding their domain. GET /logo serves a data-URL image, redirects
to a URL logo without fetching it, sandboxes SVG, and answers 404 when no
custom logo applies. Emails embed the first PNG, JPEG or GIF logo. Logo and
template writes are checked; stored templates are read at send time, always
escaped, and fall back to the built-in with a build warning when they don't
parse. The RSVP page is served byte for byte with a CSP and no-referrer. The
sign-in and RSVP pages load the logo through an image element. MT-22's
session logo follows the chain to the server-wide logo.
Acceptance tests 1 to 17; test 18 written as the ignored branding_compat.
With archiveDeletedAccountsFor set, a destroyed account's record is kept in
the fork subspace with its id, its DestroyAccount task is due at the end of
the period, and its shares are suspended both ways. Its addresses can't be
taken by new accounts, aliases, lists or masks. inbuxa:DeletedAccount/get
lists kept accounts to server and tenant administrators; /set restores one
with a new password (same id, task cancelled, shares reinstated) or destroys
it now. The destroy task also clears undelete's own records.
Acceptance test 14; test 16 written as the ignored undelete_compat.
Files, events and contacts are noted when deleted for good and archived by
the unindex task when retention is on; Sieve scripts are archived at
deletion. A restore goes back to its folder, calendar or address book if it
still exists, takes a free " (restored)" name, comes back inactive for
scripts, and is refused over quota with the item left archived.
Acceptance tests 6, 8 and 12.
Every way of deleting mail for good (JMAP, IMAP expunge, POP3, Trash
emptying, mailbox removal) notes the message's mailboxes and keywords while
archiving is on, fixing its deadline then; when its data is finally removed
it becomes an x:ArchivedItem record, written as upstream writes them, with
its copy held until the deadline. Retention is read at deletion time, so a
change applies at once. Restore puts a message back in the mailboxes it was
in (Trash only if that was all), with its keywords, and removes the record;
over quota it stays archived. x:ArchivedItem/get returns status and
accountId; query filters on type, archivedAt and text; set requests a
restore once or destroys; /changes is a fork addition. Expired items go in
the data purge. The shared account-access rule moves to jmap::inbuxa::access.
system_tests now calls undelete::test, and the archiving gate is gone.
- The recovery administrator (INBUXA_RECOVERY_ADMIN, or STALWART_RECOVERY_ADMIN)
is honored only in bootstrap and recovery mode. On a configured server it's
ignored with a startup warning. Before, it was a standing full-admin login
for as long as the variable stayed set.
- The Enterprise upsell error is replaced by "This feature isn't available in
INBUXA yet" for the features still to be rebuilt.
- Workspace warnings: 25 to 0. cargo fix removed the unused imports. The
seven places where Enterprise code used to plug in keep their parameters,
each with an inbuxa: comment naming the rebuild that uses it again. The
antispam test's mock-server imports are back behind pending-rebuild.
Upstream commit: 474dd0229cb20cf513036619781ed97bd8073c3f
Enterprise-only files removed or emptied: 63
Enterprise-only snippets removed: 117 in 50 files
Dangling module declarations removed: 5
Cargo edits turning enterprise off: 14
Verification: clean
Enterprise feature gates left for rebuilt features: 19 in 18 files
Produced by tools/fork/strip.py. The full report is in docs/fork/strip-reports/ on main.
jcoffey-dev