64 Commits
Author SHA1 Message Date
jcoffey-dev d86e7639ac Merge pull request 'Allowed IPs take the full settings reload after a write' (#42) from fix/allowed-ip-reload into main
ci / fork-checks (push) Successful in 28s
ci / build (push) Successful in 35m32s
2026-09-24 20:33:18 +00:00
jcoffey-dev fcef4b1c3f Allowed IPs take the full settings reload after a write
ci / build (pull_request) Successful in 11m59s
ci / fork-checks (pull_request) Successful in 45s
write_reload_target sent AllowedIp writes to the blocked-IP reload, but
that reload rebuilds only BlockedIps. Allowed IPs are parsed into the
core's security settings (Security::parse), which only a full reload
rebuilds, so an AllowedIp write reported x:settingsReload applied: true
while the change wasn't live until the next full reload.

AllowedIp now maps to the full reload, like the other settings objects;
BlockedIp keeps its targeted reload.

system::auto_reload::settings_reload_tests now creates an allowed IP
over JMAP and checks that is_ip_allowed sees it with no ReloadSettings,
and that destroying it takes it out again. On main it fails ("allowed
IP not in the running settings").
2026-09-24 13:20:47 -07:00
jcoffey-dev 89860aa5cc Merge pull request 'SQL pools time out; task locks are a renewed five-minute lease' (#41) from fix/pool-timeouts into main
ci / build (push) Canceled after 14m20s
ci / fork-checks (push) Successful in 33s
2026-09-24 20:18:56 +00:00
jcoffey-dev 6e50ba25a9 SQL pools time out; task locks are a renewed five-minute lease
ci / fork-checks (pull_request) Successful in 47s
ci / build (pull_request) Successful in 4m58s
A 3-node rehearsal (PostgreSQL + NATS + Garage) found two ways a crash
leaves work stuck:

Pool hangs. The PostgreSQL pool (deadpool) was built with no timeouts,
so a request waited for a free connection, and for one to be opened or
recycled, for as long as it took: forever when the server stopped
answering. MySQL's pool (mysql_async) has no wait timeout at all.

- PostgreSQL: wait 30 s (or the store's timeout if longer), create the
  store's timeout or 15 s (it bounds the whole handshake, where
  tokio-postgres's connect_timeout covers only the TCP connect), recycle
  10 s. The pool config is now always set, not only with
  poolMaxConnections.
- MySQL: every connection is taken through MysqlStore::conn(), which
  gives up after 30 s.
- Both: TCP keepalive after 60 s idle, so a server that vanished
  without closing the connection is noticed in minutes rather than the
  two-hour system default.

The DataStore schema has no pool timeout settings, so these are fixed
defaults; the store's own timeout bounds connecting on PostgreSQL.

Task locks. A task lock lasted an hour, so after a hard crash the dead
node's tasks waited up to an hour and five minutes. The lock is now a
five-minute lease: while this node runs a task, the task manager renews
its lock every third of the lifetime (InMemoryStore::renew_lock, a
compare-and-set on the store backends and SET XX EX on Redis, which
leaves a lock that already expired alone). A killed node's tasks run
elsewhere within about five minutes plus the claim recheck. A task this
node holds isn't handed to a worker again by the scan.

store::pool_timeout (new): a local listener that accepts connections
and never answers plays a hung server; a PostgreSQL store with a 2 s
timeout returns an error in about 4 s, and a MySQL store in 30 s.
Without the timeouts both wait for good. store::task_locks gains a
task held for 1.5 lock lifetimes: its lease is still held, and released
when the task ends.
2026-09-24 13:11:26 -07:00
jcoffey-dev 127ef5701d Merge pull request 'Task manager: every task type follows the node's cluster role' (#40) from fix/task-role-filtering into main
ci / build (push) Canceled after 11m59s
ci / fork-checks (push) Successful in 13s
2026-09-24 20:06:55 +00:00
jcoffey-dev 1543ea5a9e Task manager: every task type follows the node's cluster role
ci / fork-checks (pull_request) Successful in 14s
ci / build (pull_request) Successful in 3m17s
A 3-node rehearsal found taskQueueProcessing didn't filter anything:
roles.task_manager only decided whether the task manager started, and
report, ACME, DKIM, DNS, calendar, thread-merge and restore tasks ran on
any node with a task manager (manager.rs returned true for them). A node
whose role left taskQueueProcessing off still ran them if it indexed or
did maintenance.

Every task type now answers to one ClusterTaskType (task_enabled):

- IndexDocument, UnindexDocument, IndexTrace: searchIndexing
- AccountMaintenance, TenantMaintenance, DestroyAccount:
  accountMaintenance
- StoreMaintenance: storeMaintenance
- SpamFilterMaintenance: spamClassifierTraining
- DmarcReport, TlsReport: outboundMta. They build and send reports to
  other domains (TLS reports can go straight to an HTTPS endpoint),
  which is the outbound MTA's business.
- CalendarAlarmEmail, CalendarAlarmNotification, CalendarItipMessage,
  MergeThreads, RestoreArchivedItem, AcmeRenewal, DkimManagement,
  DnsManagement: taskQueueProcessing, the role for queue tasks with no
  role of their own.

A node that may not run a task leaves it unclaimed (no lock), so a node
that may picks it up. The task manager also starts on a node whose only
task role is outboundMta, so reports still run there.

cluster::task_roles::task_role_tests (new, two task managers over one
PostgreSQL store): node A (taskQueueProcessing only) runs a DNS task and
leaves an unindex task and a TLS report pending; node B (searchIndexing
and outboundMta) comes up and runs those two; a DNS task scheduled next
stays pending on B and runs on A. On main node A runs the TLS report.
2026-09-24 12:46:49 -07:00
jcoffey-dev 4cb42f28f3 Merge pull request 'Registry writes apply to the running settings without ReloadSettings' (#39) from fix/registry-auto-reload into main
ci / fork-checks (push) Successful in 18s
ci / build (push) Canceled after 32m47s
2026-09-24 19:34:08 +00:00
jcoffey-dev 2c684be5c9 Registry writes apply to the running settings without ReloadSettings
ci / fork-checks (pull_request) Successful in 44s
ci / build (pull_request) Successful in 3m21s
A 3-node rehearsal found that saving an MtaDeliverySchedule left it
unknown to the queue ("Queue strategy not found") until someone ran
x:Action ReloadSettings; only Directory and Authentication writes
reloaded (DIR-17). The admin UI has to remember a separate reload after
every save, and a script or API client that doesn't gets a server
running stale settings.

x:<Object>/set now reloads the running settings when it created,
updated or destroyed an object they are built from, and broadcasts the
same RegistryChange::Reload over the coordinator as ReloadSettings, so
every node applies it:

- Settings objects (MTA, spam filter, listeners, tracers, Sieve system
  scripts, cluster roles, directories, ...: the object types the core,
  telemetry, listener and directory builders read) get a full reload.
- Certificates, lookup stores and blocked/allowed IPs get their own
  targeted reloads.
- Accounts, domains, roles and other data read as needed, stores (they
  take a restart) and applications (their own reload action) get none.

Full reloads are coalesced: a write waits for a reload that started
after it was stored and joins one if it can, so a burst of writes, or
a request with many objects, costs one or two reloads, not one each.

The write itself is never undone. When the reload is refused (build
errors in objects that were working, the rule from the previous
commit), the set response says so in a new x:settingsReload field,
{"applied": false, "description": "Saved, but the running settings
were not reloaded. <object>: <error>"}; {"applied": true} otherwise.
The field is absent when the write needs no reload. The description
helper is shared with ReloadSettings' refusal.

Each reload sends the queue a ReloadSettings event, so the SMTP test
harness's read_event, try_read_event and assert_no_events now pass over
those; expect_reload_settings still waits for one.

system::auto_reload::settings_reload_tests (new): an MtaVirtualQueue
and an MtaDeliverySchedule created over JMAP are in the running
settings with no ReloadSettings, and gone once destroyed; eight
concurrent creates all land; a write whose reload fails is stored and
reported applied: false with the error; a domain write carries no
x:settingsReload. On main the new schedule is missing. The cluster
broadcast test (three nodes, PostgreSQL + NATS) now checks that every
node has a schedule created on node 0 without a reload.
2026-09-24 12:30:00 -07:00
jcoffey-dev 19eb25a426 Merge pull request 'Settings reload: no DNS at build time, don't refuse over old failures' (#38) from fix/reload-resilient-build-errors into main
ci / build (push) Canceled after 14m33s
ci / fork-checks (push) Successful in 44s
2026-09-24 19:19:34 +00:00
jcoffey-dev 999ae12cc7 Settings reload: no DNS at build time, don't refuse over old failures
ci / build (pull_request) Successful in 3m22s
ci / fork-checks (pull_request) Successful in 44s
A 3-node rehearsal found every settings reload refused, cluster-wide,
because one node couldn't resolve the Pyzor server:

- PyzorConfig::parse resolved the host while building the settings and
  made a failed lookup a build error. It now keeps the host and port and
  resolves when a message is checked (an IP address is used as is, a
  name is reused for five minutes, the lookup counts against the Pyzor
  timeout). A failure there is a Pyzor error for that message.
- A milter's hostname was resolved the same way, with a blocking
  to_socket_addrs in async code. An IP address is kept; a name is now
  resolved on each connection.

Other build-time I/O is already non-fatal: directories that can't
connect become unavailable with a warning (DIR-21), and the AI model
locality check only warns.

reload_registry swapped the core only when the whole build was free of
errors, while boot runs with whatever built. One failing object thus
refused every later reload, and the running settings went stale. Now a
reload is refused only for errors in objects that built when the
running settings were built (at boot or by the last applied reload):
applying it would lose those. Objects that already failed then are
missing from the running settings anyway, as at boot, so their errors
are logged and returned as known_errors but don't hold the reload back.
Refusing on new errors keeps a bad edit from taking a working object
out of service; the admin gets the error instead.

ReloadSettings now says "Settings were not reloaded." and names the
object and its error ("Tracer with id ...: Only one console tracer is
allowed"), with a count of any further errors. A refused reload after a
directory change logs its errors too.

system::reload::reload_tests (new): with Pyzor enabled on an
unresolvable host, ReloadSettings succeeds (on main it fails with
"Invalid address: failed to lookup address information"); an IP host
needs no lookup; a new build error refuses the reload, names the object
and leaves the running settings unchanged; the same error, once known
from the running settings' build, no longer blocks; once fixed, a new
error there blocks again. smtp::inbound::milter's session test now
names its milter "localhost", so the connect-time lookup is exercised.
2026-09-24 12:11:41 -07:00
jcoffey-dev 5853831bad Merge pull request 'Search: find addresses by local part, domain or name on PostgreSQL and MySQL' (#37) from fix/pg-address-search into main
ci / build (push) Failing after 3s
ci / fork-checks (push) Successful in 15s
2026-09-24 19:11:31 +00:00
jcoffey-dev 639a415a4f Search: find addresses by local part, domain or name on PostgreSQL and MySQL
ci / fork-checks (pull_request) Successful in 43s
ci / build (pull_request) Successful in 4m20s
A 3-node PostgreSQL rehearsal found IMAP SEARCH FROM "noreply" matched
0-2 messages where RocksDB matched 23 of 930. The message indexer hands
each address and display name of From/To/Cc/Bcc to the search store as
keyword text (Language::None). The built-in index splits keyword text
into lowercase runs of alphanumerics, so an address is found by its full
form, its local part, its domain or a display-name word. The SQL
backends didn't:

- PostgreSQL's text parser keeps "[email protected]" as one email
  token (host names and URLs likewise), so neither "noreply" nor
  "amazon.com" ever matched it. Keyword text is now split the same way
  as the built-in index (SpaceTokenizer) before to_tsvector on insert
  and before plainto_tsquery/phraseto_tsquery on search, still under
  the 'simple' configuration, so the GIN index keeps serving the query.
  The sort columns keep the raw text.
- MySQL's FULLTEXT parser already splits on punctuation, but InnoDB
  never indexes its stopwords ("com", "de", "www", ...) or words under
  innodb_ft_min_token_size (3), and a required +word it hasn't indexed
  matches no row. So "amazon.com", "[email protected]" or "jane doe" found
  nothing. Those words are now matched with a word-boundary REGEXP on
  the rows the indexed words select. In language text (bodies,
  subjects) they are dropped when other words remain, and only checked
  when nothing else is left, so "the invoice" no longer finds nothing
  either.

Existing PostgreSQL search indexes hold the old single-token vectors and
need a reindex (the reindexAccounts task) before address searches find
old messages. MySQL needs none: only the query changed.

store::search_tests gains test_address_search: five messages, 28
FROM/TO/CC/BCC searches by full address, local part, domain, domain
labels, display name and hyphenated local part, plus a TEXT-style OR,
with the same expected ids on every backend. It passes on RocksDB,
SQLite, PostgreSQL and MySQL; on main it fails on PostgreSQL (From
"noreply") and MySQL (From "[email protected]").
2026-09-24 11:25:25 -07:00
jcoffey-dev 9311c1a38b Merge pull request 'Coordinator: join the cluster when NATS comes up, report the connection' (#36) from fix/coordinator-retry-and-health into main
ci / build (push) Failing after 3h0m50s
ci / fork-checks (push) Successful in 40s
2026-09-24 15:51:06 +00:00
jcoffey-dev 24be4a1b85 Merge pull request 'Publish amd64 first, then arm64, on a builder that keeps its cache' (#34) from ci/faster-publish into main
ci / fork-checks (push) Successful in 53s
ci / build (push) Canceled after 5m39s
Reviewed-on: #34
2026-09-24 15:45:26 +00:00
jcoffey-dev 95f0445d83 Coordinator: join the cluster when NATS comes up, report the connection
ci / fork-checks (pull_request) Successful in 46s
ci / build (pull_request) Successful in 11m8s
A node that started while NATS was down never got a coordinator. The
connect failed at boot, bootstrap recorded a build error and the node ran
with Coordinator::None until restarted. It had no broadcast subscriber
or publisher, so cross-node push and cache invalidation to it stayed
broken, and its healthcheck said nothing about it. Losing NATS after
startup was silent too.

- The NATS client now connects in the background
  (retry_on_initial_connect): startup never waits on NATS or fails over
  it, the node gets its coordinator, subscriber and publisher at once,
  and the client keeps trying (async-nats's backoff, at most 4 s apart)
  until NATS answers. Subscriptions made meanwhile start delivering when
  it does. A configured maxReconnects still ends the attempts.
- Three new events report the connection: cluster.coordinator-connected
  (info), cluster.coordinator-disconnected (warn: lost, closed, gave up,
  or not connected within the connection timeout at startup) and
  cluster.coordinator-error (warn: a failed attempt, reported once per
  outage rather than every retry, and server errors, slow consumers and
  lame duck mode). They are in the packaged schema, ids 644 to 646.
- GET /healthz/cluster reports the coordinator: 200
  {"coordinator":"connected"}, 503 {"coordinator":"disconnected"}, or
  200 with "none" (no coordinator) or "unknown" (a backend that doesn't
  track its connection). /healthz/live and /healthz/ready are unchanged
  on purpose: a node without its coordinator still serves mail, and
  failing those would have orchestrators restart, or pull out of
  service, every node at once whenever NATS is down.

Only NATS connects lazily; the other coordinator backends still fail at
boot as before.

cluster::coordinator::coordinator_reconnect_tests starts a node against a
NATS port with nothing behind it, checks it boots with a coordinator and
reports it disconnected, subscribes, then starts NATS on that port: the
node connects on its own and the subscription receives a message from a
second client. Stopping and restarting NATS shows disconnected, then
connected, and the same subscription keeps working.
2026-09-24 08:38:59 -07:00
jcoffey-dev c974a0918e Merge pull request 'Task manager: release task locks on stop, recheck claims held elsewhere' (#35) from fix/task-lock-recovery into main
ci / build (push) Canceled after 6m47s
ci / fork-checks (push) Successful in 49s
2026-09-24 15:38:39 +00:00
jcoffey-dev 7c80a12d75 Task manager: release task locks on stop, recheck claims held elsewhere
ci / build (pull_request) Successful in 17m2s
ci / fork-checks (pull_request) Successful in 18s
A cluster rehearsal (PostgreSQL + NATS) left index tasks pending well
past the one-hour task lock after the node that claimed them was stopped
or killed. The exact cause there isn't confirmed; this closes every path
found in the task manager that stretches a takeover past the lock, or
keeps a task claimed without running it:

- A graceful stop never released the locks it held, so every task the
  node had claimed stayed blocked for an hour. The server now tracks the
  locks it holds (common::ipc::TaskLocks) and, once the shutdown signal
  arrives, stops claiming and releases them before exiting.
- A node that failed to claim a task (another node held it) set its own
  local hold for a full lock lifetime from that scan. If the holder
  claimed it just after the scan began, or ran on a clock ahead, that
  hold ran out a moment before the lock did and was set for another
  hour: two hours in all. Such claims are now tried again every five
  minutes (a twelfth of the lock lifetime), and the task manager wakes
  up for them: before, a node without a coordinator could sleep up to
  five minutes past the recheck, or until something else woke it.
- A worker that panicked took its task type down on that node for good,
  while the scan kept claiming that type's tasks and failing to hand them
  over, re-taking each lock as it expired and so starving every other
  node of them. Each batch now runs on a task of its own; a panic is
  logged, the batch's locks are released and the worker carries on. A
  failed hand-over releases the lock too.
- A claimed task the worker couldn't read, or found gone, kept its lock
  for the hour. It is released.
- An IndexDocument task for a file (not indexed) returned no result,
  which shifted every later result in the batch onto the wrong task in
  update_tasks. It returns Ignored. Nothing queues such a task today.

The lock lifetime stays one hour; it now lives per server so the tests
can shorten it.

store::task_locks::task_lock_tests plays a second node by writing its
locks straight into the in-memory store: tasks it claimed and abandoned
run here once its locks expire, including locks that outlive this node's
view of them, and a graceful stop hands this node's locks back at once
and claims nothing more. It passes on RocksDB, SQLite and PostgreSQL.
With the old recheck it fails.
2026-09-24 08:19:05 -07:00
jcoffey-dev 22d8ad8572 Publish amd64 first, then arm64, on a builder that keeps its cache
ci / fork-checks (pull_request) Successful in 49s
ci / build (pull_request) Successful in 4m30s
Two release builds side by side on one machine each take twice as long,
and production only needs amd64. publish-amd64 now pushes :<version> as
soon as the amd64 build is done; publish-arm64 builds arm64 afterwards,
then replaces :<version> with the two-platform index and moves :latest.

Both jobs use one named BuildKit builder whose container outlives the
job, so the dependency layer (cargo chef cook) is reused until the
dependencies change. The release is created after amd64; the binaries
are attached once arm64 is in.
2026-09-24 08:04:53 -07:00
jcoffey-dev 7109e67f07 Merge pull request 'Trace search: index event type and queue id as integers' (#33) from fix/pg-index-trace-types into main
ci / fork-checks (push) Successful in 30s
ci / build (push) Successful in 37m5s
2026-09-24 14:57:18 +00:00
jcoffey-dev 52b5a5f909 Mark tests/src/store/query.rs as modified by the fork
ci / fork-checks (pull_request) Successful in 1m4s
ci / build (pull_request) Successful in 4m20s
The trace document test changed an upstream file, so it carries the
AGPL section 5(a) notice (tools/fork/notice-check.py).
2026-09-24 07:39:04 -07:00
jcoffey-dev 9232662913 Trace search: index event type and queue id as integers
ci / fork-checks (pull_request) Failing after 47s
ci / build (pull_request) Successful in 4m55s
The trace index task wrote the event type (its name) and the queue id as
text, but the tracing search index types both as integers on every
backend: BIGINT on PostgreSQL and MySQL, long on Elasticsearch. On
PostgreSQL every batch holding a trace document failed with "cannot
convert between the Rust type String and the Postgres type int8", and
since a batch writes trace and email documents together, email indexing
stalled behind it.

The document is now built by trace_search_document(), which writes:

- the event type as the opening event's numeric id, the event
  x:Trace/query's event filter already matches on;
- the queue id as an integer, the first one the trace names;
- every queue id into the keywords as well, since the column holds one
  value and an SMTP session can queue several messages.

index_keyword() replaced the field on every call, so before this only the
last event type and queue id survived anyway.

x:Trace/query's queueId filter parses the id (a string, or now a number)
and matches the column or the keywords, so a session is found by any of
its queue ids on every backend. The monitoring spec says what is indexed.

Traces indexed before this on the built-in index keep their text values;
the reindexTelemetry maintenance task rebuilds them.

Tests: the search store suite builds trace documents with the index
task's code, indexes them and finds them by queue id, event type and
keyword (Sqlite, PostgreSQL, MySQL); the monitoring suite finds a real
trace by queueId through x:Trace/query.
2026-09-24 07:29:08 -07:00
jcoffey-dev 57d1c5b074 Merge pull request 'Broadcast subscriber: fix the inverted subscribe retry backoff' (#32) from fix/subscriber-backoff into main
ci / fork-checks (push) Successful in 43s
ci / build (push) Canceled after 30m24s
2026-09-24 14:26:52 +00:00
jcoffey-dev ca3abf40f0 Broadcast subscriber: fix the inverted subscribe retry backoff
ci / fork-checks (pull_request) Successful in 56s
ci / build (pull_request) Successful in 5m14s
The broadcast subscriber waited 1 << retry_count.max(6) seconds between
failed subscribe attempts. max(6) turns the cap into a floor: the first
retry waited 64 s instead of 1 s, and each later one doubled without a
bound (and would overflow the shift after enough failures).

The delay now comes from subscribe_retry_delay(), 1 s, 2 s, 4 s ... capped
at 64 s, and the retry counter saturates. A unit test pins the schedule
and the top of the range.
2026-09-24 07:01:56 -07:00
jcoffey-dev 499e4d7810 Merge pull request 'Export/import: keep archived items, spam samples and the spam model' (#31) from fix/export-all-subspaces into main
ci / fork-checks (push) Successful in 3m24s
ci / build (push) Successful in 43m50s
2026-09-23 08:50:16 +00:00
jcoffey-dev 212cd77cd3 Export/import: keep archived items, spam samples and the spam model
ci / fork-checks (pull_request) Successful in 32s
ci / build (pull_request) Successful in 7m57s
--export skipped three things, so a move from one database to another
(RocksDB to PostgreSQL, say) lost them without a word:

- archived items (subspace j), the records behind undelete;
- spam training samples (subspace w);
- the trained spam classifier and its trainer state, blobs stored under
  fixed names that no blob link points at, so the walk over links never
  reached them.

j and w now travel with the registry family, where their indexes and id
counters already were, so EXPORT_TYPES=registry keeps them consistent.
The two named blobs travel with the blob family. The file format is
unchanged and import reads any subspace it is given, so an export made
by an older binary still imports.

The full-text index (subspace z) stays out, on purpose. It belongs to one
search backend: PostgreSQL and MySQL index into their own tables and have
no z table at all, and external engines keep the index themselves. So
--import now returns the subspaces it wrote, and boot queues the
reindexAccounts and reindexTelemetry store maintenance tasks, the same
ones an administrator can queue by hand, to rebuild the index for
whichever search store the server runs with once it starts.

The round trip also turned up a loss in import itself: the SQL stores
add a negative amount with an UPDATE, which does nothing to a row that
isn't there yet, so every negative counter or quota vanished on import
into PostgreSQL, MySQL or SQLite. Import now creates the row first.

The in-memory subspaces (m, y) stay out: rate limits, locks, greylisting,
ACME challenge tokens and OAuth codes, all short-lived. Issued
certificates are registry objects and travel.

The store test now writes archived items, spam samples, directory
entries, the fork's own subspace and the named blobs, checks they come
back in place, then imports the same export into a fresh store of the
other local backend (RocksDB to SQLite, or SQLite to RocksDB), compares
it key for key and counter for counter, and checks the queued reindex.
It fails on the old export code ("Subspace j was not exported").
--help now says what an export holds.
2026-09-23 01:41:28 -07:00
jcoffey-dev 7735780807 Merge pull request 'Image build: put the vendored crate where cargo chef cooks; release 2026.9.24.3' (#30) from fix/image-vendor-before-cook into main
ci / fork-checks (push) Successful in 41s
publish / version (push) Successful in 39s
ci / build (push) Successful in 36m33s
publish / publish (push) Successful in 1h2m24s
publish / release (push) Successful in 2s
publish / binaries (push) Successful in 1m8s
Reviewed-on: #30
2026-09-23 06:10:46 +00:00
jcoffey-dev e223f7d327 Release 2026.9.24.3
ci / fork-checks (pull_request) Successful in 45s
ci / build (pull_request) Successful in 4m26s
2026.9.24.3 is 2026.9.24.2 plus the image build fix; 2026.9.24.2's tag never
published an image. Everything in 2026.9.24.2's notes applies.
2026-09-22 23:04:59 -07:00
jcoffey-dev 30df055e39 Image build: put the vendored crate where cargo chef cooks
#27 let the build context see vendor/, but the Dockerfile cooks the
dependencies before it copies the tree, from a recipe that carries only the
workspace's manifests. [patch.crates-io] points sieve-rs at vendor/, so the
cook failed the same way: failed to read /build/vendor/sieve-rs/Cargo.toml.
That's why 2026.9.24.2's publish failed. The builder stage now copies
vendor/ before cooking; a local build got past it into compiling the
dependencies.

context-check.py now also checks that each patched path is copied into the
cooking stage before the cook, and fails on the Dockerfile as it was.
2026-09-22 23:04:50 -07:00
jcoffey-dev 5393c4405a Merge pull request 'Release 2026.9.24.2' (#29) from release/2026.9.24.2 into main
ci / fork-checks (push) Successful in 50s
publish / version (push) Successful in 24s
publish / publish (push) Failing after 2m38s
publish / release (push) Skipped
publish / binaries (push) Skipped
ci / build (push) Successful in 22m35s
Reviewed-on: #29
2026-09-23 05:47:10 +00:00
jcoffey-dev cc532b914c Release 2026.9.24.2
ci / fork-checks (pull_request) Successful in 1m49s
ci / build (pull_request) Successful in 4m8s
Replaces 2026.9.24, whose tag predates the image build fix (#27) and never
published. Carries everything 2026.9.24 did -- upstream 0.16.23 and its
fixes, the scim release-profile fix -- and since then:

- identifiers renamed from the upstream name, with no aliases: the JMAP
  registry capability is urn:inbuxa:jmap:registry, WebDAV tokens
  urn:inbuxa:dav*, Sieve extensions vnd.inbuxa.*, the web interface client
  inbuxa-webui; INBUXA_* settings only. Deploy with admin and webmail
  releases that use the new names.
- the brand in lowercase where people see it.
- the spam filter rules bundled with the server; on first start they add
  the AI classifier's LLM_* scores.
- a Local AI page link in Settings › Spam Filter, for the admin release
  that draws it.
- two start-up migrations: the spam model moves to its renamed keys, and
  the web interface's old OAuth client is retired.
2026-09-22 22:40:17 -07:00
jcoffey-dev c09eff2214 Merge pull request 'Bundle the spam filter rules with the server, and link the Local AI page' (#28) from fork/bundled-spam-rules into main
ci / fork-checks (push) Successful in 20s
ci / build (push) Canceled after 7m16s
Reviewed-on: #28
2026-09-23 05:39:50 +00:00
jcoffey-dev eba4c7a32e Settings › Spam Filter gains Local AI, the AI spam filtering setup page
ci / fork-checks (pull_request) Successful in 14s
ci / build (pull_request) Successful in 4m23s
Adds a link to CustomComponent/LocalAi in the packaged schema's Settings ›
Spam Filter, above LLM Classifier, and updates the schema hash so admins
fetch the new layout rather than a cached one.

INBUXA Admin draws the page (feature/local-ai-setup); this makes it
reachable. An admin from before that page would show "Unknown component"
here, so this lands after the admin release that carries it.
2026-09-22 22:08:50 -07:00
jcoffey-dev 17426f6d60 Bundle the spam filter rules with the server
The server fetched upstream's latest published rules from GitHub at run
time: a version nobody here tested, code-like expressions from an account
we don't control, and the upstream name as a default in the admin form.

The published rules of spam-filter v3.0.2 are now embedded
(resources/spam-filter/, MIT, in THIRD-PARTY.md) and used whenever no other
source is configured. An empty setting and upstream's old default both mean
the bundled rules, so existing installs switch without a settings change;
the URL stays an operator override (https:// or file://). The schema default
is dropped and its description says what empty means, and the strip's
rename pass does the same to each import.

Rules load on first boot as before, and again whenever the bundled version
differs from the last one loaded, which only adds missing rules and tags.
That brings the AI classifier's LLM_* scores to installs that predate them:
production has none today.

upstream-watch now also opens an issue when spam-filter publishes a newer
release; resources/spam-filter/README.md says how to take it.

The antispam test now runs on the bundled rules, the path production
takes; SPAM_RULES_URL tests another set. Unit tests cover the URL handling
and that the bundled rules parse and score the AI tags as the AI spec says.
2026-09-22 22:01:30 -07:00
jcoffey-dev d7c9416713 Merge pull request 'Let the image build see the dependency Cargo patches' (#27) from fix/vendor-in-build-context into main
ci / fork-checks (push) Successful in 14s
ci / build (push) Successful in 31m11s
2026-09-23 04:48:35 +00:00
jcoffey-dev 238079da66 Let the image build see the dependency Cargo patches
ci / fork-checks (pull_request) Successful in 49s
ci / build (pull_request) Successful in 4m22s
The rename pass vendored a patched sieve-rs and pointed Cargo.toml's
[patch.crates-io] at vendor/sieve-rs. .dockerignore ignores everything and
re-includes a short list that did not have vendor on it, so the image build
had no such directory and stopped at

    failed to load source for dependency `sieve-rs`
    failed to read /build/vendor/sieve-rs/Cargo.toml

CI could not have caught that: it builds from a checkout, where the
directory is simply there, and only the image build has a context to prune.
The first that was known about it was a tag that had already been pushed.

So: vendor is re-included, and tools/fork/context-check.py now asserts the
thing that was quietly assumed -- every path a [patch] section names exists
and survives .dockerignore. It runs beside the other fork checks and takes
no toolchain.

Also, the comments in .dockerignore started with // , which Docker does not
read as a comment: they were patterns that happened to match nothing. They
are # now.
2026-09-22 21:43:37 -07:00
jcoffey-dev 0d8caaa514 Merge pull request 'Compile the scim crate in release, and check that profile in CI' (#26) from fix/scim-recursion-limit into main
ci / fork-checks (push) Successful in 1m11s
publish / version (push) Successful in 58s
publish / publish (push) Failing after 34s
publish / release (push) Skipped
publish / binaries (push) Skipped
ci / build (push) Canceled after 25m24s
2026-09-23 04:23:07 +00:00
jcoffey-dev ce6882fe93 Merge pull request 'queue_retry test: measure retries from when each attempt started' (#25) from fix/queue-retry-test into main
ci / fork-checks (push) Canceled after 1m30s
ci / build (push) Canceled after 1m30s
Reviewed-on: #25
2026-09-23 04:21:37 +00:00
jcoffey-dev 3df042e7d4 Compile the scim crate in release, and check that profile in CI
ci / name-check (pull_request) Successful in 3m31s
ci / build (pull_request) Successful in 7m28s
v2026.9.24 was tagged on a commit CI had passed, and its release build could
not compile crates/scim at all:

  error: queries overflow the depth limit!
    = note: query depth increased by 130 when computing layout of
      {async fn body of context::<impl ...>::writable_domain()}

The crate is ours, and the failure is profile-dependent: the release profile
computes those async fn layouts in one go and goes past rustc's default query
depth, while the dev profile never gets that far. CI builds dev, so CI was
green on a commit that could not be released. The tag produced no image and
no release, which is the one merciful part.

Two changes:

- #![recursion_limit = "256"] on the crate, which is what rustc itself
  suggests, with a note saying why it only shows up in release. Proved by
  building -p scim in release locally: it now finishes.

- CI builds the release profile too, on pushes to main. Pull requests stay
  on dev, where the wait is worth less. A few minutes per merge is cheaper
  than learning this from a tag, which throws away a multi-architecture
  build and leaves a version half-cut.
2026-09-22 21:13:50 -07:00
jcoffey-dev 64385007c1 Merge pull request 'Fix the antispam test: pin the rules it scores against, stop live Pyzor' (#24) from fix/antispam-test into main
ci / fork-checks (push) Successful in 2m4s
ci / build (push) Successful in 6m26s
Reviewed-on: #24
2026-09-23 04:12:40 +00:00
jcoffey-dev 4d794c6a65 Merge pull request 'Fork/brand lowercase' (#23) from fork/brand-lowercase into main
ci / fork-checks (push) Successful in 15s
ci / build (push) Canceled after 31s
Reviewed-on: #23
2026-09-23 04:12:06 +00:00
jcoffey-dev a404ca89f0 Merge pull request 'Fork/rename upstream identifiers' (#22) from fork/rename-upstream-identifiers into main
ci / fork-checks (push) Successful in 17s
ci / build (push) Canceled after 27s
Reviewed-on: #22
2026-09-23 04:11:34 +00:00
jcoffey-dev 79f54add2f Merge pull request 'Fork tooling: a build check and a rename pass in the strip, a notice check in CI' (#21) from fork/strip-build-check-and-notices into main
ci / fork-checks (push) Successful in 48s
ci / build (push) Canceled after 1m2s
Reviewed-on: #21
2026-09-23 04:10:32 +00:00
jcoffey-dev 86bf2432a2 queue_retry test: measure retries from when each attempt started
ci / name-check (pull_request) Successful in 2m30s
ci / build (pull_request) Successful in 7m33s
The server sets a deferred recipient's next retry from its clock when the
attempt defers, in whole seconds. The test subtracted its own clock taken
when the loop next saw the message, after saving and reporting, so
whenever that lag crossed a second boundary the 2 s retry measured 1 s
and the test failed. Under load, after the other SMTP tests, that was
most runs.

It now measures from when the test started the attempt, which the
server's deferral can only follow, by under a second: each retry is its
interval or one more. Each position is still checked against its own
interval, so a wrong schedule still fails.
2026-09-22 20:55:42 -07:00
jcoffey-dev 5be578ba3c antispam test: run it serially, like the tests sharing its port
ci / name-check (pull_request) Successful in 17s
ci / build (pull_request) Successful in 7m11s
It listens on HTTP 19048, as the dkim2 DSN and report tests do. Those are
marked serial, this wasn't, so when the SMTP tests ran together (as
upstream's CI runs them) it could start beside them and requests reached
whichever server had the port: missing JMAP creates here, and 'You must
authenticate first' in dkim2_dsn_is_signed.
2026-09-22 20:55:42 -07:00
jcoffey-dev f32992ca36 Fix the antispam test: pin the rules it scores against, stop live Pyzor
ci / name-check (pull_request) Successful in 17s
ci / build (pull_request) Successful in 7m12s
It failed everywhere but upstream's machines, for two reasons:

- The spam rules, which carry every score, came from a path on an
  upstream developer's own disk. Without SPAM_RULES_URL none loaded, every
  score was 0.00 and the combined case came out ham instead of spam at
  13.70. The published rules of spam-filter v3.0.2 are now pinned beside
  the test cases (Apache-2.0 or MIT, taken as MIT; in THIRD-PARTY.md).
  SPAM_RULES_URL still overrides.
- The first combined case expects a Pyzor hit, and its digest (that of an
  empty body) wasn't among the three the test mode answers, so it went to
  a public Pyzor server: it failed offline and would drift with that
  server's counts. Test mode now answers every digest from a fixed table,
  with the empty body's added, and never reaches the network.

The test passes online and offline, alone and with the rest of the SMTP
tests. queue_retry, unrelated, still fails when it runs after the others
in one process, though it passes alone every time.
2026-09-22 20:22:29 -07:00
jcoffey-dev a993f9ab01 Write the brand in lowercase where people see it
ci / fork-checks (pull_request) Successful in 47s
ci / build (pull_request) Successful in 5m4s
The name is inbuxa, lowercase, like the wordmark; INBUXA reads as an
acronym. The admin and webmail already changed. Here that's everything
the server shows people: the brand macro behind the protocol greetings,
the HTTP and SCIM realms, the startup banner and the calendar and contact
PRODID; the first-party OAuth client descriptions; the legacy-protocol
refusals; the default calendar and address book names and the SMTP
greeting default, in the code and the schema served to the admin
(checksum regenerated); startup and shutdown events; the User-Agent;
the sign-in and RSVP pages; the service units; the OpenAPI realm; the
crate descriptions and the README, where it's set in bold.

Identifiers that are uppercase for their own reasons stay: INBUXA_*
settings, SUBSPACE_INBUXA. So do code comments and the AGPL 5(a) notice
lines.

Tests follow: the IMAP ID name, the default collection names, the PRODID
in the iTIP fixtures and the CalDAV free-busy expectations, and the e2e
legacy-protocol refusals. The webdav, imap and jmap suites pass, so do
the unit tests of every crate touched, and 73 of 75 SMTP tests; of the
other two, antispam fails on main too, and queue_retry is a timing flake
that passes on its own.
2026-09-22 20:08:10 -07:00
jcoffey-dev 99096cdc9b Merge pull request 'Release 2026.9.24' (#20) from release/2026.9.24 into main
ci / name-check (push) Successful in 1m6s
publish / version (push) Successful in 1m3s
ci / build (push) Successful in 4m38s
publish / publish (push) Failing after 24m24s
publish / release (push) Skipped
publish / binaries (push) Skipped
2026-09-23 02:53:03 +00:00
jcoffey-dev 96ac70ad28 Release 2026.9.24
ci / name-check (pull_request) Successful in 15s
ci / build (pull_request) Successful in 7m10s
Carries upstream 0.16.23 -- the DSN, POP3, Sieve, DMARC-report, ACME and
DNSSEC-resolver fixes in its own change log -- with the files it changed
marked under AGPL section 5(a), and one upstream test dropped that the fork's
routing makes meaningless.

It is also the first release whose tag attaches binaries: a host install can
now fetch inbuxa-linux-amd64.tar.gz or inbuxa-linux-arm64.tar.gz instead of
pulling the image and copying the file out of it.
2026-09-22 19:45:19 -07:00
jcoffey-dev 835b278e66 Merge pull request 'Attach binaries to a release, for installs that are not containers' (#19) from release/binaries into main
ci / name-check (push) Successful in 44s
ci / build (push) Successful in 4m34s
2026-09-23 02:36:53 +00:00
jcoffey-dev cc6f1eb298 Rename the identifiers that carried the upstream name
ci / fork-checks (pull_request) Successful in 16s
ci / build (pull_request) Successful in 7m53s
Everything clients, users and operators meet now carries the fork's name,
with no aliases (SPEC.md §2.4, changed here from "protocol identifiers
stay"):

- JMAP: upstream's registry capability is urn:inbuxa:jmap:registry, beside
  the fork's own urn:inbuxa:jmap.
- WebDAV lock and sync tokens are urn:inbuxa:dav*; clients resync once.
- Sieve: vnd.inbuxa.while and vnd.inbuxa.expressions. sieve-rs spells these
  into its compiler, so it's vendored (vendor/sieve-rs, 0.7.3) and patched in;
  a unit test fails if Cargo.lock ever moves past the vendored copy. The
  trusted runtime now names itself too, rather than answering sieve-rs's
  default.
- The web interface's OAuth client is inbuxa-webui. On every start the old
  stalwart-webui client is removed and any application naming it is moved
  over.
- The spam filter's blobs are INBUXA_SPAM_*; every start moves any left
  under the old keys, so a trained model survives.
- SQL stores and log files default to inbuxa, in the code and in the
  schema served to the admin (checksum regenerated).
- Settings are INBUXA_* only. A STALWART_* variable that's set where its
  INBUXA_* one isn't stops the server at startup, naming it.
- The version-upgrade messages link docs.inbuxa.org's migration page, and
  the OpenAPI description, smtp crate metadata and web-push test fixtures
  lose the name.

Kept on purpose, allowlisted with reasons: the OAuth key-derivation
contexts (renaming them would end every session and invalidate every
sealed client id) and the hashed application prefix.

Also fixes a latent start-up failure: ensure_client updated an existing
first-party client with a revision of 0, which the registry's assertion
never matches, so adding a redirect URI or changing the webmail secret
failed start-up. And the principal session test now expects
legacyProtocols (C-1, added 2026-09-21), which it had missed.

Tested: the server builds without warnings; common's 106 unit tests,
including the vendoring check; a new integration test for the two
start-up migrations; and the webdav, jmap, imap and SMTP Sieve suites.
2026-09-22 19:33:02 -07:00
jcoffey-dev 674ae5d037 Attach binaries to a release, for installs that are not containers
ci / name-check (pull_request) Successful in 43s
ci / build (pull_request) Successful in 5m25s
A release published an image and nothing else, so there was nothing for a
host install to download -- the only way to get the binary was to pull the
image and copy it out, which makes "install without Docker" depend on
Docker.

Each release now carries inbuxa-linux-amd64.tar.gz, inbuxa-linux-arm64.tar.gz
and SHA256SUMS, named as stalwart-migrator's are.

They are taken out of the image this pipeline just pushed rather than
compiled again. A second Rust build per architecture is the slowest thing
here, and it would leave two artifacts that are meant to be the same build
and only probably are. Extracting makes that identity a fact: the binary in
the tarball is the file the image runs. `docker create` starts nothing, so
copying a file out of an arm64 image on an amd64 runner needs no emulation.

One thing the extraction cannot carry: the image grants the binary
cap_net_bind_service, and a tar archive does not keep that xattr. The
release body says so, and says what to do instead -- setcap, or
AmbientCapabilities in the unit -- because a server that cannot bind 25 and
does not say why is a bad first hour.

Checked by hand against v2026.9.23 before this landed: both architectures
extract to the right ELF, and the amd64 binary runs on a bare Debian 13 with
every library resolved and reports its own version.
2026-09-22 19:31:05 -07:00
jcoffey-dev 4799d191a0 Fork tooling: a build check and a rename pass in the strip, a notice check in CI
ci / fork-checks (pull_request) Successful in 18s
ci / build (pull_request) Successful in 7m11s
strip.py compiles the stripped tree, so a dual-licensed file that only
serves an Enterprise feature fails the import instead of the merge, as
v0.16.23's tests/src/directory/issuer.rs does. Upstream's tests of the
features the fork rebuilt are expected not to compile there and are listed
in build-check-known.txt; an error anywhere else fails the run. Checked
against both imports: v0.16.22 passes with its 16 expected errors, v0.16.23
fails on issuer.rs alone. Imports the strip leaves unused are reported.

It also renames the upstream name where clients, users or operators meet
it as an identifier, from tools/fork/renames.py: wire-protocol names, the
web interface's client id, store keys, configuration defaults and the
served schema. main is renamed with the same module, so a re-import
arrives purged and those lines don't conflict.

notice-check.py fails CI when an upstream file the fork changed, measured
against the upstream branch, lacks its AGPL 5(a) notice; --fix adds it.
It runs beside the name check in a renamed fork-checks job.

Also commits v0.16.23's strip report under docs/fork/strip-reports/, which
the import in #18 left out.
2026-09-22 19:02:34 -07:00
jcoffey-dev c5bf67f1bf Merge pull request 'Merge/upstream v0.16.23' (#18) from merge/upstream-v0.16.23 into main
ci / name-check (push) Successful in 19s
ci / build (push) Successful in 37m41s
Reviewed-on: #18
2026-09-23 00:48:23 +00:00
jcoffey-dev c240946248 Drop upstream's issuer-routing test and an import it left unused
ci / name-check (pull_request) Successful in 52s
ci / build (pull_request) Successful in 21m31s
tests/src/directory/issuer.rs, new in v0.16.23, tests routing a bearer token
to a directory by its issuer. That routing is Enterprise-only upstream (the
body of get_directory_for_issuer), and the fork doesn't build it: a token
naming no address gets the server default (DIR-2). The test also calls a
helper from upstream's Enterprise-only OIDC test, so it can't compile here.

mta.rs imported types::id::Id for code inside an Enterprise snippet; the
stripped tree leaves it unused, upstream's as well as ours.
2026-09-22 17:05:52 -07:00
jcoffey-dev ee4988e00d Mark eight more changed files (AGPL section 5(a))
These upstream files were changed after the fork marked the files it had
modified, and never got the notice: six by the listener and schema-cache
work on 2026-09-20, two by the name check. Found by diffing against the
upstream snapshot branch, as before.
2026-09-22 16:57:15 -07:00
jcoffey-dev b2ded0a776 Merge upstream v0.16.23
Five conflicts, resolved:

- crates/common/src/auth/authentication.rs: upstream's get_directory_for_token
  and JwtClaims replace extract_jwt_domain; the per-domain directory code
  (DIR-1, DIR-5 to DIR-7) is kept, and the token lookup routes through it.
  The release's one new Enterprise snippet was the body of
  get_directory_for_issuer, which stays returning None: a token naming no
  address gets the server default, as DIR-2 specifies and as v0.16.22 did.
- crates/common/src/manager/application.rs: upstream's rewrite of the tests,
  with the temp directory names renamed again, and the 5(a) notice the
  name-purge change should have added.
- crates/common/src/network/mta.rs: both sides' imports.
- crates/main/Cargo.toml: the AGPL-only license kept, version 0.16.23.
- Cargo.lock: upstream's, with the fork's crates added by Cargo.
2026-09-22 16:57:06 -07:00
jcoffey-dev 3a272096c0 Import upstream v0.16.23, stripped
trivy / Check (pull_request) Canceled after 0s
Upstream commit: 9d1c75ab68435e4417337f768291e5f947686203
Enterprise-only files removed or emptied: 63
Enterprise-only snippets removed: 118 in 50 files
Dangling module declarations removed: 5
Edits turning enterprise off: 25
Third-party code: 14 files, 0 not in THIRD-PARTY.md
Verification: clean

One snippet more than v0.16.22, in crates/common/src/auth/authentication.rs
(3, was 2).
2026-09-22 16:31:25 -07:00
jcoffey-dev b6660554e6 Merge pull request 'CI: open an issue when upstream publishes a release not yet imported' (#15) from ci/upstream-watch into main
ci / name-check (push) Successful in 17s
ci / build (push) Successful in 7m14s
Reviewed-on: #15
2026-09-22 23:23:02 +00:00
jcoffey-dev 7bda874230 Merge pull request 'CI: fail when the upstream name appears in a new string literal' (#16) from ci/name-check into main
ci / name-check (push) Successful in 1m11s
ci / build (push) Canceled after 3m26s
Reviewed-on: #16
2026-09-22 23:19:37 +00:00
jcoffey-dev a4b091578d CI: fail when the upstream name appears in a new string literal
ci / name-check (pull_request) Successful in 1m15s
ci / build (pull_request) Successful in 5m2s
tools/fork/name-check.py reads every string literal in crates/ (comments
and test directories skipped) and fails on any that carries the upstream
name without an entry in name-allowlist.txt. An upstream merge can bring
such strings in without a conflict, so it runs on every push and PR.

The first run found three the earlier sweeps missed, fixed here: the SMTP
HELP reply pointed at upstream's website (now brand_url!), the event
collector thread was named after upstream, and the FreeBSD default data
path still said /var/db/stalwart/ where Linux already had /var/lib/inbuxa/.

Two operator-visible defaults are allowlisted as open, pending a decision:
the log file prefix and the SQL stores' default database and user.
2026-09-22 16:12:47 -07:00
jcoffey-dev 39df888412 CI: open an issue when upstream publishes a release not yet imported
ci / build (pull_request) Successful in 7m22s
Reads metadata only: upstream's releases list from GitHub's API and the
head of the upstream branch from Gitea's. Nothing of upstream's is
fetched, so its history can't land here. Daily at 06:17 UTC.
2026-09-22 15:37:34 -07:00
jcoffey-dev 697f647f8b Merge pull request 'Release 2026.9.23' (#14) from release/2026.9.23 into main
publish / version (push) Successful in 14s
ci / build (push) Successful in 7m8s
publish / publish (push) Successful in 47m16s
publish / release (push) Successful in 2s
2026-09-22 20:34:46 +00:00
jcoffey-dev 14250cee03 Release 2026.9.23
ci / build (pull_request) Successful in 7m10s
Carries the version string and user-visible string fixes: nothing a user or
operator sees names the upstream project any more.
2026-09-22 13:27:05 -07:00
jcoffey-dev cea3d53eb0 Delete .gitlab-ci.yml
ci / build (push) Successful in 4m18s
2026-09-22 20:26:17 +00:00
312 changed files with 27517 additions and 1386 deletions
+8 -2
View File
@@ -1,10 +1,16 @@
// Ignore everything # Ignore everything
* *
// Allow what is needed # Allow what is needed
!crates !crates
!tests !tests
!resources !resources
# The patched dependency Cargo.toml's [patch.crates-io] points at. Without
# it the build context has no vendor/, and `cargo chef cook` fails on
# "failed to load source for dependency sieve-rs" -- which CI cannot see,
# because CI builds from a checkout and only the image build has a context.
!vendor
!Cargo.lock !Cargo.lock
!Cargo.toml !Cargo.toml
+31
View File
@@ -20,6 +20,27 @@ concurrency:
cancel-in-progress: true cancel-in-progress: true
jobs: jobs:
# What an upstream merge can bring in or leave behind without a conflict:
# the upstream name in a new string literal, and a changed upstream file
# without the AGPL 5(a) notice. Seconds, and needs no toolchain. The notice
# check diffs against the upstream snapshot branch, hence the full fetch.
fork-checks:
runs-on: light
container:
image: python:3.13-slim@sha256:8d9d0b8bcf6506481eae4907c18f5e3e7902e629f5f6d684f9e7c32e85e3ddf0 # 3.13-slim
steps:
- uses: coffey-labs/actions/checkout@fab0c4d45e0162963965f1555df27b7bed5e20ec
with:
fetch-depth: 0
- run: python3 tools/fork/name-check.py
- if: always()
run: python3 tools/fork/notice-check.py
# Cargo can patch a dependency to a directory in this repository, and
# the image builds from a context .dockerignore prunes to almost
# nothing. CI never sees the difference; a release does.
- if: always()
run: python3 tools/fork/context-check.py
build: build:
# Either runner (host1 or host2): the build needs no docker socket. # Either runner (host1 or host2): the build needs no docker socket.
runs-on: light runs-on: light
@@ -51,6 +72,16 @@ jobs:
# --no-run: the workflow compiled every test target without running them, # --no-run: the workflow compiled every test target without running them,
# which catches a test that no longer builds without paying for the suite. # which catches a test that no longer builds without paying for the suite.
- run: cargo test --workspace --locked --no-run - run: cargo test --workspace --locked --no-run
# The release profile, on main only. It is the profile the image is
# built with, and it fails in ways the dev profile does not: v2026.9.24
# was tagged on a commit whose CI was green and whose release build
# could not compile the scim crate at all. A few minutes per merge is
# cheaper than finding that out from a tag, which throws away a
# multi-architecture build and leaves a version half-cut.
#
# Pull requests stay on the dev profile, where the wait is worth less.
- if: github.event_name == 'push'
run: cargo build -p inbuxa --locked --release
# Keep the cache from growing without bound: past 60 GB the target dir # Keep the cache from growing without bound: past 60 GB the target dir
# is dropped and the next build starts cold. The download cache stays. # is dropped and the next build starts cold. The download cache stays.
# Two builds (dev + test profiles) already fill ~22 GB, so the limit # Two builds (dev + test profiles) already fill ~22 GB, so the limit
+155 -14
View File
@@ -3,11 +3,28 @@
# whether a person pushed it or weekly-release.yml created it through the # whether a person pushed it or weekly-release.yml created it through the
# releases API. # releases API.
# #
# The image is multi-arch (linux/amd64, linux/arm64) as before, but built in # The image is multi-arch (linux/amd64, linux/arm64), built by two jobs on
# one buildx run on host1 instead of one native runner per architecture: the # the image-build runner rather than one buildx run for both. The Dockerfile's
# Dockerfile's builder stage runs on the build platform and cross-compiles # builder stage runs on the build platform and cross-compiles with an aarch64
# with an aarch64 linker, so only the small final stage (apt, setcap) goes # linker, so only the small final stage (apt, setcap) goes through QEMU for
# through QEMU for arm64. No digest-joining job is needed. # arm64 -- but two release builds (LTO, one codegen unit) side by side on one
# machine each take twice as long. Production runs amd64, so amd64 goes first
# and on its own:
# * publish-amd64 pushes :<version>-amd64 and :<version>, a plain amd64
# image, as soon as its build is done. A deploy can start from it.
# * publish-arm64 then builds arm64, pushes :<version>-arm64, and replaces
# :<version> with the two-platform index. :latest moves only here, so it
# never names an image without arm64.
#
# Both jobs use one BuildKit builder, `gitea-builder`, whose container
# (buildx_buildkit_gitea-builder0) and state volume stay on the runner's host
# between jobs: a job container's `buildx create` finds the existing container
# and reuses it and its cache. The dependency build (`cargo chef cook`) is
# keyed on the recipe, which only a dependency change alters, so a release
# normally compiles just the workspace. Removing that container or its volume
# costs the next release a cold build, nothing more. The planner and dependency
# layers for the build platform are shared, so arm64 also reuses what amd64
# just did where it can.
# #
# Two guards before anything is pushed: # Two guards before anything is pushed:
# * the tag must be v<brand_version!>. The version is a string in # * the tag must be v<brand_version!>. The version is a string in
@@ -62,7 +79,7 @@ jobs:
echo "version=$V" >> "$GITHUB_OUTPUT" echo "version=$V" >> "$GITHUB_OUTPUT"
echo "version $V" echo "version $V"
publish: publish-amd64:
needs: [version] needs: [version]
runs-on: docker runs-on: docker
container: container:
@@ -81,16 +98,15 @@ jobs:
test -n "$REGISTRY" && test -n "$VERSION" test -n "$REGISTRY" && test -n "$VERSION"
test -n "$PACKAGE_TOKEN" || { echo "PACKAGE_TOKEN secret is not set on this repository" >&2; exit 1; } test -n "$PACKAGE_TOKEN" || { echo "PACKAGE_TOKEN secret is not set on this repository" >&2; exit 1; }
echo "$PACKAGE_TOKEN" | docker login -u jcoffey-dev --password-stdin "$REGISTRY" echo "$PACKAGE_TOKEN" | docker login -u jcoffey-dev --password-stdin "$REGISTRY"
docker run --privileged --rm tonistiigi/binfmt --install arm64
docker buildx create --use --name gitea-builder --driver docker-container || docker buildx use gitea-builder docker buildx create --use --name gitea-builder --driver docker-container || docker buildx use gitea-builder
# Attestations off, as before: they add manifests of their own to the # Attestations off, as before: they add manifests of their own, and the
# index, and the index should hold the two images and nothing else. # index should hold the two images and nothing else.
- run: | - run: |
docker buildx build \ docker buildx build \
--platform linux/amd64,linux/arm64 \ --platform linux/amd64 \
--provenance=false --sbom=false \ --provenance=false --sbom=false \
--tag "$IMAGE:$VERSION-amd64" \
--tag "$IMAGE:$VERSION" \ --tag "$IMAGE:$VERSION" \
--tag "$IMAGE:latest" \
--push . --push .
docker buildx imagetools inspect "$IMAGE:$VERSION" docker buildx imagetools inspect "$IMAGE:$VERSION"
# Gitea keeps a container package on its owner; linking it shows it on # Gitea keeps a container package on its owner; linking it shows it on
@@ -103,11 +119,47 @@ jobs:
- if: always() - if: always()
run: docker logout "$REGISTRY" || true run: docker logout "$REGISTRY" || true
publish-arm64:
needs: [version, publish-amd64]
runs-on: docker
container:
image: docker:28-cli@sha256:625d9431a9f54c5a2bc90f24f0e1c3d55b1349fd857dd85035f98c2c9acbdd4d # 28-cli
volumes:
- /var/run/docker.sock:/var/run/docker.sock
env:
DOCKER_BUILDKIT: "1"
REGISTRY: ${{ vars.REGISTRY }}
IMAGE: ${{ vars.REGISTRY }}/${{ github.repository }}
VERSION: ${{ needs.version.outputs.version }}
PACKAGE_TOKEN: ${{ secrets.PACKAGE_TOKEN }}
steps:
- uses: coffey-labs/actions/checkout@fab0c4d45e0162963965f1555df27b7bed5e20ec
- run: |
echo "$PACKAGE_TOKEN" | docker login -u jcoffey-dev --password-stdin "$REGISTRY"
docker run --privileged --rm tonistiigi/binfmt --install arm64
docker buildx create --use --name gitea-builder --driver docker-container || docker buildx use gitea-builder
# The index is built from the two per-architecture tags rather than from
# :<version>, which by now is the amd64 image and would be read as such.
- run: |
docker buildx build \
--platform linux/arm64 \
--provenance=false --sbom=false \
--tag "$IMAGE:$VERSION-arm64" \
--push .
docker buildx imagetools create \
--tag "$IMAGE:$VERSION" \
--tag "$IMAGE:latest" \
"$IMAGE:$VERSION-amd64" "$IMAGE:$VERSION-arm64"
docker buildx imagetools inspect "$IMAGE:$VERSION"
- if: always()
run: docker logout "$REGISTRY" || true
# The weekly release creates its Release (and so the tag) first; a tag # The weekly release creates its Release (and so the tag) first; a tag
# pushed by hand has none. Either way the tag ends up with exactly one # pushed by hand has none. Either way the tag ends up with exactly one
# Release, created after the image exists so its pull instructions work. # Release, created once the amd64 image exists so its pull instructions
# work; arm64 and the binaries follow.
release: release:
needs: [version, publish] needs: [version, publish-amd64]
runs-on: light runs-on: light
container: container:
image: python:3.13-slim@sha256:8d9d0b8bcf6506481eae4907c18f5e3e7902e629f5f6d684f9e7c32e85e3ddf0 # 3.13-slim image: python:3.13-slim@sha256:8d9d0b8bcf6506481eae4907c18f5e3e7902e629f5f6d684f9e7c32e85e3ddf0 # 3.13-slim
@@ -131,8 +183,97 @@ jobs:
except urllib.error.HTTPError as e: except urllib.error.HTTPError as e:
if e.code != 404: raise if e.code != 404: raise
image = f"{os.environ['REGISTRY']}/{os.environ['REPO']}:{version}" image = f"{os.environ['REGISTRY']}/{os.environ['REPO']}:{version}"
body = f"Container image: `{image}` (linux/amd64, linux/arm64); also `:latest`." body = (f"Container image: `{image}` (linux/amd64, linux/arm64); also `:latest`. "
"amd64 is published first; arm64 is added to the same tag when its build "
"finishes, and `:latest` moves then.\n\n"
"Binaries for a host install are attached: `inbuxa-linux-amd64.tar.gz` and "
"`inbuxa-linux-arm64.tar.gz`, with `SHA256SUMS`. Each is the binary out of this "
"release's image for that architecture, so it is the same build. The image "
"grants it `cap_net_bind_service`; a host install has to grant that itself "
"(`setcap`, or `AmbientCapabilities` in the unit) to bind port 25.")
data = json.dumps({"tag_name": tag, "name": f"INBUXA {version}", "body": body}).encode() data = json.dumps({"tag_name": tag, "name": f"INBUXA {version}", "body": body}).encode()
r = json.load(urllib.request.urlopen(urllib.request.Request(f"{api}/releases", data=data, headers=h))) r = json.load(urllib.request.urlopen(urllib.request.Request(f"{api}/releases", data=data, headers=h)))
print(f"created release {r['tag_name']}") print(f"created release {r['tag_name']}")
PY PY
# The binaries for a host install, taken out of the image that was just
# pushed rather than compiled again.
#
# Building them separately would mean a second Rust build per architecture
# -- the slowest thing this pipeline does -- and would leave two artifacts
# that are supposed to be the same build but only probably are. Extracting
# them makes that identity a fact: the binary in the tarball is the file
# the image runs.
#
# `docker create` does not start anything, so pulling an arm64 image on an
# amd64 runner and copying a file out of it needs no emulation.
binaries:
needs: [version, publish-arm64, release]
runs-on: docker
container:
image: docker:28-cli@sha256:625d9431a9f54c5a2bc90f24f0e1c3d55b1349fd857dd85035f98c2c9acbdd4d # 28-cli
volumes:
- /var/run/docker.sock:/var/run/docker.sock
env:
REGISTRY: ${{ vars.REGISTRY }}
IMAGE: ${{ vars.REGISTRY }}/${{ github.repository }}
VERSION: ${{ needs.version.outputs.version }}
TAG: ${{ github.ref_name }}
REPO: ${{ github.repository }}
PACKAGE_TOKEN: ${{ secrets.PACKAGE_TOKEN }}
TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- name: take the binaries out of the image
run: |
set -euo pipefail
echo "$PACKAGE_TOKEN" | docker login -u jcoffey-dev --password-stdin "$REGISTRY"
mkdir -p /out && cd /out
for arch in amd64 arm64; do
docker pull -q --platform "linux/$arch" "$IMAGE:$VERSION"
id="$(docker create --platform "linux/$arch" "$IMAGE:$VERSION")"
docker cp "$id:/usr/local/bin/inbuxa" "inbuxa"
docker rm -f "$id" >/dev/null
chmod 0755 inbuxa
tar -czf "inbuxa-linux-$arch.tar.gz" inbuxa
rm inbuxa
done
sha256sum inbuxa-linux-*.tar.gz > SHA256SUMS
cat SHA256SUMS
- name: attach them to the release
run: |
set -euo pipefail
apk add --no-cache -q python3
python3 - <<'PY'
import json, os, urllib.request, urllib.error, uuid, pathlib
api = f"{os.environ['CI_SERVER_INTERNAL']}/api/v1/repos/{os.environ['REPO']}"
tok = {"Authorization": f"token {os.environ['TOKEN']}"}
tag = os.environ["TAG"]
def get(path):
return json.load(urllib.request.urlopen(urllib.request.Request(api + path, headers=tok)))
rel = get(f"/releases/tags/{tag}")
assets = {a["name"]: a["id"] for a in get(f"/releases/{rel['id']}/assets")}
for path in ["/out/inbuxa-linux-amd64.tar.gz", "/out/inbuxa-linux-arm64.tar.gz", "/out/SHA256SUMS"]:
name = os.path.basename(path)
# A re-run of a tag replaces its assets rather than leaving two
# files with the same name and different contents.
if name in assets:
urllib.request.urlopen(urllib.request.Request(
f"{api}/releases/{rel['id']}/assets/{assets[name]}", headers=tok, method="DELETE"))
boundary = uuid.uuid4().hex
body = b"".join([
f"--{boundary}\r\nContent-Disposition: form-data; name=\"attachment\"; filename=\"{name}\"\r\n".encode(),
b"Content-Type: application/octet-stream\r\n\r\n",
pathlib.Path(path).read_bytes(),
f"\r\n--{boundary}--\r\n".encode(),
])
req = urllib.request.Request(
f"{api}/releases/{rel['id']}/assets?name={name}", data=body, method="POST",
headers={**tok, "Content-Type": f"multipart/form-data; boundary={boundary}"})
urllib.request.urlopen(req)
print("attached", name)
PY
- if: always()
run: docker logout "$REGISTRY" || true
+122
View File
@@ -0,0 +1,122 @@
# Watch upstream for releases the fork hasn't imported yet, and open an issue
# for each one so it waits in the tracker until someone strips it in.
#
# Reads metadata only -- the releases list from GitHub's API and the head of
# this repo's `upstream` branch from Gitea's. Nothing of upstream's is fetched,
# so none of its history (which carries the Enterprise code) can land here.
# Importing is still by hand: tools/fork/strip.py onto `upstream`, then merge,
# as docs/spec/SPEC.md §2.2 and §2.2a describe.
#
# The imported base is the tag in the `upstream` branch's head commit subject
# ("Import upstream v0.16.22, stripped"). Drafts and pre-releases are ignored.
# An issue is opened once per release: an existing one with the same title,
# open or closed, stops a second.
#
# It also watches spam-filter, whose rules the server bundles
# (resources/spam-filter/), and opens an issue for a newer release.
#
# Daily 06:17 UTC; run it by hand with workflow_dispatch.
name: upstream-watch
on:
schedule:
- cron: '17 6 * * *'
workflow_dispatch:
concurrency:
group: upstream-watch
cancel-in-progress: false
jobs:
upstream-watch:
runs-on: light
container:
image: python:3.13-slim@sha256:8d9d0b8bcf6506481eae4907c18f5e3e7902e629f5f6d684f9e7c32e85e3ddf0 # 3.13-slim
env:
TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
steps:
- shell: bash
run: |
python3 - <<'PY'
import json, os, re, sys, urllib.request
api = f"{os.environ['CI_SERVER_INTERNAL']}/api/v1/repos/{os.environ['REPO']}"
def call(method, url, body=None, token=os.environ["TOKEN"]):
headers = {"Content-Type": "application/json", "User-Agent": "inbuxa-upstream-watch"}
if token:
headers["Authorization"] = f"token {token}"
req = urllib.request.Request(url, method=method, headers=headers,
data=json.dumps(body).encode() if body is not None else None)
with urllib.request.urlopen(req, timeout=30) as r:
return json.load(r)
SEMVER = re.compile(r"^v(\d+)\.(\d+)\.(\d+)$")
def key(tag):
return tuple(int(x) for x in SEMVER.match(tag).groups())
subject = call("GET", f"{api}/branches/upstream")["commit"]["message"].splitlines()[0]
m = re.search(r"\bupstream (v\d+\.\d+\.\d+)\b", subject)
if not m:
print(f"Can't read the imported base from the upstream branch: {subject!r}", file=sys.stderr); sys.exit(1)
base = m.group(1)
# Unauthenticated: a public repo, once a day, well inside the limit.
rels = call("GET", "https://api.github.com/repos/stalwartlabs/stalwart/releases?per_page=30", token=None)
newer = sorted((r for r in rels
if not r["draft"] and not r["prerelease"] and SEMVER.match(r["tag_name"])
and key(r["tag_name"]) > key(base)),
key=lambda r: key(r["tag_name"]))
if not newer:
print(f"Up to date: {base} is the newest upstream release.")
# Titles and bodies stay free of the upstream project's name, as the
# rest of the fork's user-visible text does.
existing = {i["title"] for i in call("GET", f"{api}/issues?state=all&type=issues&q=Import+upstream&limit=50")}
for r in newer:
tag = r["tag_name"]
title = f"Import upstream {tag}"
if title in existing:
print(f"{tag}: issue already exists."); continue
body = (f"Upstream published {tag} on {r['published_at'][:10]}. "
f"The fork's imported base is {base}.\n\n"
"Import it as tools/fork/README.md describes:\n\n"
"```bash\n"
"git -C \"$UPSTREAM_CLONE\" fetch --tags\n"
f"tools/fork/strip.py --upstream \"$UPSTREAM_CLONE\" --ref {tag} --out /tmp/strip-{tag}\n"
"```\n\n"
"Commit the stripped tree to `upstream` with the strip report in the message, "
"add any new third-party notices to `THIRD-PARTY.md`, then merge `upstream` into `main`.")
issue = call("POST", f"{api}/issues", {"title": title, "body": body})
print(f"{tag}: opened #{issue['number']}.")
# The spam filter rules bundled with the server (resources/spam-filter/):
# an issue when spam-filter publishes a newer release than the one
# BUNDLED_SPAM_RULES_VERSION names on main.
src = call("GET", f"{api}/contents/crates/common/src/manager/spam_rules.rs?ref=main")
import base64
text = base64.b64decode(src["content"]).decode()
m = re.search(r'BUNDLED_SPAM_RULES_VERSION: &str = "(\d+\.\d+\.\d+)"', text)
if not m:
print("Can't read BUNDLED_SPAM_RULES_VERSION from spam_rules.rs", file=sys.stderr); sys.exit(1)
bundled = "v" + m.group(1)
rels = call("GET", "https://api.github.com/repos/stalwartlabs/spam-filter/releases?per_page=30", token=None)
newer = sorted((r for r in rels
if not r["draft"] and not r["prerelease"] and SEMVER.match(r["tag_name"])
and key(r["tag_name"]) > key(bundled)),
key=lambda r: key(r["tag_name"]))
if not newer:
print(f"Up to date: the bundled spam rules are {bundled}, the newest release."); sys.exit(0)
latest = newer[-1]
tag = latest["tag_name"]
title = f"Update the bundled spam rules to {tag}"
existing = {i["title"] for i in call("GET", f"{api}/issues?state=all&type=issues&q=bundled+spam+rules&limit=50")}
if title in existing:
print(f"spam rules {tag}: issue already exists."); sys.exit(0)
body = (f"spam-filter published {tag} on {latest['published_at'][:10]}. "
f"The server bundles {bundled}.\n\n"
"Update it as resources/spam-filter/README.md describes: take the rules file "
f"from the {tag} release (by tag, not `latest`), set BUNDLED_SPAM_RULES_VERSION, "
"and run the antispam test.")
issue = call("POST", f"{api}/issues", {"title": title, "body": body})
print(f"spam rules {tag}: opened #{issue['number']}.")
PY
-50
View File
@@ -1,50 +0,0 @@
# CI on the self-hosted GitLab, ported from .github/workflows/ci.yml when the
# GitHub account was suspended on 2026-09-20. The Actions file stays in the
# tree: it is the reference this was written from and works unchanged if the
# appeal succeeds.
#
# The image is pinned by digest, with its tag in the trailing comment. That
# replaces the SHA-pinned `uses:` in the workflow -- GitLab has no action
# allowlist, so the digest is the only thing fixing what actually runs.
#
# Not ported here:
# * cleanup.yml pruned GHCR with dataaxiom/ghcr-cleanup-action. GitLab has
# no equivalent action because it does not need one: the container
# registry has a cleanup policy on the project itself, which is where that
# job's settings now live.
# * publish.yml and release.yml still need doing; they are larger and are
# being handled separately.
stages: [build]
default:
interruptible: true
build:
stage: build
image: rust:1-bookworm@sha256:93ce27a88655056a51dbdd8f5f2d7ddc071c7b0070fb288a37b5a285fc83971e # 1-bookworm
# This is a big workspace and a cold build is expensive, so the registry and
# the target directory are cached between runs. Both are kept inside the
# project directory because that is the only path the runner will cache --
# and deliberately not on /tmp, which on this host is a tmpfs that a Rust
# build of this size has filled before.
variables:
CARGO_HOME: "$CI_PROJECT_DIR/.cargo"
CARGO_TARGET_DIR: "$CI_PROJECT_DIR/target"
CARGO_INCREMENTAL: "0"
cache:
key:
files: [Cargo.lock]
paths:
- .cargo/registry/
- target/
before_script:
- apt-get update -qq && apt-get install -y -qq --no-install-recommends clang >/dev/null
script:
- cargo build -p inbuxa --locked
# --no-run: the workflow compiled every test target without running them,
# which catches a test that no longer builds without paying for the suite.
- cargo test --workspace --locked --no-run
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
+33
View File
@@ -2,6 +2,39 @@
All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org/). All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org/).
## [0.16.23] - 2026-09-21
If you are upgrading from v0.16.x, replace the binary (or run `docker pull`). If you are upgrading from v0.15.x and below, please read the [upgrading documentation](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING/v0_16.md) for more information on how to upgrade from previous versions.
## Added
- Expressions: `bit_and` function.
## Changed
## Fixed
- MTA:
- A mailing list whose recipients include another mailing list is accepted at `RCPT TO` and then rejected at local delivery with `550 5.5.0 Mailbox not found`.
- DMARC aggregate reports carry two `spf` elements per record and the `version` element of a DMARC aggregate report is written as `1` instead of `1.0`.
- DSNs generated for an alias rewrite or a list expansion emit a doubled `addr-type` in `Original-Recipient` (`rfc822;rfc822;[email protected]`).
- DSNs that cannot be written to the store are discarded, the recipients are flagged as notified and the original message is removed from the queue, losing both the bounce and the message.
- POP3:
- `TOP msg n` counts the `n` lines from the first byte of the message instead of from the first byte of the body.
- A message whose very first line begins with `.` is not byte-stuffed.
- Spam filter: Moving or copying a message from one account into another creates no training sample, so the classifier never learns from it.
- Sieve: `envelope "orcpt"` yields the bare address for an `ORCPT` supplied over SMTP. It now carries the `addr-type` prefix in every case, as required by RFC 6009.
- ACME: The `_acme-challenge` TXT records published for a DNS-01 authorization are never removed.
- DNS: The DNSSEC resolver queries a single nameserver at a time, working around a `hickory-resolver` race that cancels the TCP retry when two nameservers return a truncated response in parallel.
- Troubleshoot tool:
- MX records are resolved through the DNSSEC-validating resolver, matching the resolver used by the delivery path.
- A TLSA lookup that fails or returns bogus records stops the delivery attempt for that host, instead of continuing without DANE.
- OIDC: Bearer tokens that carry no `email`, `preferred_username` or `upn` claim are always authenticated against the default directory.
- Meilisearch: A confirmation timeout is treated as a failed write even when `failOnTimeout` is disabled, so an index whose batches take longer than `pollInterval` x `maxRetries` never completes an indexing task and resubmits the same batch indefinitely.
- WebUI: A failed update no longer takes an `Application` offline.
- FoundationDB: The cached read version is invalidated when any broadcast is received from another node.
- Redis:
- On a cluster, the rate limiter and the blob upload quota issue `INCR` and `EXPIRE` as a `MULTI`/`EXEC` transaction, whose `MOVED` redirects collapse into a single `EXECABORT` that never refreshes the slot map.
- A connection that fails because it is addressing the wrong server is returned to the pool and reused, since the recycle check only issues `PING`.
## [0.16.22] - 2026-09-13 ## [0.16.22] - 2026-09-13
If you are upgrading from v0.16.x, replace the binary (or run `docker pull`). If you are upgrading from v0.15.x and below, please read the [upgrading documentation](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING/v0_16.md) for more information on how to upgrade from previous versions. If you are upgrading from v0.16.x, replace the binary (or run `docker pull`). If you are upgrading from v0.15.x and below, please read the [upgrading documentation](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING/v0_16.md) for more information on how to upgrade from previous versions.
Generated
+140 -132
View File
@@ -234,7 +234,7 @@ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 2.0.119", "syn 2.0.119",
"synstructure", "synstructure 0.13.2",
] ]
[[package]] [[package]]
@@ -277,9 +277,9 @@ dependencies = [
[[package]] [[package]]
name = "async-compression" name = "async-compression"
version = "0.4.46" version = "0.4.48"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4f10dafd0c8d2e51ae9a748805777613ed0bbe17bf586b76c8311f45c020a32f" checksum = "fb61aea1a7def73ee7c350a184f0e70b32c182344e2e75bf70c9b621b83417fd"
dependencies = [ dependencies = [
"compression-codecs", "compression-codecs",
"compression-core", "compression-core",
@@ -310,7 +310,7 @@ dependencies = [
"memchr", "memchr",
"pin-project", "pin-project",
"portable-atomic", "portable-atomic",
"rand 0.10.2", "rand 0.10.3",
"regex", "regex",
"rustls-native-certs", "rustls-native-certs",
"rustls-pki-types", "rustls-pki-types",
@@ -369,7 +369,7 @@ checksum = "82f6aeea286b8eb4dd3431a1be1b59d290ace00f5bfd8e2a159bc2a05e2c1667"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
@@ -874,7 +874,7 @@ dependencies = [
"log", "log",
"num", "num",
"pin-project-lite", "pin-project-lite",
"rand 0.10.2", "rand 0.10.3",
"rustls", "rustls",
"rustls-native-certs", "rustls-native-certs",
"rustls-pki-types", "rustls-pki-types",
@@ -984,7 +984,7 @@ checksum = "46d07918caa9eeaaf06b7873925c53a61daac173539b4f7715090745e44e4e69"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
@@ -1110,9 +1110,9 @@ dependencies = [
[[package]] [[package]]
name = "cc" name = "cc"
version = "1.4.6" version = "1.4.7"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a3eb0f42d6c360dc3f8a821f6bf2fdea7f72bfd36b3076eb0e6d1e9e0752fff4" checksum = "54413ede23c2daf518f35156dfde027feb2374004d63bd497f983c8db9c0e313"
dependencies = [ dependencies = [
"find-msvc-tools", "find-msvc-tools",
"jobserver", "jobserver",
@@ -1160,9 +1160,9 @@ dependencies = [
[[package]] [[package]]
name = "cfg-if" name = "cfg-if"
version = "1.0.4" version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" checksum = "4e7648175b45a9a48536d676f68d918270699102aa8dab5496df06904c914600"
[[package]] [[package]]
name = "cfg_aliases" name = "cfg_aliases"
@@ -1302,7 +1302,7 @@ dependencies = [
[[package]] [[package]]
name = "common" name = "common"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"aes-gcm-siv", "aes-gcm-siv",
"ahash", "ahash",
@@ -1402,9 +1402,9 @@ dependencies = [
[[package]] [[package]]
name = "compression-codecs" name = "compression-codecs"
version = "0.4.41" version = "0.4.43"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "58a6d0db8759036a783bc7c3f7a07f8cef3bf9470eb1db3bc86e8bcd1c5d0fe8" checksum = "bef16c47ba2797aa6a909cc37d39911f3a6743811fe7408ac0b0cc0276b656e9"
dependencies = [ dependencies = [
"compression-core", "compression-core",
"flate2", "flate2",
@@ -1487,7 +1487,7 @@ checksum = "3d52eff69cd5e647efe296129160853a42795992097e8af39800e1060caeea9b"
[[package]] [[package]]
name = "coordinator" name = "coordinator"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"async-nats", "async-nats",
"futures", "futures",
@@ -1849,7 +1849,7 @@ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"strsim", "strsim",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
@@ -1882,7 +1882,7 @@ checksum = "2ac7135c3ef02b2f7833bbeb1be5ba7f966dcde8a87c6b87f65a778d71a02785"
dependencies = [ dependencies = [
"darling_core 0.24.1", "darling_core 0.24.1",
"quote", "quote",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
@@ -1899,7 +1899,7 @@ checksum = "4583a4551df46e2792f82ceeac45e850d2e2d5debba0b91f102385cda5b11f06"
[[package]] [[package]]
name = "dav" name = "dav"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"calcard", "calcard",
"chrono", "chrono",
@@ -1922,7 +1922,7 @@ dependencies = [
[[package]] [[package]]
name = "dav-proto" name = "dav-proto"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"calcard", "calcard",
"chrono", "chrono",
@@ -2135,7 +2135,7 @@ dependencies = [
[[package]] [[package]]
name = "directory" name = "directory"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"ahash", "ahash",
"argon2 0.6.0", "argon2 0.6.0",
@@ -2192,7 +2192,7 @@ checksum = "c6232dd377dcc64799954cbd3a9bb882e9cdc1308ccd87b1c098f1fb2eaf82a8"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
@@ -2376,7 +2376,7 @@ dependencies = [
[[package]] [[package]]
name = "email" name = "email"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"aes 0.9.3", "aes 0.9.3",
"aes-gcm 0.11.1", "aes-gcm 0.11.1",
@@ -2485,10 +2485,10 @@ dependencies = [
[[package]] [[package]]
name = "event_macro" name = "event_macro"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"quote", "quote",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
@@ -2571,7 +2571,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ee93edf3c501f0035bbeffeccfed0b79e14c311f12195ec0e661e114a0f60da4" checksum = "ee93edf3c501f0035bbeffeccfed0b79e14c311f12195ec0e661e114a0f60da4"
dependencies = [ dependencies = [
"portable-atomic", "portable-atomic",
"rand 0.10.2", "rand 0.10.3",
"web-time", "web-time",
] ]
@@ -2594,9 +2594,9 @@ checksum = "28dea519a9695b9977216879a3ebfddf92f1c08c05d984f8996aecd6ecdc811d"
[[package]] [[package]]
name = "find-msvc-tools" name = "find-msvc-tools"
version = "0.1.12" version = "0.1.13"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3e0f1c7c3a72c66fd80abe965175f7523475c0489a87d3ff9d6e8c87d87a9d2d" checksum = "ef25905e51abafe4dcea6c15fec58c57b601cdbd0ee53d22ea1d3016c587d39b"
[[package]] [[package]]
name = "fixed_decimal" name = "fixed_decimal"
@@ -2710,7 +2710,7 @@ dependencies = [
"foundationdb-sys", "foundationdb-sys",
"foundationdb-tuple", "foundationdb-tuple",
"futures", "futures",
"rand 0.10.2", "rand 0.10.3",
"serde", "serde",
"serde_bytes", "serde_bytes",
"serde_json", "serde_json",
@@ -2842,7 +2842,7 @@ checksum = "9fb9654ba8355388abeb8dcb4fc62f511300867002afc858860463bdd9fe0c44"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
@@ -3013,7 +3013,7 @@ dependencies = [
[[package]] [[package]]
name = "groupware" name = "groupware"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"ahash", "ahash",
"calcard", "calcard",
@@ -3169,7 +3169,7 @@ dependencies = [
"jni", "jni",
"lru-cache", "lru-cache",
"parking_lot", "parking_lot",
"rand 0.10.2", "rand 0.10.3",
"rustls", "rustls",
"rustls-pki-types", "rustls-pki-types",
"rustls-platform-verifier", "rustls-platform-verifier",
@@ -3196,7 +3196,7 @@ dependencies = [
"jni", "jni",
"once_cell", "once_cell",
"prefix-trie", "prefix-trie",
"rand 0.10.2", "rand 0.10.3",
"ring", "ring",
"rustls-pki-types", "rustls-pki-types",
"thiserror 2.0.20", "thiserror 2.0.20",
@@ -3223,7 +3223,7 @@ dependencies = [
"ndk-context", "ndk-context",
"once_cell", "once_cell",
"parking_lot", "parking_lot",
"rand 0.10.2", "rand 0.10.3",
"resolv-conf", "resolv-conf",
"rustls", "rustls",
"smallvec", "smallvec",
@@ -3302,7 +3302,7 @@ dependencies = [
[[package]] [[package]]
name = "http" name = "http"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"async-stream", "async-stream",
"base64 0.23.1", "base64 0.23.1",
@@ -3398,7 +3398,7 @@ dependencies = [
[[package]] [[package]]
name = "http_proto" name = "http_proto"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"common", "common",
"compact_str", "compact_str",
@@ -3488,9 +3488,9 @@ dependencies = [
[[package]] [[package]]
name = "hyper-rustls" name = "hyper-rustls"
version = "0.27.9" version = "0.27.10"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "33ca68d021ef39cf6463ab54c1d0f5daf03377b70561305bb89a8f83aab66e0f" checksum = "dfa8e654703247911e29c23fbeaa261834bd9bb74efba2f9acddc37bfb127f53"
dependencies = [ dependencies = [
"http 1.5.0", "http 1.5.0",
"hyper", "hyper",
@@ -3533,7 +3533,7 @@ dependencies = [
"libc", "libc",
"percent-encoding", "percent-encoding",
"pin-project-lite", "pin-project-lite",
"socket2 0.5.10", "socket2 0.6.5",
"tokio", "tokio",
"tower-service", "tower-service",
"tracing", "tracing",
@@ -3884,7 +3884,7 @@ checksum = "65b27460c2c92b037f3f94c538ed9a3342f3fdf923606781629ccb35f82d042a"
[[package]] [[package]]
name = "imap" name = "imap"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"ahash", "ahash",
"common", "common",
@@ -3897,7 +3897,7 @@ dependencies = [
"md5", "md5",
"nlp", "nlp",
"parking_lot", "parking_lot",
"rand 0.10.2", "rand 0.10.3",
"registry", "registry",
"store", "store",
"tokio", "tokio",
@@ -3909,7 +3909,7 @@ dependencies = [
[[package]] [[package]]
name = "imap_proto" name = "imap_proto"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"ahash", "ahash",
"base64 0.23.1", "base64 0.23.1",
@@ -3924,7 +3924,7 @@ dependencies = [
[[package]] [[package]]
name = "inbuxa" name = "inbuxa"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"common", "common",
"coordinator", "coordinator",
@@ -3932,7 +3932,7 @@ dependencies = [
"directory", "directory",
"email", "email",
"groupware", "groupware",
"http 0.16.22", "http 0.16.23",
"http_proto", "http_proto",
"imap", "imap",
"jmap", "jmap",
@@ -4134,25 +4134,24 @@ checksum = "4d3667095d64c3ecffc96463a21157b04bf3e252f6e8d5750b20c02e33c194e3"
[[package]] [[package]]
name = "jieba-macros" name = "jieba-macros"
version = "0.10.3" version = "0.10.4"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "34904340bc65749a9e9a02fcc7f3368e675427c18447b9bbe02df52c15c9a36a" checksum = "455f837e9d0255b68a712200db247c68fdad4941b72471b76bfa61c3b0c1f79f"
dependencies = [ dependencies = [
"phf_codegen", "phf_codegen",
] ]
[[package]] [[package]]
name = "jieba-rs" name = "jieba-rs"
version = "0.10.3" version = "0.10.4"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bb5bdea4dc241d589e179f39d2a778f31490f3370aa2f626223dbd930ebc5c9d" checksum = "b6a8bbb0f77ee810f0689a30b7cec56b875751ef4ec2e74fd995613dc52b3ae1"
dependencies = [ dependencies = [
"bytecount", "bytecount",
"cedarwood", "cedarwood",
"include-flate", "include-flate",
"jieba-macros", "jieba-macros",
"phf 0.13.1", "phf 0.13.1",
"regex",
"rustc-hash", "rustc-hash",
] ]
@@ -4212,7 +4211,7 @@ dependencies = [
[[package]] [[package]]
name = "jmap" name = "jmap"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"async-stream", "async-stream",
"base64 0.23.1", "base64 0.23.1",
@@ -4236,7 +4235,7 @@ dependencies = [
"mail-parser", "mail-parser",
"nlp", "nlp",
"p256", "p256",
"rand 0.10.2", "rand 0.10.3",
"registry", "registry",
"reqwest 0.13.5", "reqwest 0.13.5",
"rkyv", "rkyv",
@@ -4294,7 +4293,7 @@ dependencies = [
[[package]] [[package]]
name = "jmap_proto" name = "jmap_proto"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"ahash", "ahash",
"calcard", "calcard",
@@ -4699,9 +4698,9 @@ dependencies = [
[[package]] [[package]]
name = "lru-slab" name = "lru-slab"
version = "0.1.2" version = "0.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "112b39cec0b298b6c1999fee3e31427f74f676e4cb9879ed1a121b43661a4154" checksum = "4050469837a6ff301cd14c1f8f24f88549e6d548f24f64e2148eb0f72cebc51f"
[[package]] [[package]]
name = "lz4-sys" name = "lz4-sys"
@@ -4742,9 +4741,9 @@ dependencies = [
[[package]] [[package]]
name = "mail-auth" name = "mail-auth"
version = "0.13.2" version = "0.13.3"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e11f19d98aac923fc5b7ee30c3509733a013ef546a226acb959b9202f5ca58f0" checksum = "8505122ba86e1f4adeb664196c1e787c3f29bb6e7c128e4a366d47d209911440"
dependencies = [ dependencies = [
"aws-lc-rs", "aws-lc-rs",
"flate2", "flate2",
@@ -4757,7 +4756,7 @@ dependencies = [
"mail-parser", "mail-parser",
"memchr", "memchr",
"quick-xml 0.42.0", "quick-xml 0.42.0",
"rand 0.10.2", "rand 0.10.3",
"rkyv", "rkyv",
"rsa", "rsa",
"rustls-pki-types", "rustls-pki-types",
@@ -4800,7 +4799,7 @@ dependencies = [
[[package]] [[package]]
name = "managesieve" name = "managesieve"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"common", "common",
"compact_str", "compact_str",
@@ -4935,7 +4934,7 @@ checksum = "c797b9d6bb23aab2fc369c65f871be49214f5c759af65bde26ffaaa2b646b492"
[[package]] [[package]]
name = "migration" name = "migration"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"common", "common",
"email", "email",
@@ -5066,7 +5065,7 @@ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"rustversion", "rustversion",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
@@ -5131,7 +5130,7 @@ dependencies = [
"lru", "lru",
"mysql_common", "mysql_common",
"percent-encoding", "percent-encoding",
"rand 0.10.2", "rand 0.10.3",
"rustls", "rustls",
"serde", "serde",
"socket2 0.6.5", "socket2 0.6.5",
@@ -5206,14 +5205,14 @@ dependencies = [
[[package]] [[package]]
name = "nlp" name = "nlp"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"ahash", "ahash",
"hashify", "hashify",
"jieba-rs", "jieba-rs",
"maplit", "maplit",
"psl", "psl",
"rand 0.10.2", "rand 0.10.3",
"rkyv", "rkyv",
"rust-stemmers", "rust-stemmers",
"serde", "serde",
@@ -6038,7 +6037,7 @@ dependencies = [
[[package]] [[package]]
name = "pop3" name = "pop3"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"common", "common",
"directory", "directory",
@@ -6082,7 +6081,7 @@ dependencies = [
"hmac 0.13.0", "hmac 0.13.0",
"md-5 0.11.0", "md-5 0.11.0",
"memchr", "memchr",
"rand 0.10.2", "rand 0.10.3",
"sha2 0.11.0", "sha2 0.11.0",
"stringprep", "stringprep",
] ]
@@ -6119,9 +6118,9 @@ checksum = "439ee305def115ba05938db6eb1644ff94165c5ab5e9420d1c1bcedbba909391"
[[package]] [[package]]
name = "ppmd-rust" name = "ppmd-rust"
version = "1.4.1" version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9e9219bcb9d7aca6b2f63c83cf100cf78bcd619ac46e6ecbd0dd90869a39345d" checksum = "196a7c80b9a7652aba7cc070827516c2abe4ccdf53d128e1944003cf5726cff1"
[[package]] [[package]]
name = "ppv-lite86" name = "ppv-lite86"
@@ -6206,7 +6205,7 @@ dependencies = [
"proc-macro-error-attr3", "proc-macro-error-attr3",
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
@@ -6260,7 +6259,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b570b25f7617e43d59005d0990ccb79e950a423952cea19671b7a876da390adf" checksum = "b570b25f7617e43d59005d0990ccb79e950a423952cea19671b7a876da390adf"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"itertools 0.13.0", "itertools 0.14.0",
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 2.0.119", "syn 2.0.119",
@@ -6287,9 +6286,9 @@ dependencies = [
[[package]] [[package]]
name = "psl" name = "psl"
version = "2.1.232" version = "2.1.235"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "62834e308cc83aea5e30cd8c80b8aa82cdb104a3240c7f210d4f68d46e29f308" checksum = "8319b56ff38ca0522b4e1e40bfa2b5de7f62dc89fc1e9033eac365551ec58e0e"
dependencies = [ dependencies = [
"psl-types", "psl-types",
] ]
@@ -6317,7 +6316,7 @@ checksum = "1c8d9ca532f185d5d4db7a7c9d51420b452168ea1c2b913953281bd6fe1fcbd0"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
@@ -6388,9 +6387,9 @@ dependencies = [
[[package]] [[package]]
name = "quinn" name = "quinn"
version = "0.11.11" version = "0.11.12"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c1a41e437b6bbd489372cd4971de128e85c855f56c57f283d20ff016cf7c0a8" checksum = "4051e23e9185c255a7e33ef59cdbca87a22d359052eecd22fc6b901fb37d9d11"
dependencies = [ dependencies = [
"bytes", "bytes",
"cfg_aliases", "cfg_aliases",
@@ -6399,7 +6398,7 @@ dependencies = [
"quinn-udp", "quinn-udp",
"rustc-hash", "rustc-hash",
"rustls", "rustls",
"socket2 0.5.10", "socket2 0.6.5",
"thiserror 2.0.20", "thiserror 2.0.20",
"tokio", "tokio",
"tracing", "tracing",
@@ -6408,16 +6407,16 @@ dependencies = [
[[package]] [[package]]
name = "quinn-proto" name = "quinn-proto"
version = "0.11.17" version = "0.11.18"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "04759210543be93709136e28212294a659ef5001836ff4eab4d663e4529bba83" checksum = "a9746dbde176634f4f2f1faf2404e30a31b2bc1e9cafb5329c95d8177a18c9fc"
dependencies = [ dependencies = [
"aws-lc-rs", "aws-lc-rs",
"bytes", "bytes",
"fastbloom", "fastbloom",
"getrandom 0.4.3", "getrandom 0.4.3",
"lru-slab", "lru-slab",
"rand 0.10.2", "rand 0.10.3",
"rand_pcg", "rand_pcg",
"ring", "ring",
"rustc-hash", "rustc-hash",
@@ -6440,7 +6439,7 @@ dependencies = [
"cfg_aliases", "cfg_aliases",
"libc", "libc",
"once_cell", "once_cell",
"socket2 0.5.10", "socket2 0.6.5",
"tracing", "tracing",
"windows-sys 0.61.2", "windows-sys 0.61.2",
] ]
@@ -6534,9 +6533,9 @@ dependencies = [
[[package]] [[package]]
name = "rand" name = "rand"
version = "0.10.2" version = "0.10.3"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c7f5fa3a058cd35567ef9bfa5e75732bee0f9e4c55fa90477bef2dfcdbc4be80" checksum = "65c9fb96cbc91e3478eaae79a69fcd3f1ae4ad052e471fe6732fff548984b4af"
dependencies = [ dependencies = [
"chacha20", "chacha20",
"getrandom 0.4.3", "getrandom 0.4.3",
@@ -6776,7 +6775,7 @@ dependencies = [
"num-bigint 0.5.1", "num-bigint 0.5.1",
"percent-encoding", "percent-encoding",
"pin-project-lite", "pin-project-lite",
"rand 0.10.2", "rand 0.10.3",
"rustls", "rustls",
"rustls-native-certs", "rustls-native-certs",
"ryu", "ryu",
@@ -6800,11 +6799,10 @@ dependencies = [
[[package]] [[package]]
name = "redox_users" name = "redox_users"
version = "0.5.2" version = "0.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a4e608c6638b9c18977b00b475ac1f28d14e84b27d8d42f70e0bf1e3dec127ac" checksum = "60dc65c0ff1a7ae1294b0c67b9f14baf70b644404010370171787bfac1038fc0"
dependencies = [ dependencies = [
"getrandom 0.2.17",
"libredox", "libredox",
"thiserror 2.0.20", "thiserror 2.0.20",
] ]
@@ -6826,7 +6824,7 @@ checksum = "92ecd8964f8453721699a1ed72037b0db49ce2f5a5138486ee89bed6f67cdf3a"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
@@ -6860,7 +6858,7 @@ checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4"
[[package]] [[package]]
name = "registry" name = "registry"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"ahash", "ahash",
"hashify", "hashify",
@@ -7044,7 +7042,7 @@ checksum = "1c25ef604ac7dd839d44d64648952ea23c97866f124ff671b0ed2cf3ad9bb06e"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
@@ -7225,9 +7223,9 @@ dependencies = [
[[package]] [[package]]
name = "rustix" name = "rustix"
version = "1.1.4" version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190" checksum = "891efababe418670775f199f0d233d84843c227a0949a883ce15b37c78d6629d"
dependencies = [ dependencies = [
"bitflags 2.13.2", "bitflags 2.13.2",
"errno", "errno",
@@ -7412,12 +7410,12 @@ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"serde_derive_internals", "serde_derive_internals",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
name = "scim" name = "scim"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"ahash", "ahash",
"base64 0.23.1", "base64 0.23.1",
@@ -7443,7 +7441,7 @@ dependencies = [
[[package]] [[package]]
name = "scim-proto" name = "scim-proto"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"hashify", "hashify",
"serde", "serde",
@@ -7580,7 +7578,7 @@ dependencies = [
"sha2 0.10.9", "sha2 0.10.9",
"sha3 0.10.9", "sha3 0.10.9",
"slh-dsa", "slh-dsa",
"thiserror 1.0.69", "thiserror 2.0.20",
"twofish", "twofish",
"typenum", "typenum",
"x25519-dalek", "x25519-dalek",
@@ -7624,7 +7622,7 @@ checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
@@ -7635,7 +7633,7 @@ checksum = "f852137cce035d6a4df67ccce505ff6b3e9fd3a10e3e52b24dc71e650bb1a9bd"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
@@ -7671,7 +7669,7 @@ checksum = "8d3b1629de253c70a0508c3899572da79ca359fdab27c7920ff00406df418906"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
@@ -7716,7 +7714,7 @@ dependencies = [
"darling 0.24.1", "darling 0.24.1",
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
@@ -7764,12 +7762,12 @@ checksum = "a22144e767da4ddd8416dbf383700542ffd8a5dc493dfecedfe1fe3ad03c98ae"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
name = "services" name = "services"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"aes-gcm 0.11.1", "aes-gcm 0.11.1",
"aho-corasick", "aho-corasick",
@@ -7960,8 +7958,6 @@ checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
[[package]] [[package]]
name = "sieve-rs" name = "sieve-rs"
version = "0.7.3" version = "0.7.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bd00a548fde57bd0c8e7c13ae65fc5fe30bd923ff8655c81e010f3f02a90997b"
dependencies = [ dependencies = [
"ahash", "ahash",
"arc-swap", "arc-swap",
@@ -8084,7 +8080,7 @@ checksum = "ba467056f1b547ed52077911161fc86985becbc60e8e1857c8a144dab0def891"
[[package]] [[package]]
name = "smtp" name = "smtp"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"ahash", "ahash",
"base64 0.23.1", "base64 0.23.1",
@@ -8099,7 +8095,7 @@ dependencies = [
"mail-builder 1.0.0", "mail-builder 1.0.0",
"mail-parser", "mail-parser",
"parking_lot", "parking_lot",
"rand 0.10.2", "rand 0.10.3",
"registry", "registry",
"reqwest 0.13.5", "reqwest 0.13.5",
"rkyv", "rkyv",
@@ -8175,7 +8171,7 @@ dependencies = [
[[package]] [[package]]
name = "spam-filter" name = "spam-filter"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"common", "common",
"compact_str", "compact_str",
@@ -8295,7 +8291,7 @@ checksum = "a2eb9349b6444b326872e140eb1cf5e7c522154d69e7a0ffb0fb81c06b37543f"
[[package]] [[package]]
name = "store" name = "store"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"ahash", "ahash",
"arc-swap", "arc-swap",
@@ -8321,7 +8317,7 @@ dependencies = [
"parking_lot", "parking_lot",
"r2d2", "r2d2",
"radsort", "radsort",
"rand 0.10.2", "rand 0.10.3",
"rayon", "rayon",
"redis", "redis",
"registry", "registry",
@@ -8417,9 +8413,9 @@ dependencies = [
[[package]] [[package]]
name = "syn" name = "syn"
version = "3.0.5" version = "3.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "12df2e0110f65b775f769bb17ef989067a1d931b2eb822bd4346631eeada89f9" checksum = "8593e8e72159ed2257d083c7a454a85cbf854f37a0966d8d483aff8c8a3ebcee"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
@@ -8446,6 +8442,17 @@ dependencies = [
"syn 2.0.119", "syn 2.0.119",
] ]
[[package]]
name = "synstructure"
version = "0.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "901704edd0dfe137f1987838ee4f259e4e063c31371bdb423f7ae38ec6f77f02"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.6",
]
[[package]] [[package]]
name = "sysinfo" name = "sysinfo"
version = "0.37.2" version = "0.37.2"
@@ -8544,7 +8551,7 @@ dependencies = [
[[package]] [[package]]
name = "tests" name = "tests"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"ahash", "ahash",
"aws-lc-rs", "aws-lc-rs",
@@ -8566,7 +8573,7 @@ dependencies = [
"form_urlencoded", "form_urlencoded",
"futures", "futures",
"groupware", "groupware",
"http 0.16.22", "http 0.16.23",
"http_proto", "http_proto",
"hyper", "hyper",
"hyper-util", "hyper-util",
@@ -8581,6 +8588,7 @@ dependencies = [
"mail-builder 1.0.0", "mail-builder 1.0.0",
"mail-parser", "mail-parser",
"managesieve", "managesieve",
"migration",
"nlp", "nlp",
"pop3", "pop3",
"quick-xml 0.41.0", "quick-xml 0.41.0",
@@ -8652,7 +8660,7 @@ checksum = "bc04cd3e1236dd4a98afca4569f2deb3f120e5422a4023be2cb683f8486292af"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
@@ -8790,7 +8798,7 @@ checksum = "78773a2a397f451582ce068015985c33193cf6dea8b74d2a639fe457b2f07b0e"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
@@ -8812,7 +8820,7 @@ dependencies = [
"pin-project-lite", "pin-project-lite",
"postgres-protocol", "postgres-protocol",
"postgres-types", "postgres-types",
"rand 0.10.2", "rand 0.10.3",
"socket2 0.6.5", "socket2 0.6.5",
"tokio", "tokio",
"tokio-util", "tokio-util",
@@ -8997,7 +9005,7 @@ dependencies = [
"constant_time_eq", "constant_time_eq",
"hmac 0.13.0", "hmac 0.13.0",
"percent-encoding", "percent-encoding",
"rand 0.10.2", "rand 0.10.3",
"serde", "serde",
"sha1 0.11.0", "sha1 0.11.0",
"sha2 0.11.0", "sha2 0.11.0",
@@ -9136,7 +9144,7 @@ dependencies = [
[[package]] [[package]]
name = "trc" name = "trc"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"ahash", "ahash",
"base64 0.23.1", "base64 0.23.1",
@@ -9195,7 +9203,7 @@ dependencies = [
"http 1.5.0", "http 1.5.0",
"httparse", "httparse",
"log", "log",
"rand 0.10.2", "rand 0.10.3",
"sha1 0.11.0", "sha1 0.11.0",
"thiserror 2.0.20", "thiserror 2.0.20",
] ]
@@ -9245,7 +9253,7 @@ checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20"
[[package]] [[package]]
name = "types" name = "types"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"blake3", "blake3",
"compact_str", "compact_str",
@@ -9297,9 +9305,9 @@ checksum = "0b993bddc193ae5bd0d623b49ec06ac3e9312875fdae725a975c51db1cc1677f"
[[package]] [[package]]
name = "unicode-ident" name = "unicode-ident"
version = "1.0.24" version = "1.0.26"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" checksum = "d245f478577f809a851594d02313b640fb437e0bb33866753cff937863096954"
[[package]] [[package]]
name = "unicode-normalization" name = "unicode-normalization"
@@ -9414,7 +9422,7 @@ checksum = "b6c140620e7ffbb22c2dee59cafe6084a59b5ffc27a8859a5f0d494b5d52b6be"
[[package]] [[package]]
name = "utils" name = "utils"
version = "0.16.22" version = "0.16.23"
dependencies = [ dependencies = [
"ahash", "ahash",
"arcstr", "arcstr",
@@ -9616,7 +9624,7 @@ dependencies = [
"bumpalo", "bumpalo",
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 3.0.5", "syn 3.0.6",
"wasm-bindgen-shared", "wasm-bindgen-shared",
] ]
@@ -10125,14 +10133,14 @@ dependencies = [
[[package]] [[package]]
name = "yoke-derive" name = "yoke-derive"
version = "0.8.2" version = "0.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "de844c262c8848816172cef550288e7dc6c7b7814b4ee56b3e1553f275f1858e" checksum = "33811428bee40dbceb6d545e95754741d17a6aef9a4849f0fd62e2ba4f412a78"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 2.0.119", "syn 3.0.6",
"synstructure", "synstructure 0.14.0",
] ]
[[package]] [[package]]
@@ -10655,14 +10663,14 @@ dependencies = [
[[package]] [[package]]
name = "zerofrom-derive" name = "zerofrom-derive"
version = "0.1.7" version = "0.1.8"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "11532158c46691caf0f2593ea8358fed6bbf68a0315e80aae9bd41fbade684a1" checksum = "f75b4683f6c7f45248d4d64056a24298c6281e0993356d7d1b4a1a962ef10d4a"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 2.0.119", "syn 3.0.6",
"synstructure", "synstructure 0.14.0",
] ]
[[package]] [[package]]
@@ -10717,7 +10725,7 @@ checksum = "34df6fc39dbd26ddc9c10e6a2984476e13acce22e64e4487636ef494369225da"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quote", "quote",
"syn 3.0.5", "syn 3.0.6",
] ]
[[package]] [[package]]
@@ -10749,9 +10757,9 @@ dependencies = [
[[package]] [[package]]
name = "zlib-rs" name = "zlib-rs"
version = "0.6.7" version = "0.6.8"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "34b31d188d9d685a4f9c7b46d6e36631b07058d2cfe190267adce54dc230bf12" checksum = "b268e58e7c693d7c271f93ffc4ba3b380412554231c85bf61ca7af91042a4112"
[[package]] [[package]]
name = "zmij" name = "zmij"
+9
View File
@@ -1,5 +1,7 @@
[workspace] [workspace]
resolver = "2" resolver = "2"
# Vendored crates are patched in below, not built as members.
exclude = ["vendor"]
members = [ members = [
"crates/main", "crates/main",
"crates/types", "crates/types",
@@ -78,3 +80,10 @@ incremental = false
debug-assertions = false debug-assertions = false
overflow-checks = false overflow-checks = false
rpath = false rpath = false
# inbuxa: sieve-rs spells upstream's name into its Sieve extension names
# (vnd.stalwart.*), which scripts `require` and ManageSieve advertises.
# vendor/sieve-rs is the published 0.7.3 with those renamed; see its
# VENDORED.md. Re-vendor when the version in Cargo.lock moves.
[patch.crates-io]
sieve-rs = { path = "vendor/sieve-rs" }
+4
View File
@@ -19,6 +19,10 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
g++-x86-64-linux-gnu binutils-x86-64-linux-gnu g++-x86-64-linux-gnu binutils-x86-64-linux-gnu
RUN rustup target add "$(cat /target.txt)" RUN rustup target add "$(cat /target.txt)"
COPY --from=planner /recipe.json /recipe.json COPY --from=planner /recipe.json /recipe.json
# inbuxa: [patch.crates-io] points sieve-rs at vendor/, and the recipe only
# carries the workspace's own manifests, so cooking the dependencies needs the
# vendored crate itself (the context allows it since #27; this puts it here).
COPY vendor/ vendor/
RUN RUSTFLAGS="$(cat /flags.txt)" cargo chef cook --target "$(cat /target.txt)" --release --no-default-features --features "sqlite postgres mysql rocks s3 redis azure nats" --recipe-path /recipe.json RUN RUSTFLAGS="$(cat /flags.txt)" cargo chef cook --target "$(cat /target.txt)" --release --no-default-features --features "sqlite postgres mysql rocks s3 redis azure nats" --recipe-path /recipe.json
COPY . . COPY . .
RUN RUSTFLAGS="$(cat /flags.txt)" cargo build --target "$(cat /target.txt)" --release -p inbuxa --no-default-features --features "sqlite postgres mysql rocks s3 redis azure nats" RUN RUSTFLAGS="$(cat /flags.txt)" cargo build --target "$(cat /target.txt)" --release -p inbuxa --no-default-features --features "sqlite postgres mysql rocks s3 redis azure nats"
+8 -7
View File
@@ -8,13 +8,13 @@
--- ---
**INBUXA** is a mail and collaboration server: JMAP, IMAP, POP3, SMTP, **inbuxa** is a mail and collaboration server: JMAP, IMAP, POP3, SMTP,
CalDAV, CardDAV and WebDAV, in one Rust binary, with ihasmail as its web front CalDAV, CardDAV and WebDAV, in one Rust binary, with ihasmail as its web front
end. It is a fork of [Stalwart](https://github.com/stalwartlabs/stalwart). end. It is a fork of [Stalwart](https://github.com/stalwartlabs/stalwart).
Project site: [inbuxa.org](https://inbuxa.org). Documentation: [docs.inbuxa.org](https://docs.inbuxa.org). Project site: [inbuxa.org](https://inbuxa.org). Documentation: [docs.inbuxa.org](https://docs.inbuxa.org).
Stalwart ships some features only in a paid Enterprise Edition: multi-tenancy, Stalwart ships some features only in a paid Enterprise Edition: multi-tenancy,
masked email, undelete and others. INBUXA ships everything to everybody under masked email, undelete and others. **inbuxa** ships everything to everybody under
the AGPL-3.0, rebuilding those features independently and without using any the AGPL-3.0, rebuilding those features independently and without using any
of Stalwart's Enterprise code. of Stalwart's Enterprise code.
@@ -46,25 +46,26 @@ docker build -t inbuxa . # or the container image
``` ```
Settings are read from `INBUXA_*` environment variables. An existing Stalwart Settings are read from `INBUXA_*` environment variables. An existing Stalwart
install's `STALWART_*` variables still work, with a warning to rename them. install's `STALWART_*` variables aren't read: the server stops at startup and
names each one to rename.
New installs keep their data in `/var/lib/inbuxa` and logs in New installs keep their data in `/var/lib/inbuxa` and logs in
`/var/log/inbuxa`. Existing installs keep the paths their configuration `/var/log/inbuxa`. Existing installs keep the paths their configuration
already names, so none of their data moves. already names, so none of their data moves.
## License and credits ## License and credits
INBUXA is free software under the [GNU Affero General Public License, **inbuxa** is free software under the [GNU Affero General Public License,
version 3](./LICENSES/AGPL-3.0-only.txt). version 3](./LICENSES/AGPL-3.0-only.txt).
It is a fork of Stalwart, copyright © Stalwart Labs LLC, **modified by It is a fork of Stalwart, copyright © Stalwart Labs LLC, **modified by
Coffey Labs in 2026**. Upstream's copyright notices are kept on every file Coffey Labs in 2026**. Upstream's copyright notices are kept on every file
they cover, and every upstream file this fork changed says so in its header, they cover, and every upstream file this fork changed says so in its header,
under the notice it came with. Stalwart's files are dual-licensed under the notice it came with. Stalwart's files are dual-licensed
AGPL-3.0-only or Stalwart's Enterprise License, and INBUXA takes them under AGPL-3.0-only or Stalwart's Enterprise License, and **inbuxa** takes them under
the AGPL-3.0 only. A few of those files also carry code from other projects the AGPL-3.0 only. A few of those files also carry code from other projects
under MIT or BSD licenses, which stays under those licenses; under MIT or BSD licenses, which stays under those licenses;
[THIRD-PARTY.md](./THIRD-PARTY.md) lists it with its notices. "Stalwart" is [THIRD-PARTY.md](./THIRD-PARTY.md) lists it with its notices. "Stalwart" is
Stalwart Labs' name. INBUXA isn't affiliated with or endorsed by Stalwart Stalwart Labs' name. **inbuxa** isn't affiliated with or endorsed by Stalwart
Labs. Labs.
The INBUXA mark reuses ihasmail's cat-and-envelope artwork. The **inbuxa** mark reuses ihasmail's cat-and-envelope artwork.
+1
View File
@@ -24,6 +24,7 @@ carry their own license files.
| `crates/common/src/network/acme/directory.rs`, `crates/common/src/network/acme/jose.rs`, `crates/common/src/network/acme/order.rs` | [rustls-acme](https://github.com/FlorianUekermann/rustls-acme) (MIT or Apache-2.0) | Copyright (c) Florian Uekermann | | `crates/common/src/network/acme/directory.rs`, `crates/common/src/network/acme/jose.rs`, `crates/common/src/network/acme/order.rs` | [rustls-acme](https://github.com/FlorianUekermann/rustls-acme) (MIT or Apache-2.0) | Copyright (c) Florian Uekermann |
| `crates/types/src/id.rs` | [crockford](https://github.com/archer884/crockford) (MIT or Apache-2.0) | Copyright (c) 2017 J/A <archer884@gmail.com> | | `crates/types/src/id.rs` | [crockford](https://github.com/archer884/crockford) (MIT or Apache-2.0) | Copyright (c) 2017 J/A <archer884@gmail.com> |
| `crates/nlp/src/tokenizers/types.rs` | test cases from [linkify](https://github.com/robinst/linkify) (MIT or Apache-2.0) | Copyright (c) 2017 Robin Stocker | | `crates/nlp/src/tokenizers/types.rs` | test cases from [linkify](https://github.com/robinst/linkify) (MIT or Apache-2.0) | Copyright (c) 2017 Robin Stocker |
| `resources/spam-filter/spam-filter-rules.json.gz` | the published rules of [spam-filter](https://github.com/stalwartlabs/spam-filter) v3.0.2, unmodified, built into the server as its default spam rules (MIT or Apache-2.0) | Copyright (C) 2024, Stalwart Labs LLC |
Each notice above applies with this permission notice: Each notice above applies with this permission notice:
+7 -7
View File
@@ -1,8 +1,8 @@
openapi: 3.0.3 openapi: 3.0.3
info: info:
title: Stalwart Management API title: inbuxa Management API
description: | description: |
REST Management API for Stalwart server. These endpoints are helpers REST Management API for the inbuxa server. These endpoints are helpers
that complement the JMAP API — most of the server's configuration and data that complement the JMAP API — most of the server's configuration and data
is managed via JMAP (see `POST /jmap/`). The endpoints documented here cover is managed via JMAP (see `POST /jmap/`). The endpoints documented here cover
interactive login, account introspection, configuration schema retrieval and interactive login, account introspection, configuration schema retrieval and
@@ -12,11 +12,11 @@ info:
name: AGPL-3.0-only OR LicenseRef-SEL name: AGPL-3.0-only OR LicenseRef-SEL
servers: servers:
- url: https://{host} - url: https://{host}
description: Stalwart server description: inbuxa server
variables: variables:
host: host:
default: mail.example.com default: mail.example.com
description: The hostname of Stalwart server description: The hostname of the inbuxa server
security: security:
- bearerAuth: [] - bearerAuth: []
- basicAuth: [] - basicAuth: []
@@ -154,7 +154,7 @@ paths:
operationId: getSchema operationId: getSchema
summary: Return the configuration schema at a specific hash summary: Return the configuration schema at a specific hash
description: | description: |
Returns the JSON Schema describing the full Stalwart configuration tree. Returns the JSON Schema describing the full inbuxa configuration tree.
The response is always gzip-encoded (`Content-Encoding: gzip`) and served The response is always gzip-encoded (`Content-Encoding: gzip`) and served
with an immutable cache policy — the schema for a given hash never with an immutable cache policy — the schema for a given hash never
changes. If the hash does not match the server's current schema, the changes. If the hash does not match the server's current schema, the
@@ -183,7 +183,7 @@ paths:
application/json: application/json:
schema: schema:
type: object type: object
description: JSON Schema document describing Stalwart config description: JSON Schema document describing inbuxa config
additionalProperties: true additionalProperties: true
'302': '302':
description: Redirect to the current schema URL when the hash is stale description: Redirect to the current schema URL when the hash is stale
@@ -395,7 +395,7 @@ components:
WWW-Authenticate: WWW-Authenticate:
schema: schema:
type: string type: string
example: Bearer realm="Stalwart Server" example: Bearer realm="inbuxa Server"
content: content:
application/problem+json: application/problem+json:
schema: schema:
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "common" name = "common"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
build = "build.rs" build = "build.rs"
+147 -32
View File
@@ -11,7 +11,7 @@ use crate::{
auth::{ auth::{
AccessToken, AuthRequest, DomainCache, AccessToken, AuthRequest, DomainCache,
credential::{ApiKey, AppPassword}, credential::{ApiKey, AppPassword},
oauth::GrantType, oauth::{GrantType, token::TOKEN_HEADER},
}, },
}; };
use base64::{Engine, engine::general_purpose}; use base64::{Engine, engine::general_purpose};
@@ -23,7 +23,8 @@ use registry::schema::{
enums::Permission, enums::Permission,
structs::{self, Credential}, structs::{self, Credential},
}; };
use std::{net::IpAddr, sync::Arc}; use serde::Deserialize;
use std::{borrow::Cow, net::IpAddr, sync::Arc};
use store::write::now; use store::write::now;
use trc::AddContext; use trc::AddContext;
@@ -321,19 +322,12 @@ impl Server {
// Obtain external directory, if any. When no username is supplied // Obtain external directory, if any. When no username is supplied
// (e.g. HTTP bearer auth), peek at the JWT claims to find the // (e.g. HTTP bearer auth), peek at the JWT claims to find the
// user's domain so per-domain OIDC directories are reachable. // user's domain so per-domain OIDC directories are reachable.
let directory = if let Some(username) = username.as_deref().map(UsernameParts::new) let directory = match username.as_deref().map(UsernameParts::new) {
{ Some(username) => match username.auth_as().domain() {
if let Some(domain_name) = username.auth_as().domain() { Some(domain_name) => self.get_directory_for_domain(domain_name).await?,
self.get_directory_for_domain(domain_name).await? None => self.get_directory_for_token(token).await?,
} else if let Some(domain_name) = extract_jwt_domain(token) { },
self.get_directory_for_domain(&domain_name).await? None => self.get_directory_for_token(token).await?,
} else {
self.get_default_directory()
}
} else if let Some(domain_name) = extract_jwt_domain(token) {
self.get_directory_for_domain(&domain_name).await?
} else {
self.get_default_directory()
}; };
// Try external directory authentication first if supported, then fallback to internal OAuth. // Try external directory authentication first if supported, then fallback to internal OAuth.
@@ -563,6 +557,29 @@ impl Server {
}) })
} }
async fn get_directory_for_token(&self, token: &str) -> trc::Result<Option<&Arc<Directory>>> {
let Some(payload) = JwtClaims::decode_payload(token) else {
return Ok(self.get_default_directory());
};
let Some(claims) = JwtClaims::parse(&payload) else {
return Ok(self.get_default_directory());
};
match (claims.domain(), claims.iss.as_deref()) {
(Some(domain_name), _) => self.get_directory_for_domain(domain_name).await,
(None, Some(issuer)) => Ok(self
.get_directory_for_issuer(issuer)
.or_else(|| self.get_default_directory())),
(None, None) => Ok(self.get_default_directory()),
}
}
/// inbuxa: DIR-2: a token naming no address gets the server default, so
/// no directory is chosen by issuer.
fn get_directory_for_issuer(&self, _issuer: &str) -> Option<&Arc<Directory>> {
None
}
/// inbuxa: DIR-1, DIR-5: as above, for a domain already read. A /// inbuxa: DIR-1, DIR-5: as above, for a domain already read. A
/// `directoryId` naming no directory the server built is unavailable, /// `directoryId` naming no directory the server built is unavailable,
/// never the internal directory. /// never the internal directory.
@@ -622,25 +639,50 @@ pub fn unavailable_directory() -> &'static Arc<Directory> {
}) })
} }
fn extract_jwt_domain(token: &str) -> Option<String> { #[derive(Deserialize)]
let mut parts = token.split('.'); struct JwtClaims<'x> {
let _header = parts.next()?; #[serde(borrow, default)]
let payload = parts.next()?; iss: Option<Cow<'x, str>>,
let _signature = parts.next()?; #[serde(borrow, default)]
if parts.next().is_some() { email: Option<Cow<'x, str>>,
return None; #[serde(borrow, default)]
} preferred_username: Option<Cow<'x, str>>,
let payload_bytes = general_purpose::URL_SAFE_NO_PAD.decode(payload).ok()?; #[serde(borrow, default)]
let claims: serde_json::Value = serde_json::from_slice(&payload_bytes).ok()?; upn: Option<Cow<'x, str>>,
for claim in ["email", "preferred_username", "upn"] { }
if let Some(val) = claims.get(claim).and_then(|v| v.as_str())
&& let Some((_, domain)) = val.rsplit_once('@') impl<'x> JwtClaims<'x> {
&& !domain.is_empty() fn decode_payload(token: &str) -> Option<Vec<u8>> {
{ if token.starts_with(TOKEN_HEADER) {
return Some(domain.to_ascii_lowercase()); return None;
} }
let mut parts = token.split('.');
let _header = parts.next()?;
let payload = parts.next()?;
let _signature = parts.next()?;
if parts.next().is_some() {
return None;
}
general_purpose::URL_SAFE_NO_PAD.decode(payload).ok()
}
fn parse(payload: &'x [u8]) -> Option<Self> {
serde_json::from_slice(payload).ok()
}
fn domain(&self) -> Option<&str> {
[&self.email, &self.preferred_username, &self.upn]
.into_iter()
.flatten()
.find_map(|claim| {
claim
.rsplit_once('@')
.map(|(_, domain)| domain)
.filter(|domain| !domain.is_empty())
})
} }
None
} }
impl UsernameParts { impl UsernameParts {
@@ -738,3 +780,76 @@ impl AuthRequest {
} }
} }
} }
#[cfg(test)]
mod tests {
use super::*;
fn jwt(payload: &str) -> String {
format!(
"eyJhbGciOiJSUzI1NiJ9.{}.c2lnbmF0dXJl",
general_purpose::URL_SAFE_NO_PAD.encode(payload)
)
}
fn hints(token: &str) -> Option<(Option<String>, Option<String>)> {
let payload = JwtClaims::decode_payload(token)?;
let claims = JwtClaims::parse(&payload)?;
Some((
claims.domain().map(str::to_string),
claims.iss.as_deref().map(str::to_string),
))
}
#[test]
fn jwt_claims_are_extracted() {
for (payload, domain, issuer) in [
(
r#"{"iss":"https://idp.example.org","email":"[email protected]"}"#,
Some("Example.ORG"),
Some("https://idp.example.org"),
),
(
r#"{"preferred_username":"[email protected]","upn":"[email protected]"}"#,
Some("example.net"),
None,
),
(
r#"{"email":"broken@","upn":"[email protected]"}"#,
Some("example.com"),
None,
),
(
r#"{"iss":"https://idp.example.org","sub":"5db2d1b6","aud":["a","b"],"scope":"openid"}"#,
None,
Some("https://idp.example.org"),
),
(r#"{"sub":"5db2d1b6"}"#, None, None),
(r#"{"email":"[email protected]"}"#, Some("example.net"), None),
] {
assert_eq!(
hints(&jwt(payload)),
Some((domain.map(str::to_string), issuer.map(str::to_string))),
"Unexpected claims for {payload}"
);
}
}
#[test]
fn non_jwt_tokens_are_ignored() {
for token in [
"sw1.eyJhbGciOiJSUzI1NiJ9.eyJpc3MiOiJodHRwczovL2lkcC5leGFtcGxlLm9yZyJ9",
"sw1.eyJhbGciOiJSUzI1NiJ9",
"opaque-token",
"one.two",
"one.two.three.four",
"",
] {
assert!(
JwtClaims::decode_payload(token).is_none(),
"Token {token:?} was parsed as a JWT"
);
}
}
}
+1 -1
View File
@@ -17,7 +17,7 @@ pub const FAILED_TO_DECODE_TOKEN: &str = concat!(
"the Authentication object." "the Authentication object."
); );
const TOKEN_HEADER: &str = "sw1."; pub(crate) const TOKEN_HEADER: &str = "sw1.";
const TOKEN_KEY_CONTEXT: &str = "stalwart-oauth-token-sw1"; const TOKEN_KEY_CONTEXT: &str = "stalwart-oauth-token-sw1";
const OAUTH_EPOCH: u64 = 946684800; // Jan 1, 2000 const OAUTH_EPOCH: u64 = 946684800; // Jan 1, 2000
+284 -30
View File
@@ -13,20 +13,27 @@ use crate::{
storage::Storage, storage::Storage,
telemetry::Telemetry, telemetry::Telemetry,
}, },
ipc::{QueueEvent, RegistryChange}, ipc::{BroadcastEvent, QueueEvent, RegistryChange},
network::security::{BlockedIps, IpWithTtl}, network::security::{BlockedIps, IpWithTtl},
}; };
use ahash::AHashMap; use ahash::AHashMap;
use directory::Directories; use directory::Directories;
use registry::{ use registry::{
schema::{prelude::ObjectType, structs::BlockedIp}, schema::{prelude::ObjectType, structs::BlockedIp},
types::error::{Error, Warning}, types::{
error::{Error, Warning},
id::ObjectId,
},
}; };
use std::sync::Arc; use std::sync::Arc;
use store::{LookupStores, registry::bootstrap::Bootstrap, write::now}; use store::{LookupStores, registry::bootstrap::Bootstrap, write::now};
pub struct ReloadResult { pub struct ReloadResult {
/// Errors that kept the reload from being applied.
pub errors: Vec<Error>, pub errors: Vec<Error>,
/// inbuxa: errors in objects that already failed when the running
/// settings were built; logged, but they don't refuse a reload.
pub known_errors: Vec<Error>,
pub warnings: Vec<Warning>, pub warnings: Vec<Warning>,
pub replaced_core: bool, pub replaced_core: bool,
} }
@@ -114,42 +121,60 @@ impl Server {
directories: directory.directories, directories: directory.directories,
}; };
// Parse tracers // inbuxa: upstream swapped the core only when the whole build
// was free of errors, while boot runs with whatever built. So one
// object that failed (a DNS lookup that timed out, say) refused
// every later reload, cluster-wide when the reload came from
// ReloadSettings, and the running settings went stale. Now a
// reload is refused only for errors in objects that built when
// the running settings were built: those would be lost by
// applying it. Objects that already failed then are missing
// from the running settings anyway, as at boot, so their
// errors are reported but don't hold the reload back.
let tracers = Telemetry::parse(&mut bootstrap, &storage).await; let tracers = Telemetry::parse(&mut bootstrap, &storage).await;
let core = Box::pin(Core::parse(&mut bootstrap, storage)).await;
let mut servers = Listeners::parse(&mut bootstrap).await;
if bootstrap.errors.is_empty() { if !self.has_new_build_errors(&bootstrap.errors) {
let core = Box::pin(Core::parse(&mut bootstrap, storage)).await; servers
.parse_tcp_acceptors(&mut bootstrap, self.inner.clone())
.await;
if bootstrap.errors.is_empty() { if !self.has_new_build_errors(&bootstrap.errors) {
let mut servers = Listeners::parse(&mut bootstrap).await; // Update core
servers self.inner.shared_core.store(core.into());
.parse_tcp_acceptors(&mut bootstrap, self.inner.clone())
.await;
if bootstrap.errors.is_empty() { // Update tracers
// Update core tracers.update();
self.inner.shared_core.store(core.into());
// Update tracers // Reload queue settings
self.inner
.ipc
.queue_tx
.send(QueueEvent::ReloadSettings)
.await
.ok();
tracers.update(); self.record_build_errors(&bootstrap.errors);
// Reload queue settings return Ok(ReloadResult {
self.inner errors: Vec::new(),
.ipc known_errors: bootstrap.errors,
.queue_tx warnings: bootstrap.warnings,
.send(QueueEvent::ReloadSettings) replaced_core: true,
.await });
.ok();
return Ok(ReloadResult {
errors: bootstrap.errors,
warnings: bootstrap.warnings,
replaced_core: true,
});
}
} }
} }
let (known_errors, errors) = std::mem::take(&mut bootstrap.errors)
.into_iter()
.partition(|error| self.is_known_build_error(error));
return Ok(ReloadResult {
errors,
known_errors,
warnings: bootstrap.warnings,
replaced_core: false,
});
} }
} }
@@ -163,7 +188,7 @@ impl ReloadResult {
} }
pub fn log(&self) { pub fn log(&self) {
for error in &self.errors { for error in self.errors.iter().chain(&self.known_errors) {
error.log(); error.log();
} }
for warning in &self.warnings { for warning in &self.warnings {
@@ -176,8 +201,237 @@ impl From<Bootstrap> for ReloadResult {
fn from(bootstrap: Bootstrap) -> Self { fn from(bootstrap: Bootstrap) -> Self {
Self { Self {
errors: bootstrap.errors, errors: bootstrap.errors,
known_errors: Vec::new(),
warnings: bootstrap.warnings, warnings: bootstrap.warnings,
replaced_core: false, replaced_core: false,
} }
} }
} }
// inbuxa: which objects failed to build for the running settings
impl Server {
/// Records the objects that failed to build for the settings now running.
pub fn record_build_errors(&self, errors: &[Error]) {
*self.inner.data.build_errors.lock() = errors.iter().filter_map(error_object).collect();
}
fn is_known_build_error(&self, error: &Error) -> bool {
error_object(error).is_some_and(|id| self.inner.data.build_errors.lock().contains(&id))
}
fn has_new_build_errors(&self, errors: &[Error]) -> bool {
errors.iter().any(|error| !self.is_known_build_error(error))
}
}
fn error_object(error: &Error) -> Option<ObjectId> {
match error {
Error::Validation { object_id, .. }
| Error::Build { object_id, .. }
| Error::NotFound { object_id } => Some(*object_id),
Error::Internal { object_id, .. } => *object_id,
}
}
// inbuxa: upstream applied a registry write to the running settings only on
// an explicit x:Action ReloadSettings (Directory and Authentication aside), so
// a new MtaDeliverySchedule, say, stayed unknown ("Queue strategy not found")
// until someone reloaded. Writes to objects the settings are built from now
// reload them, here and across the cluster, as ReloadSettings does.
/// Coalesces the full reloads that registry writes trigger: a write waits for
/// a reload that started after it was stored, and joins one if it can, so a
/// burst of writes costs a reload or two rather than one each.
#[derive(Default)]
pub struct SettingsReloadGate {
requested: std::sync::atomic::AtomicU64,
state: tokio::sync::Mutex<SettingsReloadState>,
}
#[derive(Default)]
struct SettingsReloadState {
completed: u64,
refused: Option<String>,
}
/// The reload a write to `object` calls for: the object to reload, or None
/// when the running settings don't hold that object (accounts, domains and
/// other data read as needed, stores, which take a restart, and objects with
/// reload actions of their own, such as applications). Blocked IPs have a
/// reload of their own; allowed IPs take the full one.
pub fn write_reload_target(object: ObjectType) -> Option<ObjectType> {
match object {
ObjectType::Certificate => Some(ObjectType::Certificate),
ObjectType::MemoryLookupKey
| ObjectType::MemoryLookupKeyValue
| ObjectType::HttpLookup
| ObjectType::StoreLookup => Some(ObjectType::StoreLookup),
ObjectType::BlockedIp => Some(ObjectType::BlockedIp),
// Allowed IPs are part of the core's security settings
// (Security::parse), which only a full reload rebuilds; the blocked-IP
// reload doesn't touch them
ObjectType::AllowedIp
| ObjectType::AcmeProvider
| ObjectType::AddressBook
| ObjectType::AiModel
| ObjectType::Asn
| ObjectType::Authentication
| ObjectType::Cache
| ObjectType::Calendar
| ObjectType::CalendarAlarm
| ObjectType::CalendarScheduling
| ObjectType::ClusterRole
| ObjectType::DataRetention
| ObjectType::Directory
| ObjectType::DkimReportSettings
| ObjectType::DmarcReportSettings
| ObjectType::DnsResolver
| ObjectType::DsnReportSettings
| ObjectType::Email
| ObjectType::EventTracingLevel
| ObjectType::FileStorage
| ObjectType::Http
| ObjectType::HttpForm
| ObjectType::Imap
| ObjectType::Jmap
| ObjectType::Metrics
| ObjectType::MtaConnectionStrategy
| ObjectType::MtaDeliverySchedule
| ObjectType::MtaExtensions
| ObjectType::MtaHook
| ObjectType::MtaInboundSession
| ObjectType::MtaInboundThrottle
| ObjectType::MtaMilter
| ObjectType::MtaOutboundStrategy
| ObjectType::MtaOutboundThrottle
| ObjectType::MtaQueueQuota
| ObjectType::MtaRoute
| ObjectType::MtaStageAuth
| ObjectType::MtaStageConnect
| ObjectType::MtaStageData
| ObjectType::MtaStageEhlo
| ObjectType::MtaStageMail
| ObjectType::MtaStageRcpt
| ObjectType::MtaSts
| ObjectType::MtaTlsStrategy
| ObjectType::MtaVirtualQueue
| ObjectType::NetworkListener
| ObjectType::OidcProvider
| ObjectType::ReportSettings
| ObjectType::Search
| ObjectType::Security
| ObjectType::SenderAuth
| ObjectType::Sharing
| ObjectType::SieveSystemInterpreter
| ObjectType::SieveSystemScript
| ObjectType::SieveUserInterpreter
| ObjectType::SieveUserScript
| ObjectType::SpamClassifier
| ObjectType::SpamDnsblServer
| ObjectType::SpamDnsblSettings
| ObjectType::SpamFileExtension
| ObjectType::SpamPyzor
| ObjectType::SpamRule
| ObjectType::SpamSettings
| ObjectType::SpamTag
| ObjectType::SpfReportSettings
| ObjectType::SystemSettings
| ObjectType::TaskManager
| ObjectType::TlsReportSettings
| ObjectType::Tracer
| ObjectType::WebDav
| ObjectType::WebHook => Some(object),
_ => None,
}
}
impl Server {
/// Applies a stored registry write to `object` to the running settings,
/// and on success tells the other nodes to do the same. Returns None when
/// the write needs no reload, Some(Ok(())) when it was applied, and
/// Some(Err(reason)) when the reload was refused (the write stays stored;
/// ReloadSettings reports the same errors).
pub async fn reload_after_write(&self, object: ObjectType) -> Option<Result<(), String>> {
let target = write_reload_target(object)?;
let change = RegistryChange::Reload(target);
if matches!(
target,
ObjectType::Certificate | ObjectType::StoreLookup | ObjectType::BlockedIp
) {
// Cheap, and limited to their own objects
let result = self.reload_and_broadcast(change).await;
return Some(result);
}
let gate = &self.inner.data.settings_reload;
let ticket = gate
.requested
.fetch_add(1, std::sync::atomic::Ordering::SeqCst)
+ 1;
let mut state = gate.state.lock().await;
if state.completed >= ticket {
// A reload that started after this write was stored has run
return Some(state.refused.clone().map_or(Ok(()), Err));
}
let covers = gate.requested.load(std::sync::atomic::Ordering::SeqCst);
let result = self.reload_and_broadcast(change).await;
state.completed = covers;
state.refused = result.clone().err();
Some(result)
}
async fn reload_and_broadcast(&self, change: RegistryChange) -> Result<(), String> {
match Box::pin(self.reload_registry(change)).await {
Ok(reload) if !reload.has_errors() => {
reload.log();
self.cluster_broadcast(BroadcastEvent::RegistryChange(change))
.await;
Ok(())
}
Ok(reload) => {
reload.log();
let reason = describe_reload_errors(&reload.errors);
trc::event!(
Registry(trc::RegistryEvent::BuildWarning),
Details = "Settings didn't reload after a registry write",
Reason = reason.clone(),
);
Err(reason)
}
Err(err) => {
let reason = err.to_string();
trc::error!(err.details("Failed to reload settings after a registry write"));
Err(reason)
}
}
}
}
/// inbuxa: a refused reload's errors in a sentence: the first one, naming its
/// object, and how many more there are.
pub fn describe_reload_errors(errors: &[Error]) -> String {
let mut description = match errors.first() {
Some(Error::Build { object_id, message }) => format!("{object_id}: {message}"),
Some(Error::Validation { object_id, errors }) => format!(
"{object_id}: {}",
errors
.iter()
.map(|err| err.to_string())
.collect::<Vec<_>>()
.join("; ")
),
Some(Error::Internal {
object_id: Some(object_id),
error,
}) => format!("{object_id}: {error}"),
Some(Error::Internal { error, .. }) => error.to_string(),
Some(Error::NotFound { object_id }) => format!("{object_id} was not found"),
None => String::new(),
};
let more = errors.len().saturating_sub(1);
if more > 0 {
description.push_str(&format!(" ({more} more in the server log.)"));
}
description
}
+6
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
use super::server::tls::build_self_signed_cert; use super::server::tls::build_self_signed_cert;
@@ -91,9 +93,11 @@ impl Data {
registry_id_gen: id_generator.clone(), registry_id_gen: id_generator.clone(),
span_id_gen: id_generator, span_id_gen: id_generator,
queue_status: true.into(), queue_status: true.into(),
settings_reload: Default::default(),
applications, applications,
logos: Default::default(), logos: Default::default(),
smtp_connectors: TlsConnectors::try_new().failed("Failed to build TLS connectors"), smtp_connectors: TlsConnectors::try_new().failed("Failed to build TLS connectors"),
build_errors: Default::default(),
asn_geo_data: Default::default(), asn_geo_data: Default::default(),
} }
} }
@@ -232,9 +236,11 @@ impl Default for Data {
span_id_gen: Default::default(), span_id_gen: Default::default(),
registry_id_gen: Default::default(), registry_id_gen: Default::default(),
queue_status: true.into(), queue_status: true.into(),
settings_reload: Default::default(),
applications: WebApplications::new(), applications: WebApplications::new(),
logos: Default::default(), logos: Default::default(),
smtp_connectors: TlsConnectors::try_new().unwrap(), smtp_connectors: TlsConnectors::try_new().unwrap(),
build_errors: Default::default(),
asn_geo_data: Default::default(), asn_geo_data: Default::default(),
lookup_stores: Default::default(), lookup_stores: Default::default(),
} }
+23 -1
View File
@@ -143,7 +143,9 @@ impl Scripting {
.with_cpu_limit(trusted.max_cpu_cycles as usize) .with_cpu_limit(trusted.max_cpu_cycles as usize)
.with_max_nested_includes(trusted.max_nested_includes as usize) .with_max_nested_includes(trusted.max_nested_includes as usize)
.with_max_received_headers(trusted.max_received_headers as usize) .with_max_received_headers(trusted.max_received_headers as usize)
.with_default_duplicate_expiry(trusted.duplicate_expiry.into_inner().as_secs()); .with_default_duplicate_expiry(trusted.duplicate_expiry.into_inner().as_secs())
// inbuxa: without it, `environment "name"` answers sieve-rs's default
.with_env_variable("name", types::brand_server!());
trusted_runtime.set_local_hostname(local_hostname.clone()); trusted_runtime.set_local_hostname(local_hostname.clone());
untrusted_runtime.set_local_hostname(local_hostname); untrusted_runtime.set_local_hostname(local_hostname);
@@ -279,3 +281,23 @@ impl Clone for Scripting {
} }
} }
} }
#[cfg(test)]
mod tests {
use sieve::compiler::grammar::Capability;
// inbuxa: sieve-rs is vendored (vendor/sieve-rs) to carry the fork's
// name in its Sieve extensions. If Cargo.lock moves sieve-rs past the
// vendored version, Cargo drops the patch with only a warning and
// upstream's spelling comes back; this fails instead.
#[test]
fn sieve_extensions_carry_the_fork_name() {
for (capability, name) in [
(Capability::While, "vnd.inbuxa.while"),
(Capability::Expressions, "vnd.inbuxa.expressions"),
] {
assert_eq!(capability.to_string(), name);
assert_eq!(Capability::parse(name), capability);
}
}
}
@@ -16,7 +16,6 @@ use mail_auth::common::resolver::ToReverseName;
use nlp::classifier::model::{CcfhClassifier, FhClassifier}; use nlp::classifier::model::{CcfhClassifier, FhClassifier};
use registry::schema::{ use registry::schema::{
enums::{ExpressionVariable, ModelSize}, enums::{ExpressionVariable, ModelSize},
prelude::ObjectType,
structs::{ structs::{
self, SpamDnsblServer, SpamDnsblSettings, SpamFileExtension, SpamPyzor, SpamRule, self, SpamDnsblServer, SpamDnsblSettings, SpamFileExtension, SpamPyzor, SpamRule,
SpamSettings, SpamTag, SpamSettings, SpamTag,
@@ -25,10 +24,10 @@ use registry::schema::{
use sieve::SpamStatus; use sieve::SpamStatus;
use std::{ use std::{
net::{IpAddr, SocketAddr}, net::{IpAddr, SocketAddr},
time::Duration, sync::Arc,
time::{Duration, Instant},
}; };
use store::registry::{RegistryObject, bootstrap::Bootstrap}; use store::registry::{RegistryObject, bootstrap::Bootstrap};
use tokio::net::lookup_host;
use utils::{cache::CacheItemWeight, glob::GlobMap}; use utils::{cache::CacheItemWeight, glob::GlobMap};
#[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default)] #[derive(rkyv::Archive, rkyv::Deserialize, rkyv::Serialize, Debug, Default)]
@@ -157,7 +156,11 @@ pub struct FtrlParameters {
#[derive(Debug, Clone)] #[derive(Debug, Clone)]
pub struct PyzorConfig { pub struct PyzorConfig {
pub address: SocketAddr, // inbuxa: the server is resolved when a message is checked, not while the
// settings are built (see PyzorConfig::address)
pub host: String,
pub port: u16,
pub resolved: Arc<parking_lot::Mutex<Option<(SocketAddr, Instant)>>>,
pub timeout: Duration, pub timeout: Duration,
pub min_count: u64, pub min_count: u64,
pub min_wl_count: u64, pub min_wl_count: u64,
@@ -243,7 +246,8 @@ impl SpamFilterConfig {
spam_threshold: spam.score_spam.into_inner() as f32, spam_threshold: spam.score_spam.into_inner() as f32,
}, },
grey_list_expiry: spam.greylist_for.map(|d| d.into_inner().as_secs()), grey_list_expiry: spam.greylist_for.map(|d| d.into_inner().as_secs()),
spam_rules_url: spam.spam_filter_rules_url, // inbuxa: unset, empty or upstream's old default means the bundled rules
spam_rules_url: crate::manager::spam_rules::rules_url(spam.spam_filter_rules_url),
url_client: utils::http::http_client_builder(true) url_client: utils::http::http_client_builder(true)
.pool_max_idle_per_host(0) .pool_max_idle_per_host(0)
.redirect(reqwest::redirect::Policy::none()) .redirect(reqwest::redirect::Policy::none())
@@ -473,31 +477,15 @@ impl PyzorConfig {
return None; return None;
} }
let port = pyzor.port; // inbuxa: upstream resolved the host here and reported a failed lookup
let host = pyzor.host; // as a build error, so a DNS hiccup on one node refused every settings
let address = match lookup_host(format!("{host}:{port}")) // reload on it (and, from the node that ran ReloadSettings, across the
.await // cluster). The lookup now happens when a message is checked; a
.map(|mut a| a.next()) // failure there is logged as a Pyzor error for that message.
{
Ok(Some(address)) => address,
Ok(None) => {
bp.build_error(
ObjectType::SpamPyzor.singleton(),
"Invalid address: No addresses found.",
);
return None;
}
Err(err) => {
bp.build_error(
ObjectType::SpamPyzor.singleton(),
format!("Invalid address: {}", err),
);
return None;
}
};
PyzorConfig { PyzorConfig {
address, host: pyzor.host,
port: pyzor.port as u16,
resolved: Default::default(),
timeout: pyzor.timeout.into_inner(), timeout: pyzor.timeout.into_inner(),
min_count: pyzor.block_count, min_count: pyzor.block_count,
min_wl_count: pyzor.allow_count, min_wl_count: pyzor.allow_count,
@@ -507,6 +495,35 @@ impl PyzorConfig {
} }
} }
// inbuxa: how long a resolved Pyzor address is reused
const PYZOR_RESOLVE_TTL: Duration = Duration::from_secs(300);
impl PyzorConfig {
/// The server's address: the host itself when it is an IP address,
/// otherwise the first address it resolves to, reused for five minutes.
pub async fn address(&self) -> std::io::Result<SocketAddr> {
if let Ok(ip) = self.host.parse::<IpAddr>() {
return Ok(SocketAddr::new(ip, self.port));
}
if let Some((address, resolved_at)) = *self.resolved.lock()
&& resolved_at.elapsed() < PYZOR_RESOLVE_TTL
{
return Ok(address);
}
let address = tokio::net::lookup_host((self.host.as_str(), self.port))
.await?
.next()
.ok_or_else(|| {
std::io::Error::new(
std::io::ErrorKind::NotFound,
format!("{} has no addresses", self.host),
)
})?;
*self.resolved.lock() = Some((address, Instant::now()));
Ok(address)
}
}
impl ClassifierConfig { impl ClassifierConfig {
pub async fn parse(bp: &mut Bootstrap) -> Option<Self> { pub async fn parse(bp: &mut Bootstrap) -> Option<Self> {
let classifier = bp.setting_infallible::<structs::SpamClassifier>().await; let classifier = bp.setting_infallible::<structs::SpamClassifier>().await;
@@ -214,6 +214,7 @@ impl Resolvers {
let config_dnssec = resolver_config.clone(); let config_dnssec = resolver_config.clone();
let mut opts_dnssec = opts.clone(); let mut opts_dnssec = opts.clone();
opts_dnssec.validate = true; opts_dnssec.validate = true;
opts_dnssec.num_concurrent_reqs = 1;
let dnssec = DnssecResolver { let dnssec = DnssecResolver {
resolver: TokioResolver::builder_with_config( resolver: TokioResolver::builder_with_config(
@@ -343,6 +344,7 @@ impl Default for Resolvers {
let config_dnssec = config.clone(); let config_dnssec = config.clone();
let mut opts_dnssec = opts.clone(); let mut opts_dnssec = opts.clone();
opts_dnssec.validate = true; opts_dnssec.validate = true;
opts_dnssec.num_concurrent_reqs = 1;
Self { Self {
dns: MessageAuthenticator::new(config, opts).expect("Failed to build DNS resolver"), dns: MessageAuthenticator::new(config, opts).expect("Failed to build DNS resolver"),
+13 -14
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
use self::resolver::Policy; use self::resolver::Policy;
@@ -22,7 +24,7 @@ use registry::schema::{
}; };
use smtp_proto::*; use smtp_proto::*;
use std::{ use std::{
net::{SocketAddr, ToSocketAddrs}, net::{IpAddr, SocketAddr},
str::FromStr, str::FromStr,
time::Duration, time::Duration,
}; };
@@ -384,19 +386,16 @@ impl SessionConfig {
Some(Milter { Some(Milter {
enable: bp.compile_expr(id, &milter.ctx_enable()), enable: bp.compile_expr(id, &milter.ctx_enable()),
id, id,
addrs: format!("{}:{}", milter.hostname, milter.port) // inbuxa: upstream resolved the hostname here (a
.to_socket_addrs() // blocking lookup) and made a failure a build error,
.map_err(|err| { // which refused the whole settings reload. An IP
bp.build_error( // address is kept as is; a name is resolved on each
id, // connection (MilterClient::connect).
format!( addrs: milter
"Unable to resolve milter hostname {}: {}", .hostname
milter.hostname, err .parse::<IpAddr>()
), .map(|ip| vec![SocketAddr::new(ip, milter.port as u16)])
) .unwrap_or_default(),
})
.ok()?
.collect(),
hostname: milter.hostname, hostname: milter.hostname,
port: milter.port as u16, port: milter.port as u16,
timeout_connect: milter.timeout_connect.into_inner(), timeout_connect: milter.timeout_connect.into_inner(),
+7
View File
@@ -23,6 +23,13 @@ pub(crate) fn fn_is_number(v: Vec<Variable>) -> Variable {
matches!(&v[0], Variable::Integer(_) | Variable::Float(_)).into() matches!(&v[0], Variable::Integer(_) | Variable::Float(_)).into()
} }
pub(crate) fn fn_bit_and(v: Vec<Variable>) -> Variable {
match (v[0].to_integer(), v[1].to_integer()) {
(Some(lhs), Some(rhs)) => Variable::Integer(lhs & rhs),
_ => Variable::Integer(0),
}
}
pub(crate) fn fn_is_ip_addr(v: Vec<Variable>) -> Variable { pub(crate) fn fn_is_ip_addr(v: Vec<Variable>) -> Variable {
v[0].to_string() v[0].to_string()
.as_str() .as_str()
+1
View File
@@ -46,6 +46,7 @@ pub(crate) const FUNCTIONS: &[(&str, fn(Vec<Variable>) -> Variable, u32)] = &[
("email_part", email::fn_email_part, 2), ("email_part", email::fn_email_part, 2),
("is_empty", misc::fn_is_empty, 1), ("is_empty", misc::fn_is_empty, 1),
("is_number", misc::fn_is_number, 1), ("is_number", misc::fn_is_number, 1),
("bit_and", misc::fn_bit_and, 2),
("is_ip_addr", misc::fn_is_ip_addr, 1), ("is_ip_addr", misc::fn_is_ip_addr, 1),
("is_ipv4_addr", misc::fn_is_ipv4_addr, 1), ("is_ipv4_addr", misc::fn_is_ipv4_addr, 1),
("is_ipv6_addr", misc::fn_is_ipv6_addr, 1), ("is_ipv6_addr", misc::fn_is_ipv6_addr, 1),
+69
View File
@@ -335,3 +335,72 @@ impl EmailPush {
} }
} }
} }
/// inbuxa: the task locks this node holds, so a graceful stop can hand them
/// back instead of leaving the tasks blocked until the locks expire.
pub struct TaskLocks {
held: parking_lot::Mutex<ahash::AHashSet<u64>>,
stopping: AtomicBool,
expiry: std::sync::atomic::AtomicU64,
}
impl TaskLocks {
/// How long a task lock lasts, in seconds, unless it is released first
/// or renewed. inbuxa: upstream held a lock for an hour, so a killed
/// node's tasks waited that long; the lock is now a five-minute lease
/// that the task manager renews every third of it while the task runs
/// (renew_task_locks), so a dead node's tasks run elsewhere within
/// minutes.
pub const DEFAULT_EXPIRY: u64 = 5 * 60;
pub fn is_stopping(&self) -> bool {
self.stopping.load(Ordering::Acquire)
}
/// Stops new claims and returns the ids of every lock still held.
pub fn stop(&self) -> Vec<u64> {
self.stopping.store(true, Ordering::Release);
self.held.lock().drain().collect()
}
pub fn insert(&self, id: u64) {
self.held.lock().insert(id);
}
pub fn remove(&self, id: u64) {
self.held.lock().remove(&id);
}
pub fn held(&self) -> usize {
self.held.lock().len()
}
/// inbuxa: the tasks this node holds, to renew their locks.
pub fn held_ids(&self) -> Vec<u64> {
self.held.lock().iter().copied().collect()
}
/// inbuxa: whether this node holds (and is running) the task.
pub fn is_held(&self, id: u64) -> bool {
self.held.lock().contains(&id)
}
pub fn expiry(&self) -> u64 {
self.expiry.load(Ordering::Relaxed)
}
/// Changes the lock lifetime; the tests shorten it.
pub fn set_expiry(&self, seconds: u64) {
self.expiry.store(seconds.max(1), Ordering::Relaxed);
}
}
impl Default for TaskLocks {
fn default() -> Self {
Self {
held: Default::default(),
stopping: AtomicBool::new(false),
expiry: std::sync::atomic::AtomicU64::new(Self::DEFAULT_EXPIRY),
}
}
}
+8
View File
@@ -161,11 +161,17 @@ pub struct Data {
pub span_id_gen: SnowflakeIdGenerator, pub span_id_gen: SnowflakeIdGenerator,
pub registry_id_gen: SnowflakeIdGenerator, pub registry_id_gen: SnowflakeIdGenerator,
pub queue_status: AtomicBool, pub queue_status: AtomicBool,
// inbuxa: coalesces the settings reloads registry writes trigger
pub settings_reload: cache::reload::SettingsReloadGate,
pub applications: WebApplications, pub applications: WebApplications,
pub logos: Mutex<AHashMap<Box<str>, LogoCache>>, pub logos: Mutex<AHashMap<Box<str>, LogoCache>>,
pub smtp_connectors: TlsConnectors, pub smtp_connectors: TlsConnectors,
// inbuxa: the objects that failed to build when the running settings
// were built, at boot or by the last applied reload (see reload_registry)
pub build_errors: Mutex<AHashSet<registry::types::id::ObjectId>>,
} }
#[derive(Clone)] #[derive(Clone)]
@@ -279,6 +285,8 @@ pub struct HttpAuthCache {
pub struct Ipc { pub struct Ipc {
pub push_tx: mpsc::Sender<PushEvent>, pub push_tx: mpsc::Sender<PushEvent>,
pub task_tx: Arc<Notify>, pub task_tx: Arc<Notify>,
// inbuxa: task locks held by this node, released on a graceful stop
pub task_locks: Arc<crate::ipc::TaskLocks>,
pub queue_tx: mpsc::Sender<QueueEvent>, pub queue_tx: mpsc::Sender<QueueEvent>,
pub report_tx: mpsc::Sender<ReportingEvent>, pub report_tx: mpsc::Sender<ReportingEvent>,
pub broadcast_tx: Option<mpsc::Sender<BroadcastEvent>>, pub broadcast_tx: Option<mpsc::Sender<BroadcastEvent>>,
+233 -117
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
use crate::{Server, manager::fetch_resource}; use crate::{Server, manager::fetch_resource};
@@ -11,8 +13,11 @@ use registry::schema::{enums::CompressionAlgo, structs::Application};
use std::{ use std::{
borrow::Cow, borrow::Cow,
io::{self, Cursor, Read}, io::{self, Cursor, Read},
path::PathBuf, path::{Path, PathBuf},
sync::Arc, sync::{
Arc,
atomic::{AtomicU64, Ordering},
},
time::Duration, time::Duration,
}; };
use store::{ use store::{
@@ -36,16 +41,18 @@ enum IndexEdit<'x> {
pub struct WebApplications { pub struct WebApplications {
applications: ArcSwap<Vec<WebApplicationManager>>, applications: ArcSwap<Vec<WebApplicationManager>>,
routes: ArcSwap<AHashMap<String, Arc<AppRoutes>>>, routes: ArcSwap<AHashMap<String, Arc<AppRoutes>>>,
generation: AtomicU64,
} }
pub struct AppRoutes { pub struct AppRoutes {
resources: AHashMap<String, Resource<PathBuf>>, resources: AHashMap<String, Resource<PathBuf>>,
oauth_client_id_meta: Option<String>, oauth_client_id_meta: Option<String>,
_bundle_dir: TempDir,
} }
#[derive(Clone)] #[derive(Clone)]
pub struct WebApplicationManager { pub struct WebApplicationManager {
bundle_path: TempDir, base_path: PathBuf,
prefixes: Vec<String>, prefixes: Vec<String>,
description: String, description: String,
url: String, url: String,
@@ -79,6 +86,7 @@ impl WebApplications {
Self { Self {
applications: ArcSwap::new(Arc::new(Vec::new())), applications: ArcSwap::new(Arc::new(Vec::new())),
routes: ArcSwap::new(Arc::new(AHashMap::new())), routes: ArcSwap::new(Arc::new(AHashMap::new())),
generation: AtomicU64::new(0),
} }
} }
@@ -128,48 +136,55 @@ impl WebApplications {
} }
pub async fn unpack_all(&self, server: &Server, update: bool) { pub async fn unpack_all(&self, server: &Server, update: bool) {
let mut routes = AHashMap::new(); let previous = self.routes.load_full();
let sweep_orphans = previous.is_empty();
let mut routes = AHashMap::with_capacity(previous.len());
for app in self.applications.load().as_ref() { for app in self.applications.load().as_ref() {
if update && let Err(err) = app.delete(server).await { match app
trc::event!( .unpack(server, self.next_generation(), update, sweep_orphans)
Resource(trc::ResourceEvent::Error), .await
Reason = err, {
Url = app.url.clone(), Ok(app_routes) => {
Details = format!( let app_routes = Arc::new(app_routes);
"Failed to delete application bundle for prefixes: {}",
app.prefixes.join(", ")
)
);
}
match app.unpack(server).await {
Ok(resources) => {
let app_routes = Arc::new(AppRoutes {
resources,
oauth_client_id_meta: app
.oauth_client_id
.as_deref()
.map(oauth_client_id_meta),
});
for prefix in &app.prefixes { for prefix in &app.prefixes {
routes.insert(prefix.clone(), app_routes.clone()); routes.insert(prefix.clone(), app_routes.clone());
} }
} }
Err(err) => { Err(err) => {
let mut is_retained = false;
for prefix in &app.prefixes {
if let Some(app_routes) = previous.get(prefix) {
routes.insert(prefix.clone(), app_routes.clone());
is_retained = true;
}
}
trc::event!( trc::event!(
Resource(trc::ResourceEvent::Error), Resource(trc::ResourceEvent::Error),
Reason = err, Reason = err,
Url = app.url.clone(), Url = app.url.clone(),
Details = format!( Details = format!(
"Failed to unpack application for prefixes: {}", "Failed to unpack application for prefixes: {}, {}",
app.prefixes.join(", ") app.prefixes.join(", "),
if is_retained {
"the previously unpacked bundle remains in service"
} else {
"no bundle is available to serve"
}
) )
); );
} }
} }
} }
self.routes.store(Arc::new(routes)); self.routes.store(Arc::new(routes));
} }
fn next_generation(&self) -> u64 {
self.generation.fetch_add(1, Ordering::Relaxed)
}
} }
impl WebApplicationManager { impl WebApplicationManager {
@@ -182,7 +197,7 @@ impl WebApplicationManager {
.join(app.id.id().to_string()); .join(app.id.id().to_string());
Self { Self {
bundle_path: TempDir::new(base_path), base_path,
blob_key: BlobHash::generate(format!("{}{}", APP_BLOB_PREFIX, app.id.id()).as_bytes()), blob_key: BlobHash::generate(format!("{}{}", APP_BLOB_PREFIX, app.id.id()).as_bytes()),
url: app.object.resource_url, url: app.object.resource_url,
description: app.object.description, description: app.object.description,
@@ -202,82 +217,43 @@ impl WebApplicationManager {
} }
} }
async fn unpack(&self, server: &Server) -> trc::Result<AHashMap<String, Resource<PathBuf>>> { async fn unpack(
// Delete any existing bundles &self,
self.bundle_path.clean().await.map_err(unpack_error)?; server: &Server,
generation: u64,
// Obtain application bundle force_refresh: bool,
let bundle = if let Some(bundle) = server sweep_orphans: bool,
.blob_store() ) -> trc::Result<AppRoutes> {
.get_blob(self.blob_key.as_slice(), 0..usize::MAX) let cached = if force_refresh {
.await? None
{
bundle
} else { } else {
// Fetch app bundle
let resource = fetch_resource(&self.url, None, Duration::from_secs(60), MAX_APP_SIZE)
.await
.map_err(|err| {
trc::ResourceEvent::Error
.caused_by(trc::location!())
.ctx(Key::Url, self.url.clone())
.reason(err)
.details("Failed to fetch application bundle")
})?;
// Store in blob store for future use
server server
.blob_store() .blob_store()
.put_blob(self.blob_key.as_slice(), &resource, CompressionAlgo::None) .get_blob(self.blob_key.as_slice(), 0..usize::MAX)
.await .await?
.caused_by(trc::location!())?; };
let is_cached = cached.is_some();
// Schedule expiration let bundle = match cached {
let mut batch = BatchBuilder::new(); Some(bundle) => bundle,
batch None => self.fetch().await?,
.set(
BlobOp::Link {
hash: self.blob_key.clone(),
to: BlobLink::Temporary {
until: now() + self.expiry,
},
},
vec![],
)
.set(
BlobOp::Commit {
hash: self.blob_key.clone(),
},
Vec::new(),
);
server
.store()
.write(batch.build_all())
.await
.caused_by(trc::location!())?;
trc::event!(
Resource(trc::ResourceEvent::ApplicationUpdated),
Url = self.url.clone(),
Details = self.description.clone(),
);
resource
}; };
let staging = TempDir::new(self.base_path.join(format!("{:x}-{generation:x}", now())));
staging.create().await.map_err(unpack_error)?;
let url = self.url.clone(); let url = self.url.clone();
let bundle_path = self.bundle_path.path.clone(); let bundle_path = staging.path.clone();
let routes = tokio::task::spawn_blocking(move || -> trc::Result<_> { let (resources, bundle) = tokio::task::spawn_blocking(move || -> trc::Result<_> {
let mut bundle = zip::ZipArchive::new(Cursor::new(bundle)).map_err(|err| { let mut archive = zip::ZipArchive::new(Cursor::new(bundle)).map_err(|err| {
trc::ResourceEvent::Error trc::ResourceEvent::Error
.caused_by(trc::location!()) .caused_by(trc::location!())
.reason(err) .reason(err)
.ctx(Key::Url, url.clone()) .ctx(Key::Url, url.clone())
.details("Failed to decompress application bundle") .details("Failed to decompress application bundle")
})?; })?;
let mut routes = AHashMap::new(); let mut resources = AHashMap::with_capacity(archive.len());
for i in 0..bundle.len() { for i in 0..archive.len() {
let mut file = bundle.by_index(i).map_err(|err| { let mut file = archive.by_index(i).map_err(|err| {
trc::ResourceEvent::Error trc::ResourceEvent::Error
.caused_by(trc::location!()) .caused_by(trc::location!())
.reason(err) .reason(err)
@@ -315,9 +291,9 @@ impl WebApplicationManager {
contents: path, contents: path,
}; };
routes.insert(file_name, resource); resources.insert(file_name, resource);
} }
Ok(routes) Ok((resources, archive.into_inner().into_inner()))
}) })
.await .await
.map_err(|err| { .map_err(|err| {
@@ -327,21 +303,81 @@ impl WebApplicationManager {
.details("Bundle unpack task panicked") .details("Bundle unpack task panicked")
})??; })??;
if !is_cached && let Err(err) = self.cache(server, &bundle).await {
trc::event!(
Resource(trc::ResourceEvent::Error),
Reason = err,
Url = self.url.clone(),
Details = "Failed to cache application bundle, it will be downloaded again"
);
}
if sweep_orphans {
remove_siblings(&self.base_path, &staging.path).await;
}
trc::event!( trc::event!(
Resource(trc::ResourceEvent::ApplicationUnpacked), Resource(trc::ResourceEvent::ApplicationUnpacked),
Url = self.url.clone(), Url = self.url.clone(),
Path = self.bundle_path.path.to_string_lossy().into_owned(), Path = staging.path.to_string_lossy().into_owned(),
); );
Ok(routes) Ok(AppRoutes {
resources,
oauth_client_id_meta: self.oauth_client_id.as_deref().map(oauth_client_id_meta),
_bundle_dir: staging,
})
} }
async fn delete(&self, server: &Server) -> trc::Result<()> { async fn fetch(&self) -> trc::Result<Vec<u8>> {
fetch_resource(&self.url, None, Duration::from_secs(60), MAX_APP_SIZE)
.await
.map_err(|err| {
trc::ResourceEvent::Error
.caused_by(trc::location!())
.ctx(Key::Url, self.url.clone())
.reason(err)
.details("Failed to fetch application bundle")
})
}
async fn cache(&self, server: &Server, bundle: &[u8]) -> trc::Result<()> {
server server
.blob_store() .blob_store()
.delete_blob(self.blob_key.as_slice()) .put_blob(self.blob_key.as_slice(), bundle, CompressionAlgo::None)
.await .await
.map(|_| ()) .caused_by(trc::location!())?;
let mut batch = BatchBuilder::new();
batch
.set(
BlobOp::Link {
hash: self.blob_key.clone(),
to: BlobLink::Temporary {
until: now() + self.expiry,
},
},
vec![],
)
.set(
BlobOp::Commit {
hash: self.blob_key.clone(),
},
Vec::new(),
);
server
.store()
.write(batch.build_all())
.await
.caused_by(trc::location!())?;
trc::event!(
Resource(trc::ResourceEvent::ApplicationUpdated),
Url = self.url.clone(),
Details = self.description.clone(),
);
Ok(())
} }
pub async fn delete_bundle(server: &Server, app_id: Id) -> trc::Result<()> { pub async fn delete_bundle(server: &Server, app_id: Id) -> trc::Result<()> {
@@ -361,7 +397,6 @@ impl Resource<Vec<u8>> {
} }
} }
#[derive(Clone)]
pub struct TempDir { pub struct TempDir {
pub path: PathBuf, pub path: PathBuf,
} }
@@ -371,11 +406,36 @@ impl TempDir {
TempDir { path } TempDir { path }
} }
pub async fn clean(&self) -> io::Result<()> { pub async fn create(&self) -> io::Result<()> {
if tokio::fs::metadata(&self.path).await.is_ok() { if tokio::fs::metadata(&self.path).await.is_ok() {
let _ = tokio::fs::remove_dir_all(&self.path).await; let _ = tokio::fs::remove_dir_all(&self.path).await;
} }
tokio::fs::create_dir(&self.path).await tokio::fs::create_dir_all(&self.path).await
}
}
impl Drop for TempDir {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.path);
}
}
async fn remove_siblings(base_path: &Path, keep: &Path) {
let Ok(mut entries) = tokio::fs::read_dir(base_path).await else {
return;
};
while let Ok(Some(entry)) = entries.next_entry().await {
let path = entry.path();
if path == keep {
continue;
}
if matches!(entry.file_type().await, Ok(file_type) if file_type.is_dir()) {
let _ = tokio::fs::remove_dir_all(&path).await;
} else {
let _ = tokio::fs::remove_file(&path).await;
}
} }
} }
@@ -385,12 +445,6 @@ fn unpack_error(err: std::io::Error) -> trc::Error {
.details("Failed to unpack application bundle") .details("Failed to unpack application bundle")
} }
impl Drop for TempDir {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.path);
}
}
impl Default for WebApplications { impl Default for WebApplications {
fn default() -> Self { fn default() -> Self {
Self::new() Self::new()
@@ -460,12 +514,12 @@ mod tests {
#[test] #[test]
fn index_is_rewritten_with_the_prefix_and_client_id() { fn index_is_rewritten_with_the_prefix_and_client_id() {
let meta = oauth_client_id_meta("stalwart-webui"); let meta = oauth_client_id_meta("inbuxa-webui");
let html = String::from_utf8(rewrite_index(INDEX, "admin", Some(&meta))).unwrap(); let html = String::from_utf8(rewrite_index(INDEX, "admin", Some(&meta))).unwrap();
assert!(html.contains("<base href=\"/admin/\" />"), "{html}"); assert!(html.contains("<base href=\"/admin/\" />"), "{html}");
assert!( assert!(
html.contains("<meta name=\"oauth-client-id\" content=\"stalwart-webui\" />"), html.contains("<meta name=\"oauth-client-id\" content=\"inbuxa-webui\" />"),
"{html}" "{html}"
); );
assert!(html.contains("<title>Portal</title>"), "{html}"); assert!(html.contains("<title>Portal</title>"), "{html}");
@@ -487,7 +541,7 @@ mod tests {
#[test] #[test]
fn index_without_a_placeholder_is_left_alone() { fn index_without_a_placeholder_is_left_alone() {
let bundle = "<head>\n <base href=\"/\" />\n</head>"; let bundle = "<head>\n <base href=\"/\" />\n</head>";
let meta = oauth_client_id_meta("stalwart-webui"); let meta = oauth_client_id_meta("inbuxa-webui");
let html = String::from_utf8(rewrite_index(bundle, "admin", Some(&meta))).unwrap(); let html = String::from_utf8(rewrite_index(bundle, "admin", Some(&meta))).unwrap();
assert_eq!(html, "<head>\n <base href=\"/admin/\" />\n</head>"); assert_eq!(html, "<head>\n <base href=\"/admin/\" />\n</head>");
@@ -521,9 +575,9 @@ mod tests {
); );
} }
async fn fixture(name: &str, client_id: Option<&str>) -> (WebApplications, TempDir) { async fn fixture(name: &str, client_id: Option<&str>) -> WebApplications {
let dir = TempDir::new(std::env::temp_dir().join(format!("inbuxa-app-{name}"))); let dir = TempDir::new(std::env::temp_dir().join(format!("inbuxa-app-{name}")));
dir.clean().await.unwrap(); dir.create().await.unwrap();
tokio::fs::write(dir.path.join("index.html"), INDEX) tokio::fs::write(dir.path.join("index.html"), INDEX)
.await .await
.unwrap(); .unwrap();
@@ -544,6 +598,7 @@ mod tests {
let routes = Arc::new(AppRoutes { let routes = Arc::new(AppRoutes {
resources, resources,
oauth_client_id_meta: client_id.map(oauth_client_id_meta), oauth_client_id_meta: client_id.map(oauth_client_id_meta),
_bundle_dir: dir,
}); });
let mut map = AHashMap::new(); let mut map = AHashMap::new();
@@ -553,7 +608,7 @@ mod tests {
let apps = WebApplications::new(); let apps = WebApplications::new();
apps.routes.store(Arc::new(map)); apps.routes.store(Arc::new(map));
(apps, dir) apps
} }
async fn serve_html(apps: &WebApplications, prefix: &str, path: &str) -> String { async fn serve_html(apps: &WebApplications, prefix: &str, path: &str) -> String {
@@ -565,7 +620,7 @@ mod tests {
#[tokio::test] #[tokio::test]
async fn serving_index_injects_the_prefix_and_client_id() { async fn serving_index_injects_the_prefix_and_client_id() {
let (apps, _dir) = fixture("serve-configured", Some("pocket-id-client")).await; let apps = fixture("serve-configured", Some("pocket-id-client")).await;
let html = serve_html(&apps, "admin", "index.html").await; let html = serve_html(&apps, "admin", "index.html").await;
assert!(html.contains("<base href=\"/admin/\" />"), "{html}"); assert!(html.contains("<base href=\"/admin/\" />"), "{html}");
@@ -584,7 +639,7 @@ mod tests {
#[tokio::test] #[tokio::test]
async fn unknown_paths_fall_back_to_a_rewritten_index() { async fn unknown_paths_fall_back_to_a_rewritten_index() {
let (apps, _dir) = fixture("serve-fallback", Some("pocket-id-client")).await; let apps = fixture("serve-fallback", Some("pocket-id-client")).await;
let html = serve_html(&apps, "admin", "settings/directory").await; let html = serve_html(&apps, "admin", "settings/directory").await;
assert!(html.contains("<base href=\"/admin/\" />"), "{html}"); assert!(html.contains("<base href=\"/admin/\" />"), "{html}");
@@ -596,7 +651,7 @@ mod tests {
#[tokio::test] #[tokio::test]
async fn assets_and_unknown_prefixes_are_untouched() { async fn assets_and_unknown_prefixes_are_untouched() {
let (apps, _dir) = fixture("serve-assets", Some("pocket-id-client")).await; let apps = fixture("serve-assets", Some("pocket-id-client")).await;
let served = apps.serve("admin", "app.js").await.unwrap().unwrap(); let served = apps.serve("admin", "app.js").await.unwrap().unwrap();
assert_eq!(served.resource.contents, b"export const x = 1;\n"); assert_eq!(served.resource.contents, b"export const x = 1;\n");
@@ -608,7 +663,7 @@ mod tests {
#[tokio::test] #[tokio::test]
async fn serving_index_without_a_client_id_keeps_the_placeholder() { async fn serving_index_without_a_client_id_keeps_the_placeholder() {
let (apps, _dir) = fixture("serve-unconfigured", None).await; let apps = fixture("serve-unconfigured", None).await;
let html = serve_html(&apps, "admin", "index.html").await; let html = serve_html(&apps, "admin", "index.html").await;
assert!(html.contains("<base href=\"/admin/\" />"), "{html}"); assert!(html.contains("<base href=\"/admin/\" />"), "{html}");
@@ -624,4 +679,65 @@ mod tests {
assert_eq!(rewrite_index(bundle, "admin", None), bundle.as_bytes()); assert_eq!(rewrite_index(bundle, "admin", None), bundle.as_bytes());
} }
#[tokio::test]
async fn missing_parent_directories_are_created() {
let base = std::env::temp_dir().join("inbuxa-app-nested");
let _ = tokio::fs::remove_dir_all(&base).await;
let dir = TempDir::new(base.join("webui").join("0"));
dir.create().await.unwrap();
assert!(tokio::fs::metadata(&dir.path).await.is_ok());
drop(dir);
let _ = tokio::fs::remove_dir_all(&base).await;
}
#[tokio::test]
async fn dropping_the_routes_removes_the_bundle_directory() {
let apps = fixture("drop-guard", None).await;
let path = apps
.routes
.load()
.get("admin")
.unwrap()
._bundle_dir
.path
.clone();
assert!(tokio::fs::metadata(&path).await.is_ok());
apps.routes.store(Arc::new(AHashMap::new()));
assert!(tokio::fs::metadata(&path).await.is_err());
}
#[tokio::test]
async fn sweeping_orphans_spares_the_current_generation() {
let base = std::env::temp_dir().join("inbuxa-app-sweep");
let _ = tokio::fs::remove_dir_all(&base).await;
let current = TempDir::new(base.join("1"));
current.create().await.unwrap();
let orphan = base.join("0");
tokio::fs::create_dir_all(&orphan).await.unwrap();
let stray = base.join("webui.zip");
tokio::fs::write(&stray, b"not a bundle").await.unwrap();
remove_siblings(&base, &current.path).await;
assert!(tokio::fs::metadata(&current.path).await.is_ok());
assert!(tokio::fs::metadata(&orphan).await.is_err());
assert!(tokio::fs::metadata(&stray).await.is_err());
drop(current);
let _ = tokio::fs::remove_dir_all(&base).await;
}
#[test]
fn generations_never_repeat() {
let apps = WebApplications::new();
assert_ne!(apps.next_generation(), apps.next_generation());
}
} }
+25 -6
View File
@@ -23,6 +23,13 @@ use utils::{UnwrapFailure, codec::leb128::Leb128_};
pub(super) const MAGIC_MARKER: u8 = 123; pub(super) const MAGIC_MARKER: u8 = 123;
// inbuxa: blobs kept under a fixed name instead of a content hash. Nothing
// links to them, so the export names them outright.
const NAMED_BLOBS: &[&[u8]] = &[
crate::manager::SPAM_CLASSIFIER_KEY,
crate::manager::SPAM_TRAINER_KEY,
];
#[derive(Debug, Clone, Copy, Hash, PartialEq, Eq)] #[derive(Debug, Clone, Copy, Hash, PartialEq, Eq)]
pub(super) enum Family { pub(super) enum Family {
Data = 0, Data = 0,
@@ -143,15 +150,21 @@ impl Core {
.await .await
.failed("Failed to iterate over data store"); .failed("Failed to iterate over data store");
for hash in blobs { // inbuxa: the trained spam classifier and its trainer state are
// blobs stored under fixed names with no blob link, so the walk
// over links above never reaches them.
let named = NAMED_BLOBS.iter().map(|key| key.to_vec());
for key in blobs
.into_iter()
.map(|hash| hash.as_slice().to_vec())
.chain(named)
{
if let Some(blob) = blob_store if let Some(blob) = blob_store
.get_blob(hash.as_slice(), 0..usize::MAX) .get_blob(&key, 0..usize::MAX)
.await .await
.failed("Failed to get blob") .failed("Failed to get blob")
{ {
writer writer.send((key, blob)).failed("Failed to send key");
.send((hash.as_slice().to_vec(), blob))
.failed("Failed to send key");
} }
} }
}), }),
@@ -323,7 +336,13 @@ impl Family {
SUBSPACE_REGISTRY_IDX, SUBSPACE_REGISTRY_IDX,
SUBSPACE_REGISTRY_PK, SUBSPACE_REGISTRY_PK,
SUBSPACE_DIRECTORY, SUBSPACE_DIRECTORY,
store::SUBSPACE_INBUXA, // inbuxa: masked email // inbuxa: registry objects the upstream list left out, so an
// export dropped them: archived items (undelete) and spam
// training samples. Their indexes and id counters already
// travel in this family and in `data`, so they ride along.
SUBSPACE_DELETED_ITEMS,
SUBSPACE_SPAM_SAMPLES,
store::SUBSPACE_INBUXA, // inbuxa: the fork's own data (masked email, undelete, policies)
], ],
Family::Changelog => &[SUBSPACE_LOGS], Family::Changelog => &[SUBSPACE_LOGS],
Family::Queue => &[SUBSPACE_QUEUE_MESSAGE, SUBSPACE_QUEUE_EVENT], Family::Queue => &[SUBSPACE_QUEUE_MESSAGE, SUBSPACE_QUEUE_EVENT],
+15 -4
View File
@@ -54,6 +54,13 @@ Options:
-o, --console Open the store console -o, --console Open the store console
-h, --help Print help -h, --help Print help
-V, --version Print version -V, --version Print version
An export holds everything in the data and blob stores except short-lived
in-memory state (rate limits, locks, greylisting) and the full-text search
index, which belongs to one search backend. An import into an empty store
queues the index to be rebuilt when the server next starts. EXPORT_TYPES
limits an export to some of: data, registry, blob, changelog, queue, report,
telemetry, tasks.
"# "#
); );
@@ -233,6 +240,9 @@ impl BootManager {
.parse_tcp_acceptors(&mut bootstrap, inner.clone()) .parse_tcp_acceptors(&mut bootstrap, inner.clone())
.await; .await;
// inbuxa: a reload isn't refused over objects that failed here
inner.build_server().record_build_errors(&bootstrap.errors);
BootManager { BootManager {
inner, inner,
bootstrap, bootstrap,
@@ -256,10 +266,10 @@ impl BootManager {
telemetry.enable(); telemetry.enable();
// Parse settings and restore // Parse settings and restore
Box::pin(Core::parse(&mut bootstrap, storage)) let core = Box::pin(Core::parse(&mut bootstrap, storage)).await;
.await let imported = core.restore(path).await;
.restore(path) // inbuxa: the search index isn't exported; rebuild it
.await; core.queue_reindex(&imported).await;
std::process::exit(0); std::process::exit(0);
} }
StoreOp::Console => { StoreOp::Console => {
@@ -290,6 +300,7 @@ pub fn build_ipc(has_pubsub: bool) -> (Ipc, IpcReceivers) {
report_tx, report_tx,
broadcast_tx: has_pubsub.then_some(broadcast_tx), broadcast_tx: has_pubsub.then_some(broadcast_tx),
task_tx: Arc::new(Notify::new()), task_tx: Arc::new(Notify::new()),
task_locks: Arc::new(crate::ipc::TaskLocks::default()),
train_task_controller: Arc::new(TrainTaskController::default()), train_task_controller: Arc::new(TrainTaskController::default()),
}, },
IpcReceivers { IpcReceivers {
+14 -5
View File
@@ -530,13 +530,22 @@ async fn insert_safe_defaults(bp: &mut Bootstrap) -> trc::Result<()> {
use store::write::BatchBuilder; use store::write::BatchBuilder;
use types::id::Id; use types::id::Id;
if bp.registry.count_object(ObjectType::SpamRule).await? == 0 // inbuxa: rules are always to hand, since a copy ships with the server
&& bp // (spam_rules). They load on first boot, and again when the bundled
.registry // version differs from the one last loaded, which only adds what's
// missing: new tags and rules, never a changed score.
let rules_url = super::spam_rules::rules_url(
bp.registry
.object::<SpamSettings>(Id::singleton()) .object::<SpamSettings>(Id::singleton())
.await? .await?
.is_none_or(|spam| spam.spam_filter_rules_url.is_some()) .and_then(|spam| spam.spam_filter_rules_url),
{ );
let bundled_is_new = rules_url.is_none()
&& super::spam_rules::applied_version(&bp.data_store)
.await?
.as_deref()
!= Some(super::spam_rules::BUNDLED_SPAM_RULES_VERSION);
if bp.registry.count_object(ObjectType::SpamRule).await? == 0 || bundled_is_new {
let mut batch = BatchBuilder::new(); let mut batch = BatchBuilder::new();
batch.schedule_task(Task::SpamFilterMaintenance(TaskSpamFilterMaintenance { batch.schedule_task(Task::SpamFilterMaintenance(TaskSpamFilterMaintenance {
maintenance_type: TaskSpamFilterMaintenanceType::UpdateRules, maintenance_type: TaskSpamFilterMaintenanceType::UpdateRules,
+68 -10
View File
@@ -10,7 +10,7 @@
//! that ship with it are registered for it, on every start: //! that ship with it are registered for it, on every start:
//! //!
//! - the web interface the server serves itself (`Application`, `/admin` and //! - the web interface the server serves itself (`Application`, `/admin` and
//! `/account`), as its OAuth client id, `stalwart-webui` unless the //! `/account`), as its OAuth client id, `inbuxa-webui` unless the
//! application names another; //! application names another;
//! - INBUXA Admin hosted elsewhere, as `inbuxa-admin`, when `INBUXA_ADMIN_URL` //! - INBUXA Admin hosted elsewhere, as `inbuxa-admin`, when `INBUXA_ADMIN_URL`
//! is set; //! is set;
@@ -29,7 +29,7 @@ use directory::core::secret::{hash_secret, verify_secret_hash};
use registry::{ use registry::{
schema::{ schema::{
enums::{PasswordHashAlgorithm, ServiceProtocol}, enums::{PasswordHashAlgorithm, ServiceProtocol},
prelude::{ObjectType, Property, UTCDateTime}, prelude::{Object, ObjectInner, ObjectType, Property, UTCDateTime},
structs::{Application, OAuthClient, SystemSettings}, structs::{Application, OAuthClient, SystemSettings},
}, },
types::map::Map, types::map::Map,
@@ -40,9 +40,12 @@ use store::registry::{
}; };
/// The client id the upstream web interface uses when its application names none. /// The client id the upstream web interface uses when its application names none.
pub const WEB_INTERFACE_CLIENT_ID: &str = "stalwart-webui"; pub const WEB_INTERFACE_CLIENT_ID: &str = "inbuxa-webui";
pub const ADMIN_CLIENT_ID: &str = "inbuxa-admin"; pub const ADMIN_CLIENT_ID: &str = "inbuxa-admin";
pub const WEBMAIL_CLIENT_ID: &str = "ihasmail-inbuxa"; pub const WEBMAIL_CLIENT_ID: &str = "ihasmail-inbuxa";
/// The web interface's client id before the fork renamed it (SPEC §2.4).
/// Only ever read to retire it.
const LEGACY_WEB_INTERFACE_CLIENT_ID: &str = "stalwart-webui";
#[derive(Debug, Clone, PartialEq, Eq)] #[derive(Debug, Clone, PartialEq, Eq)]
pub struct FirstPartyClient { pub struct FirstPartyClient {
@@ -102,7 +105,7 @@ pub fn first_party_clients(
if let Some(url) = admin_url.map(|url| url.trim().trim_end_matches('/')).filter(|url| !url.is_empty()) { if let Some(url) = admin_url.map(|url| url.trim().trim_end_matches('/')).filter(|url| !url.is_empty()) {
clients.push(FirstPartyClient { clients.push(FirstPartyClient {
client_id: ADMIN_CLIENT_ID.to_string(), client_id: ADMIN_CLIENT_ID.to_string(),
description: "INBUXA Admin".to_string(), description: "inbuxa Admin".to_string(),
redirect_uris: vec![format!("{url}/oauth/callback")], redirect_uris: vec![format!("{url}/oauth/callback")],
secret: None, secret: None,
}); });
@@ -187,6 +190,7 @@ fn env(name: &str) -> Option<String> {
} }
pub(crate) async fn ensure_first_party_clients(bp: &mut Bootstrap) -> trc::Result<()> { pub(crate) async fn ensure_first_party_clients(bp: &mut Bootstrap) -> trc::Result<()> {
retire_legacy_web_interface_client(bp).await?;
let system = bp.setting_infallible::<SystemSettings>().await; let system = bp.setting_infallible::<SystemSettings>().await;
let base_url = base_url(bp, &system); let base_url = base_url(bp, &system);
let applications = bp let applications = bp
@@ -213,6 +217,56 @@ pub(crate) async fn ensure_first_party_clients(bp: &mut Bootstrap) -> trc::Resul
Ok(()) Ok(())
} }
/// An install from before the rename, upstream's or this fork's, has the web
/// interface registered as `stalwart-webui`, and may
/// have an application naming it. The application is moved to the current id
/// and the old client removed, so the old id stops working rather than
/// living on as an alias; anyone signed in to the web interface signs in
/// again. Runs on every start and does nothing once both are gone.
async fn retire_legacy_web_interface_client(bp: &mut Bootstrap) -> trc::Result<()> {
for app in bp.list_infallible::<Application>().await {
if app.object.oauth_client_id.as_deref() != Some(LEGACY_WEB_INTERFACE_CLIENT_ID) {
continue;
}
let mut updated = app.object.clone();
updated.oauth_client_id = Some(WEB_INTERFACE_CLIENT_ID.to_string());
// The old object carries its revision: the write asserts on it.
let current = Object::with_revision(ObjectInner::from(app.object), app.revision);
let result = bp
.registry
.write(RegistryWrite::update(app.id.id(), &updated.into(), &current))
.await?;
if !matches!(result, RegistryWriteResult::Success(_)) {
return Err(trc::StoreEvent::UnexpectedError
.into_err()
.details("Failed to move an application to the renamed web interface client.")
.reason(result.to_string())
.caused_by(trc::location!()));
}
}
if let Some(object_id) = bp
.registry
.primary_key(
ObjectType::OAuthClient.into(),
Property::ClientId,
LEGACY_WEB_INTERFACE_CLIENT_ID.as_bytes().to_vec(),
)
.await?
{
let result = bp.registry.write(RegistryWrite::delete(object_id)).await?;
if !matches!(result, RegistryWriteResult::Success(_)) {
return Err(trc::StoreEvent::UnexpectedError
.into_err()
.details("Failed to remove the web interface's pre-rename OAuth client.")
.reason(result.to_string())
.caused_by(trc::location!()));
}
}
Ok(())
}
async fn ensure_client(bp: &mut Bootstrap, client: FirstPartyClient) -> trc::Result<()> { async fn ensure_client(bp: &mut Bootstrap, client: FirstPartyClient) -> trc::Result<()> {
let existing = match bp let existing = match bp
.registry .registry
@@ -223,15 +277,18 @@ async fn ensure_client(bp: &mut Bootstrap, client: FirstPartyClient) -> trc::Res
) )
.await? .await?
{ {
// inbuxa: read as an Object, keeping the revision the update below
// asserts on (a bare OAuthClient converts back with revision 0, which
// never matches, so any update failed start-up).
Some(object_id) => bp Some(object_id) => bp
.registry .registry
.object::<OAuthClient>(object_id.id()) .get(object_id)
.await? .await?
.map(|object| (object_id.id(), object)), .map(|object| (object_id.id(), object.revision, OAuthClient::from(object))),
None => None, None => None,
}; };
let result = if let Some((id, current)) = existing { let result = if let Some((id, revision, current)) = existing {
let mut updated = current.clone(); let mut updated = current.clone();
for uri in &client.redirect_uris { for uri in &client.redirect_uris {
if !updated.redirect_uris.contains(uri) { if !updated.redirect_uris.contains(uri) {
@@ -255,8 +312,9 @@ async fn ensure_client(bp: &mut Bootstrap, client: FirstPartyClient) -> trc::Res
if updated == current { if updated == current {
return Ok(()); return Ok(());
} }
let current = Object::with_revision(ObjectInner::from(current), revision);
bp.registry bp.registry
.write(RegistryWrite::update(id, &updated.into(), &current.into())) .write(RegistryWrite::update(id, &updated.into(), &current))
.await? .await?
} else { } else {
let secret = match &client.secret { let secret = match &client.secret {
@@ -298,7 +356,7 @@ mod tests {
fn web_interface() -> Application { fn web_interface() -> Application {
Application { Application {
description: "INBUXA Web Interface".to_string(), description: "inbuxa Web Interface".to_string(),
enabled: true, enabled: true,
url_prefix: Map::new(vec!["/admin".into(), "/account".into()]), url_prefix: Map::new(vec!["/admin".into(), "/account".into()]),
..Default::default() ..Default::default()
@@ -312,7 +370,7 @@ mod tests {
clients, clients,
vec![FirstPartyClient { vec![FirstPartyClient {
client_id: WEB_INTERFACE_CLIENT_ID.to_string(), client_id: WEB_INTERFACE_CLIENT_ID.to_string(),
description: "INBUXA Web Interface (served by this server)".to_string(), description: "inbuxa Web Interface (served by this server)".to_string(),
redirect_uris: vec![ redirect_uris: vec![
"https://mail.example.org/admin/oauth/callback".to_string(), "https://mail.example.org/admin/oauth/callback".to_string(),
"https://mail.example.org/account/oauth/callback".to_string(), "https://mail.example.org/account/oauth/callback".to_string(),
+3 -2
View File
@@ -22,9 +22,10 @@ pub mod console;
pub mod defaults; pub mod defaults;
pub mod first_party; pub mod first_party;
pub mod restore; pub mod restore;
pub mod spam_rules; // inbuxa: rules bundled with the server
pub const SPAM_TRAINER_KEY: &[u8] = "STALWART_SPAM_TRAIN_DATA.lz4".as_bytes(); pub const SPAM_TRAINER_KEY: &[u8] = "INBUXA_SPAM_TRAIN_DATA.lz4".as_bytes();
pub const SPAM_CLASSIFIER_KEY: &[u8] = "STALWART_SPAM_CLASSIFIER_MODEL.lz4".as_bytes(); pub const SPAM_CLASSIFIER_KEY: &[u8] = "INBUXA_SPAM_CLASSIFIER_MODEL.lz4".as_bytes();
pub async fn fetch_resource( pub async fn fetch_resource(
url: &str, url: &str,
+84 -15
View File
@@ -9,15 +9,22 @@
use super::backup::MAGIC_MARKER; use super::backup::MAGIC_MARKER;
use crate::{Core, DATABASE_SCHEMA_VERSION}; use crate::{Core, DATABASE_SCHEMA_VERSION};
use lz4_flex::frame::FrameDecoder; use lz4_flex::frame::FrameDecoder;
use registry::schema::enums::CompressionAlgo; use registry::{
schema::{
enums::{CompressionAlgo, TaskStoreMaintenanceType},
structs::{Task, TaskStatus, TaskStoreMaintenance},
},
types::EnumImpl,
};
use std::{ use std::{
fs::File, fs::File,
io::{BufReader, ErrorKind, Read}, io::{BufReader, ErrorKind, Read},
path::{Path, PathBuf}, path::{Path, PathBuf},
}; };
use store::{ use store::{
BlobStore, IterateParams, SUBSPACE_BLOBS, SUBSPACE_COUNTER, SUBSPACE_INDEXES, SUBSPACE_QUOTA, BlobStore, IterateParams, SUBSPACE_BLOBS, SUBSPACE_COUNTER, SUBSPACE_INDEXES,
SUBSPACE_REGISTRY_PK, Store, U32_LEN, SUBSPACE_PROPERTY, SUBSPACE_QUOTA, SUBSPACE_REGISTRY_PK, SUBSPACE_TELEMETRY_SPAN, Store,
U32_LEN,
write::{ write::{
AnyClass, AnyKey, BatchBuilder, ValueClass, AnyClass, AnyKey, BatchBuilder, ValueClass,
key::{DeserializeBigEndian, is_node_id_key}, key::{DeserializeBigEndian, is_node_id_key},
@@ -27,7 +34,9 @@ use types::{collection::Collection, field::Field};
use utils::{UnwrapFailure, failed}; use utils::{UnwrapFailure, failed};
impl Core { impl Core {
pub async fn restore(&self, src: PathBuf) { /// Imports an export into an empty store and returns the subspaces it
/// wrote. inbuxa: the caller hands them to [`Core::queue_reindex`].
pub async fn restore(&self, src: PathBuf) -> Vec<u8> {
// Backup the core // Backup the core
let paths = if src.is_dir() { let paths = if src.is_dir() {
let mut paths = Vec::new(); let mut paths = Vec::new();
@@ -64,6 +73,13 @@ impl Core {
std::process::exit(1); std::process::exit(1);
} }
let mut imported = paths
.iter()
.map(|path| KeyValueReader::new(path).subspace)
.collect::<Vec<_>>();
imported.sort_unstable();
imported.dedup();
let mut tasks = Vec::new(); let mut tasks = Vec::new();
for path in paths { for path in paths {
let storage = self.storage.clone(); let storage = self.storage.clone();
@@ -76,6 +92,54 @@ impl Core {
for task in tasks { for task in tasks {
task.await.failed("Failed to wait for task"); task.await.failed("Failed to wait for task");
} }
imported
}
/// inbuxa: an export never carries the full-text index. It is built by
/// and for one search backend (the SQL stores index into their own
/// tables, the key-value stores into a subspace, external engines keep it
/// themselves), so it would be wrong or unreadable after a move to
/// another one. Instead, an import queues the same reindex tasks an
/// administrator can queue by hand (`reindexAccounts` and
/// `reindexTelemetry` store maintenance), and the server rebuilds the
/// index for whatever search store it is configured with once it starts.
pub async fn queue_reindex(&self, imported: &[u8]) -> Vec<TaskStoreMaintenanceType> {
let mut queued = Vec::new();
if imported.contains(&SUBSPACE_PROPERTY) {
queued.push(TaskStoreMaintenanceType::ReindexAccounts);
}
if imported.contains(&SUBSPACE_TELEMETRY_SPAN) {
queued.push(TaskStoreMaintenanceType::ReindexTelemetry);
}
if queued.is_empty() {
return queued;
}
let mut batch = BatchBuilder::new();
for maintenance_type in &queued {
batch.schedule_task(Task::StoreMaintenance(TaskStoreMaintenance {
maintenance_type: *maintenance_type,
status: TaskStatus::now(),
shard_index: None,
}));
}
self.storage
.data
.write(batch.build_all())
.await
.failed("Failed to queue the reindex tasks");
println!(
"Queued {} to rebuild the search index; it runs when the server starts.",
queued
.iter()
.map(|t| t.as_str())
.collect::<Vec<_>>()
.join(" and ")
);
queued
} }
} }
@@ -125,17 +189,22 @@ async fn restore_file(store: Store, blob_store: BlobStore, path: &Path) {
} }
SUBSPACE_COUNTER | SUBSPACE_QUOTA => { SUBSPACE_COUNTER | SUBSPACE_QUOTA => {
while let Some((key, value)) = reader.next() { while let Some((key, value)) = reader.next() {
batch.add( let class = ValueClass::Any(AnyClass {
ValueClass::Any(AnyClass { subspace: reader.subspace,
subspace: reader.subspace, key,
key, });
}), let value = u64::from_le_bytes(
u64::from_le_bytes( value
value .try_into()
.try_into() .expect("Failed to deserialize counter/quota"),
.expect("Failed to deserialize counter/quota"), ) as i64;
) as i64, // inbuxa: the SQL stores add a negative amount with an UPDATE,
); // which does nothing to a row that isn't there yet, so a
// negative counter vanished on import. Create the row first.
if value < 0 {
batch.add(class.clone(), 0);
}
batch.add(class, value);
if batch.is_large_batch() { if batch.is_large_batch() {
store store
.write(batch.build_all()) .write(batch.build_all())
+103
View File
@@ -0,0 +1,103 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! inbuxa: the spam filter rules that ship with the server.
//!
//! Upstream fetches its latest published rules from GitHub at run time, so
//! scoring changes with a release nobody here tested and depends on reaching
//! it. The fork embeds a pinned copy (resources/spam-filter/, with its version
//! and license) and uses it whenever no other source is configured. The rules
//! URL remains an operator override (`https://` or `file://`).
//!
//! Loading rules only ever adds what's missing, never changes an existing rule
//! or score. They load on first boot, and again whenever the bundled version
//! differs from the one last applied, so an upgrade brings new tags (the AI
//! classifier's `LLM_*` scores, say) to an install that already had rules.
use std::io::Read;
use store::{
SUBSPACE_INBUXA, Store, ValueKey,
write::{AnyClass, BatchBuilder, ValueClass},
};
use trc::AddContext;
/// The version of spam-filter the embedded rules come from.
pub const BUNDLED_SPAM_RULES_VERSION: &str = "3.0.2";
static BUNDLED_SPAM_RULES: &[u8] =
include_bytes!("../../../../resources/spam-filter/spam-filter-rules.json.gz");
/// Upstream's default rules source, the value every install created before
/// the rules were bundled has saved. Read only to treat it as unset.
const LEGACY_DEFAULT_URL: &str =
"https://github.com/stalwartlabs/spam-filter/releases/latest/download/spam-filter-rules.json.gz";
/// The URL to fetch rules from, or `None` for the bundled rules. An empty
/// setting and upstream's old default both mean the bundled rules.
pub fn rules_url(configured: Option<String>) -> Option<String> {
configured.filter(|url| !url.trim().is_empty() && url != LEGACY_DEFAULT_URL)
}
/// The bundled rules, uncompressed: the same JSON the rules URL serves.
pub fn bundled_rules() -> Result<Vec<u8>, String> {
let mut json = Vec::new();
mail_auth::flate2::read::GzDecoder::new(BUNDLED_SPAM_RULES)
.read_to_end(&mut json)
.map_err(|err| format!("Failed to decompress the bundled spam rules: {err}"))?;
Ok(json)
}
fn applied_key() -> ValueClass {
ValueClass::Any(AnyClass {
subspace: SUBSPACE_INBUXA,
key: b"Sr".to_vec(),
})
}
/// The bundled version last loaded into the registry, if any.
pub async fn applied_version(data: &Store) -> trc::Result<Option<String>> {
data.get_value::<String>(ValueKey::from(applied_key()))
.await
.caused_by(trc::location!())
}
/// Records that the bundled rules of this version have been loaded.
pub async fn set_applied_version(data: &Store, version: &str) -> trc::Result<()> {
let mut batch = BatchBuilder::new();
batch.set(applied_key(), version.as_bytes().to_vec());
data.write(batch.build_all())
.await
.caused_by(trc::location!())
.map(|_| ())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn upstream_default_and_empty_mean_bundled() {
assert_eq!(rules_url(None), None);
assert_eq!(rules_url(Some(String::new())), None);
assert_eq!(rules_url(Some(" ".into())), None);
assert_eq!(rules_url(Some(LEGACY_DEFAULT_URL.into())), None);
assert_eq!(
rules_url(Some("file:///srv/rules.json.gz".into())).as_deref(),
Some("file:///srv/rules.json.gz")
);
}
#[test]
fn bundled_rules_parse_and_score_the_ai_tags() {
let rules: serde_json::Value = serde_json::from_slice(&bundled_rules().unwrap()).unwrap();
let tags = rules["SpamTag"].as_array().unwrap();
for (tag, score) in [("LLM_UNSOLICITED_HIGH", 3.0), ("LLM_LEGITIMATE_HIGH", -3.0)] {
let found = tags.iter().find(|t| t["tag"] == tag).unwrap();
assert_eq!(found["score"].as_f64(), Some(score), "{tag}");
}
assert!(!rules["SpamRule"].as_array().unwrap().is_empty());
}
}
+55 -12
View File
@@ -98,7 +98,38 @@ impl AcmeRequestBuilder {
reuse_key_pem: Option<String>, reuse_key_pem: Option<String>,
dns_parameters: Option<AcmeDnsParameters>, dns_parameters: Option<AcmeDnsParameters>,
) -> AcmeResult<PemCert> { ) -> AcmeResult<PemCert> {
let mut params = CertificateParams::new(domains.clone()).map_err(|err| { let mut published = BTreeSet::new();
let result = self
.run_order(
server,
&domains,
reuse_key_pem,
dns_parameters.as_ref(),
&mut published,
)
.await;
if let Some(dns_parameters) = &dns_parameters {
for (zone, challenge_name) in published {
let _ = dns_parameters
.updater
.delete_rrset(&zone, &challenge_name, dns_update::DnsRecordType::TXT)
.await;
}
}
result
}
async fn run_order(
&self,
server: &Server,
domains: &[String],
reuse_key_pem: Option<String>,
dns_parameters: Option<&AcmeDnsParameters>,
published: &mut BTreeSet<(String, String)>,
) -> AcmeResult<PemCert> {
let mut params = CertificateParams::new(domains.to_vec()).map_err(|err| {
AcmeError::Crypto(format!("Failed to create certificate params: {}", err)) AcmeError::Crypto(format!("Failed to create certificate params: {}", err))
})?; })?;
params.distinguished_name = DistinguishedName::new(); params.distinguished_name = DistinguishedName::new();
@@ -110,7 +141,7 @@ impl AcmeRequestBuilder {
AcmeError::Crypto(format!("Failed to generate key pair: {}", err)) AcmeError::Crypto(format!("Failed to generate key pair: {}", err))
})?, })?,
}; };
let response = self.new_order(domains.clone()).await?; let response = self.new_order(domains.to_vec()).await?;
let order_url = response.location; let order_url = response.location;
let mut order = response.body; let mut order = response.body;
let mut retry_after = None; let mut retry_after = None;
@@ -119,7 +150,7 @@ impl AcmeRequestBuilder {
Acme(AcmeEvent::OrderStart), Acme(AcmeEvent::OrderStart),
Url = self.directory.new_order.to_string(), Url = self.directory.new_order.to_string(),
Details = order_url.to_string(), Details = order_url.to_string(),
Hostname = domains.as_slice(), Hostname = domains,
Type = self.challenge.as_str(), Type = self.challenge.as_str(),
); );
@@ -128,19 +159,20 @@ impl AcmeRequestBuilder {
OrderStatus::Pending => { OrderStatus::Pending => {
if matches!(self.challenge, ChallengeType::Dns01) { if matches!(self.challenge, ChallengeType::Dns01) {
for url in &order.authorizations { for url in &order.authorizations {
self.authorize(server, url, dns_parameters.as_ref()).await?; self.authorize(server, url, dns_parameters, Some(published))
.await?;
} }
} else { } else {
let auth_futures = order let auth_futures = order
.authorizations .authorizations
.iter() .iter()
.map(|url| self.authorize(server, url, dns_parameters.as_ref())); .map(|url| self.authorize(server, url, dns_parameters, None));
try_join_all(auth_futures).await?; try_join_all(auth_futures).await?;
} }
trc::event!( trc::event!(
Acme(AcmeEvent::AuthCompleted), Acme(AcmeEvent::AuthCompleted),
Url = self.directory.new_order.to_string(), Url = self.directory.new_order.to_string(),
Hostname = domains.as_slice(), Hostname = domains,
); );
let response = self.order(&order_url).await?; let response = self.order(&order_url).await?;
order = response.body; order = response.body;
@@ -151,7 +183,7 @@ impl AcmeRequestBuilder {
trc::event!( trc::event!(
Acme(AcmeEvent::OrderProcessing), Acme(AcmeEvent::OrderProcessing),
Url = self.directory.new_order.to_string(), Url = self.directory.new_order.to_string(),
Hostname = domains.as_slice(), Hostname = domains,
Total = i, Total = i,
); );
@@ -179,7 +211,7 @@ impl AcmeRequestBuilder {
trc::event!( trc::event!(
Acme(AcmeEvent::OrderReady), Acme(AcmeEvent::OrderReady),
Url = self.directory.new_order.to_string(), Url = self.directory.new_order.to_string(),
Hostname = domains.as_slice(), Hostname = domains,
); );
let csr = params.serialize_request(&key_pair).map_err(|err| { let csr = params.serialize_request(&key_pair).map_err(|err| {
@@ -192,10 +224,10 @@ impl AcmeRequestBuilder {
trc::event!( trc::event!(
Acme(AcmeEvent::OrderValid), Acme(AcmeEvent::OrderValid),
Url = self.directory.new_order.to_string(), Url = self.directory.new_order.to_string(),
Hostname = domains.as_slice(), Hostname = domains,
); );
let certificate = self.select_certificate(&domains, certificate).await?; let certificate = self.select_certificate(domains, certificate).await?;
return Ok(PemCert { return Ok(PemCert {
certificate, certificate,
@@ -213,7 +245,7 @@ impl AcmeRequestBuilder {
Acme(AcmeEvent::OrderInvalid), Acme(AcmeEvent::OrderInvalid),
Url = self.directory.new_order.to_string(), Url = self.directory.new_order.to_string(),
Details = order_url.to_string(), Details = order_url.to_string(),
Hostname = domains.as_slice(), Hostname = domains,
Reason = reason.clone(), Reason = reason.clone(),
); );
@@ -228,6 +260,7 @@ impl AcmeRequestBuilder {
server: &Server, server: &Server,
url: &String, url: &String,
dns_parameters: Option<&AcmeDnsParameters>, dns_parameters: Option<&AcmeDnsParameters>,
published: Option<&mut BTreeSet<(String, String)>>,
) -> AcmeResult<()> { ) -> AcmeResult<()> {
let response = self let response = self
.auth(url) .auth(url)
@@ -289,7 +322,12 @@ impl AcmeRequestBuilder {
.await?; .await?;
} }
ChallengeType::Dns01 => { ChallengeType::Dns01 => {
let dns_parameters = dns_parameters.unwrap(); let Some(dns_parameters) = dns_parameters else {
return Err(AcmeError::Invalid(
"DNS-01 challenge requested but a DNS provider was not configured"
.to_string(),
));
};
let domain = domain.strip_prefix("*.").unwrap_or(&domain); let domain = domain.strip_prefix("*.").unwrap_or(&domain);
let zone = dns_parameters let zone = dns_parameters
@@ -310,6 +348,11 @@ impl AcmeRequestBuilder {
) )
.await .await
.map_err(AcmeError::Dns)?; .map_err(AcmeError::Dns)?;
if let Some(published) = published {
published.insert((zone.to_string(), challenge_name.clone()));
}
dns_parameters dns_parameters
.updater .updater
.wait_for_txt_propagation(&challenge_name, zone, &proof) .wait_for_txt_propagation(&challenge_name, zone, &proof)
+30
View File
@@ -1150,6 +1150,36 @@ impl DnsUpdater {
Ok(()) Ok(())
} }
pub async fn delete_rrset(
&self,
origin: &str,
name: &str,
record_type: DnsRecordType,
) -> Result<(), String> {
if let Err(err) = self
.updater
.set_rrset(
name,
record_type,
self.ttl.as_secs() as u32,
Vec::new(),
origin,
)
.await
{
trc::event!(
Dns(DnsEvent::RecordDeletionFailed),
Hostname = name.to_string(),
Details = origin.to_string(),
Type = record_type.as_str(),
Reason = err.to_string(),
);
return Err(format!("Failed to delete DNS RRSet: {}", err));
}
Ok(())
}
pub async fn add_to_rrset( pub async fn add_to_rrset(
&self, &self,
origin: &str, origin: &str,
+15 -15
View File
@@ -299,28 +299,28 @@ impl LegacyProtocol {
pub fn refusal(&self, scope: RefusalScope) -> &'static str { pub fn refusal(&self, scope: RefusalScope) -> &'static str {
match (scope, self) { match (scope, self) {
(RefusalScope::Server, LegacyProtocol::Imap) => { (RefusalScope::Server, LegacyProtocol::Imap) => {
"This server allows only INBUXA webmail and JMAP apps. This mail app can't sign in." "This server allows only inbuxa webmail and JMAP apps. This mail app can't sign in."
} }
(RefusalScope::Server, LegacyProtocol::Pop3) => { (RefusalScope::Server, LegacyProtocol::Pop3) => {
"[AUTH] This server allows only INBUXA webmail and JMAP apps. This mail app can't sign in." "[AUTH] This server allows only inbuxa webmail and JMAP apps. This mail app can't sign in."
} }
(RefusalScope::Server, LegacyProtocol::ManageSieve) => { (RefusalScope::Server, LegacyProtocol::ManageSieve) => {
"This server allows only INBUXA webmail and JMAP apps." "This server allows only inbuxa webmail and JMAP apps."
} }
(RefusalScope::Server, LegacyProtocol::Submission) => { (RefusalScope::Server, LegacyProtocol::Submission) => {
"535 5.7.0 This server allows only INBUXA webmail and JMAP apps. This mail app can't send.\r\n" "535 5.7.0 This server allows only inbuxa webmail and JMAP apps. This mail app can't send.\r\n"
} }
(RefusalScope::Tenant(_), LegacyProtocol::Imap) => { (RefusalScope::Tenant(_), LegacyProtocol::Imap) => {
"Your organization allows only INBUXA webmail and JMAP apps. This mail app can't sign in." "Your organization allows only inbuxa webmail and JMAP apps. This mail app can't sign in."
} }
(RefusalScope::Tenant(_), LegacyProtocol::Pop3) => { (RefusalScope::Tenant(_), LegacyProtocol::Pop3) => {
"[AUTH] Your organization allows only INBUXA webmail and JMAP apps. This mail app can't sign in." "[AUTH] Your organization allows only inbuxa webmail and JMAP apps. This mail app can't sign in."
} }
(RefusalScope::Tenant(_), LegacyProtocol::ManageSieve) => { (RefusalScope::Tenant(_), LegacyProtocol::ManageSieve) => {
"Your organization allows only INBUXA webmail and JMAP apps." "Your organization allows only inbuxa webmail and JMAP apps."
} }
(RefusalScope::Tenant(_), LegacyProtocol::Submission) => { (RefusalScope::Tenant(_), LegacyProtocol::Submission) => {
"535 5.7.0 Your organization allows only INBUXA webmail and JMAP apps. This mail app can't send.\r\n" "535 5.7.0 Your organization allows only inbuxa webmail and JMAP apps. This mail app can't send.\r\n"
} }
} }
} }
@@ -541,7 +541,7 @@ mod tests {
let server = RefusalScope::Server; let server = RefusalScope::Server;
assert_eq!( assert_eq!(
LegacyProtocol::Imap.refusal(server), LegacyProtocol::Imap.refusal(server),
"This server allows only INBUXA webmail and JMAP apps. This mail app can't sign in." "This server allows only inbuxa webmail and JMAP apps. This mail app can't sign in."
); );
assert!( assert!(
LegacyProtocol::Pop3 LegacyProtocol::Pop3
@@ -550,11 +550,11 @@ mod tests {
); );
assert_eq!( assert_eq!(
LegacyProtocol::ManageSieve.refusal(server), LegacyProtocol::ManageSieve.refusal(server),
"This server allows only INBUXA webmail and JMAP apps." "This server allows only inbuxa webmail and JMAP apps."
); );
assert_eq!( assert_eq!(
LegacyProtocol::Submission.refusal(server), LegacyProtocol::Submission.refusal(server),
"535 5.7.0 This server allows only INBUXA webmail and JMAP apps. This mail app can't send.\r\n" "535 5.7.0 This server allows only inbuxa webmail and JMAP apps. This mail app can't send.\r\n"
); );
} }
@@ -564,19 +564,19 @@ mod tests {
let tenant = RefusalScope::Tenant(7); let tenant = RefusalScope::Tenant(7);
assert_eq!( assert_eq!(
LegacyProtocol::Imap.refusal(tenant), LegacyProtocol::Imap.refusal(tenant),
"Your organization allows only INBUXA webmail and JMAP apps. This mail app can't sign in." "Your organization allows only inbuxa webmail and JMAP apps. This mail app can't sign in."
); );
assert_eq!( assert_eq!(
LegacyProtocol::Pop3.refusal(tenant), LegacyProtocol::Pop3.refusal(tenant),
"[AUTH] Your organization allows only INBUXA webmail and JMAP apps. This mail app can't sign in." "[AUTH] Your organization allows only inbuxa webmail and JMAP apps. This mail app can't sign in."
); );
assert_eq!( assert_eq!(
LegacyProtocol::ManageSieve.refusal(tenant), LegacyProtocol::ManageSieve.refusal(tenant),
"Your organization allows only INBUXA webmail and JMAP apps." "Your organization allows only inbuxa webmail and JMAP apps."
); );
assert_eq!( assert_eq!(
LegacyProtocol::Submission.refusal(tenant), LegacyProtocol::Submission.refusal(tenant),
"535 5.7.0 Your organization allows only INBUXA webmail and JMAP apps. This mail app can't send.\r\n" "535 5.7.0 Your organization allows only inbuxa webmail and JMAP apps. This mail app can't send.\r\n"
); );
let err = LegacyProtocol::Imap.refused(tenant, Some("example.org".into())); let err = LegacyProtocol::Imap.refused(tenant, Some("example.org".into()));
assert_eq!(err.value_as_str(trc::Key::Policy), Some("tenant")); assert_eq!(err.value_as_str(trc::Key::Policy), Some("tenant"));
+2
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
use super::{ use super::{
+2
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
use self::limiter::{ConcurrencyLimiter, InFlight}; use self::limiter::{ConcurrencyLimiter, InFlight};
+56 -1
View File
@@ -23,6 +23,7 @@ use crate::{
manager::SPAM_CLASSIFIER_KEY, manager::SPAM_CLASSIFIER_KEY,
network::RcptResolution, network::RcptResolution,
}; };
use ahash::AHashSet;
use directory::Recipient; use directory::Recipient;
use mail_auth::IpLookupStrategy; use mail_auth::IpLookupStrategy;
use registry::schema::enums::ExpressionVariable; use registry::schema::enums::ExpressionVariable;
@@ -37,6 +38,7 @@ use store::{
write::{AlignedBytes, Archive, QueueClass, ValueClass}, write::{AlignedBytes, Archive, QueueClass, ValueClass},
}; };
use trc::{AddContext, SpamEvent}; use trc::{AddContext, SpamEvent};
use utils::DomainPart;
impl Server { impl Server {
pub async fn rcpt_resolve( pub async fn rcpt_resolve(
@@ -163,7 +165,10 @@ impl Server {
} }
EmailCache::MailingList(id) => { EmailCache::MailingList(id) => {
if let Some(list) = self.try_list(id).await? { if let Some(list) = self.try_list(id).await? {
return Ok(RcptResolution::Expand(list.recipients.clone())); return Ok(RcptResolution::Expand(
self.expand_nested_lists(id, list.recipients.clone())
.await?,
));
} else { } else {
self.inner self.inner
.cache .cache
@@ -195,6 +200,56 @@ impl Server {
} }
} }
async fn expand_nested_lists(
&self,
list_id: u32,
recipients: Arc<[Box<str>]>,
) -> trc::Result<Arc<[Box<str>]>> {
let mut has_nested = false;
for member in recipients.iter() {
if let Some(EmailCache::MailingList(_)) = self.rcpt_id_from_email(member).await? {
has_nested = true;
break;
}
}
if !has_nested {
return Ok(recipients);
}
let mut expanded = Vec::with_capacity(recipients.len());
let mut seen: AHashSet<Box<str>> = AHashSet::with_capacity(recipients.len());
let mut visited = AHashSet::from_iter([list_id]);
let mut pending: Vec<Arc<[Box<str>]>> = Vec::new();
let mut members = recipients;
loop {
for member in members.iter() {
if let Some(EmailCache::MailingList(nested_id)) =
self.rcpt_id_from_email(member).await?
{
if !visited.insert(nested_id) {
continue;
}
if let Some(nested) = self.try_list(nested_id).await? {
pending.push(nested.recipients.clone());
continue;
}
}
if seen.insert(member.to_canonical_address().into()) {
expanded.push(member.clone());
}
}
let Some(next) = pending.pop() else {
break;
};
members = next;
}
Ok(expanded.into())
}
pub async fn get_dkim_signers( pub async fn get_dkim_signers(
&self, &self,
domain: &str, domain: &str,
+10 -8
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD}; use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};
@@ -313,16 +315,16 @@ B4yDfR2rGOd2H6Kv3fQNHPj9Nu5Tks8QYMLzrX8ONCNoFnNUQl9S0r0QS6phVqD0
#[test] #[test]
fn contact_is_normalized_to_a_uri() { fn contact_is_normalized_to_a_uri() {
for (input, expected) in [ for (input, expected) in [
("hello@stalw.art", Some("mailto:hello@stalw.art")), ("hello@example.org", Some("mailto:hello@example.org")),
(" hello@stalw.art ", Some("mailto:hello@stalw.art")), (" hello@example.org ", Some("mailto:hello@example.org")),
("mailto:hello@stalw.art", Some("mailto:hello@stalw.art")), ("mailto:hello@example.org", Some("mailto:hello@example.org")),
("MAILTO:hello@stalw.art", Some("MAILTO:hello@stalw.art")), ("MAILTO:hello@example.org", Some("MAILTO:hello@example.org")),
( (
"https://stalw.art/contact", "https://example.org/contact",
Some("https://stalw.art/contact"), Some("https://example.org/contact"),
), ),
("stalw.art", None), ("example.org", None),
("http://stalw.art", None), ("http://example.org", None),
("tel:+123456789", None), ("tel:+123456789", None),
("", None), ("", None),
] { ] {
+2 -2
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "coordinator" name = "coordinator"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
@@ -8,7 +8,7 @@ store = { path = "../store" }
registry = { path = "../registry" } registry = { path = "../registry" }
trc = { path = "../trc" } trc = { path = "../trc" }
futures = { version = "0.3", optional = true } futures = { version = "0.3", optional = true }
tokio = { version = "1.53", features = ["sync", "fs", "io-util"] } tokio = { version = "1.53", features = ["sync", "fs", "io-util", "rt", "time"] }
async-nats = { version = "0.50", default-features = false, features = ["server_2_10", "server_2_11", "aws-lc-rs"], optional = true } async-nats = { version = "0.50", default-features = false, features = ["server_2_10", "server_2_11", "aws-lc-rs"], optional = true }
zenoh = { version = "1.10.0", default-features = false, features = ["auth_pubkey", "transport_multilink", "transport_compression", "transport_quic", "transport_tcp", "transport_tls", "transport_udp"], optional = true } zenoh = { version = "1.10.0", default-features = false, features = ["auth_pubkey", "transport_multilink", "transport_compression", "transport_quic", "transport_tcp", "transport_tls", "transport_udp"], optional = true }
rdkafka = { version = "0.39", features = ["cmake-build"], optional = true } rdkafka = { version = "0.39", features = ["cmake-build"], optional = true }
+118 -2
View File
@@ -2,13 +2,22 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
use std::sync::Arc; use std::{
sync::{
Arc,
atomic::{AtomicBool, Ordering},
},
time::Duration,
};
use crate::Coordinator; use crate::Coordinator;
use async_nats::Client; use async_nats::Client;
use registry::schema::structs::NatsCoordinator; use registry::schema::structs::NatsCoordinator;
use trc::ClusterEvent;
pub mod pubsub; pub mod pubsub;
@@ -47,9 +56,116 @@ impl NatsPubSub {
opts = opts.token(credentials); opts = opts.token(credentials);
} }
// inbuxa: connect in the background and keep trying, so a node that
// starts while NATS is down still joins the cluster once NATS is
// back, instead of running without a coordinator until restarted;
// and report the connection going and coming back
let reporter = Arc::new(Reporter::default());
opts = opts.retry_on_initial_connect().event_callback({
let reporter = reporter.clone();
move |event| {
let reporter = reporter.clone();
async move { reporter.report(event) }
}
});
let connection_timeout = config.timeout_connection.into_inner();
async_nats::connect_with_options(config.addresses.into_inner(), opts) async_nats::connect_with_options(config.addresses.into_inner(), opts)
.await .await
.map(|client| Coordinator::Nats(Arc::new(NatsPubSub { client }))) .map(|client| {
reporter.watch_first_connection(client.clone(), connection_timeout);
Coordinator::Nats(Arc::new(NatsPubSub { client }))
})
.map_err(|err| format!("Failed to connect to Nats: {}", err)) .map_err(|err| format!("Failed to connect to Nats: {}", err))
} }
/// inbuxa: whether the client is connected to a NATS server right now.
pub fn is_connected(&self) -> bool {
matches!(
self.client.connection_state(),
async_nats::connection::State::Connected
)
}
}
/// inbuxa: reports the client's connection events as the server's own.
#[derive(Default)]
struct Reporter {
connected_once: AtomicBool,
// A failed attempt raises an error each time the client retries, every
// few seconds while NATS is down: report the first after each change
error_reported: AtomicBool,
}
impl Reporter {
fn report(&self, event: async_nats::Event) {
match event {
async_nats::Event::Connected => {
self.connected_once.store(true, Ordering::Relaxed);
self.error_reported.store(false, Ordering::Relaxed);
trc::event!(Cluster(ClusterEvent::CoordinatorConnected), Type = "nats");
}
async_nats::Event::Disconnected => {
self.error_reported.store(false, Ordering::Relaxed);
trc::event!(
Cluster(ClusterEvent::CoordinatorDisconnected),
Type = "nats",
Details = "Connection lost; reconnecting in the background",
);
}
async_nats::Event::Closed => {
trc::event!(
Cluster(ClusterEvent::CoordinatorDisconnected),
Type = "nats",
Details = "Connection closed; no further attempts will be made",
);
}
async_nats::Event::ClientError(async_nats::ClientError::MaxReconnects) => {
trc::event!(
Cluster(ClusterEvent::CoordinatorDisconnected),
Type = "nats",
Details = "Gave up reconnecting (maxReconnects reached)",
);
}
async_nats::Event::ClientError(err) => {
if !self.error_reported.swap(true, Ordering::Relaxed) {
trc::event!(
Cluster(ClusterEvent::CoordinatorError),
Type = "nats",
Details = "Connection attempt failed; retrying",
Reason = err.to_string(),
);
}
}
event => {
trc::event!(
Cluster(ClusterEvent::CoordinatorError),
Type = "nats",
Details = event.to_string(),
);
}
}
}
/// The first connection is made in the background, so say so when it
/// hasn't been made within the connection timeout. The client keeps
/// trying, and reports the connection when it comes.
fn watch_first_connection(self: &Arc<Self>, client: Client, timeout: Duration) {
let reporter = self.clone();
tokio::spawn(async move {
tokio::time::sleep(timeout).await;
if !reporter.connected_once.load(Ordering::Relaxed)
&& !matches!(
client.connection_state(),
async_nats::connection::State::Connected
)
{
trc::event!(
Cluster(ClusterEvent::CoordinatorDisconnected),
Type = "nats",
Details = "Not connected at startup; retrying in the background",
);
}
});
}
} }
+13
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
use crate::{Coordinator, Msg, PubSubStream}; use crate::{Coordinator, Msg, PubSubStream};
@@ -43,6 +45,17 @@ impl Coordinator {
pub fn is_none(&self) -> bool { pub fn is_none(&self) -> bool {
matches!(self, Coordinator::None) matches!(self, Coordinator::None)
} }
/// inbuxa: whether the coordinator is connected right now, for the
/// backends that track it (NATS); `None` for the others and when no
/// coordinator is configured.
pub fn is_connected(&self) -> Option<bool> {
match self {
#[cfg(feature = "nats")]
Coordinator::Nats(store) => Some(store.is_connected()),
_ => None,
}
}
} }
impl PubSubStream { impl PubSubStream {
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "dav-proto" name = "dav-proto"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "dav" name = "dav"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+3 -1
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
use super::ETag; use super::ETag;
@@ -490,7 +492,7 @@ impl LockRequestHandler for Server {
for cond in &if_.list { for cond in &if_.list {
match cond { match cond {
Condition::StateToken { token, .. } => { Condition::StateToken { token, .. } => {
if token.starts_with("urn:stalwart:davsync:") { if token.starts_with("urn:inbuxa:davsync:") {
needs_sync_token = true; needs_sync_token = true;
} else { } else {
needs_lock_token = true; needs_lock_token = true;
+7 -5
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
use crate::{DavError, DavResourceName}; use crate::{DavError, DavResourceName};
@@ -181,12 +183,12 @@ impl OwnedUri<'_> {
impl Urn { impl Urn {
pub fn try_extract_sync_id(token: &str) -> Option<&str> { pub fn try_extract_sync_id(token: &str) -> Option<&str> {
token token
.strip_prefix("urn:stalwart:davsync:") .strip_prefix("urn:inbuxa:davsync:")
.map(|x| x.split_once(':').map(|(x, _)| x).unwrap_or(x)) .map(|x| x.split_once(':').map(|(x, _)| x).unwrap_or(x))
} }
pub fn parse(input: &str) -> Option<Self> { pub fn parse(input: &str) -> Option<Self> {
let inbox = input.strip_prefix("urn:stalwart:")?; let inbox = input.strip_prefix("urn:inbuxa:")?;
let (kind, id) = inbox.split_once(':')?; let (kind, id) = inbox.split_once(':')?;
match kind { match kind {
"davlock" => u64::from_str_radix(id, 16).ok().map(Urn::Lock), "davlock" => u64::from_str_radix(id, 16).ok().map(Urn::Lock),
@@ -223,12 +225,12 @@ impl Urn {
impl Display for Urn { impl Display for Urn {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self { match self {
Urn::Lock(id) => write!(f, "urn:stalwart:davlock:{id:x}",), Urn::Lock(id) => write!(f, "urn:inbuxa:davlock:{id:x}",),
Urn::Sync { id, seq } => { Urn::Sync { id, seq } => {
if *seq == 0 { if *seq == 0 {
write!(f, "urn:stalwart:davsync:{id:x}") write!(f, "urn:inbuxa:davsync:{id:x}")
} else { } else {
write!(f, "urn:stalwart:davsync:{id:x}:{seq:x}") write!(f, "urn:inbuxa:davsync:{id:x}:{seq:x}")
} }
} }
} }
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "directory" name = "directory"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+1 -1
View File
@@ -40,7 +40,7 @@ impl OpenIdDirectory {
pub async fn new(config: OidcConfig) -> Result<Self, OidcError> { pub async fn new(config: OidcConfig) -> Result<Self, OidcError> {
let http = utils::http::http_client_builder(false) let http = utils::http::http_client_builder(false)
.user_agent("INBUXA/1.0") // types::brand!(); this crate does not depend on types .user_agent("inbuxa/1.0") // types::brand!(); this crate does not depend on types
.timeout(Duration::from_secs(30)) .timeout(Duration::from_secs(30))
.build() .build()
.map_err(|e| OidcError::Network(format!("HTTP client build failed: {e}")))?; .map_err(|e| OidcError::Network(format!("HTTP client build failed: {e}")))?;
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "email" name = "email"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+8
View File
@@ -22,6 +22,8 @@ use std::{borrow::Cow, future::Future};
use store::ahash::AHashMap; use store::ahash::AHashMap;
use types::blob_hash::BlobHash; use types::blob_hash::BlobHash;
pub const ORCPT_ADDR_TYPE: &str = "rfc822;";
#[derive(Debug)] #[derive(Debug)]
pub struct IngestMessage { pub struct IngestMessage {
pub sender_address: String, pub sender_address: String,
@@ -40,6 +42,12 @@ pub struct IngestRecipient {
} }
impl IngestRecipient { impl IngestRecipient {
pub fn orcpt_parameter(&self) -> Option<String> {
self.orcpt
.as_deref()
.map(|orcpt| format!("{ORCPT_ADDR_TYPE}{orcpt}"))
}
pub fn is_spam(&self) -> bool { pub fn is_spam(&self) -> bool {
self.spam_percentage self.spam_percentage
.is_some_and(|percentage| percentage >= 50) .is_some_and(|percentage| percentage >= 50)
+2 -1
View File
@@ -126,6 +126,7 @@ impl SieveScriptIngest for Server {
.caused_by(trc::location!())?; .caused_by(trc::location!())?;
// Create Sieve instance // Create Sieve instance
let orcpt = envelope_to.orcpt_parameter();
let mut instance = self.core.sieve.untrusted_runtime.filter_parsed(message); let mut instance = self.core.sieve.untrusted_runtime.filter_parsed(message);
// Set account name and email // Set account name and email
@@ -141,7 +142,7 @@ impl SieveScriptIngest for Server {
// Set envelope // Set envelope
instance.set_envelope(Envelope::From, envelope_from); instance.set_envelope(Envelope::From, envelope_from);
instance.set_envelope(Envelope::To, envelope_to.address.as_str()); instance.set_envelope(Envelope::To, envelope_to.address.as_str());
if let Some(orcpt) = &envelope_to.orcpt { if let Some(orcpt) = &orcpt {
instance.set_envelope(Envelope::Orcpt, orcpt.as_str()); instance.set_envelope(Envelope::Orcpt, orcpt.as_str());
} }
instance.set_spam_status(spam_status(envelope_to.spam_percentage)); instance.set_spam_status(spam_status(envelope_to.spam_percentage));
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "inbuxa-features" name = "inbuxa-features"
description = "INBUXA's rebuilt features: behavior Stalwart ships only in its Enterprise Edition, rebuilt clean-room" description = "inbuxa's rebuilt features: behavior Stalwart ships only in its Enterprise Edition, rebuilt clean-room"
license = "AGPL-3.0-only" license = "AGPL-3.0-only"
version = "0.16.22" version = "0.16.22"
edition = "2024" edition = "2024"
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "groupware" name = "groupware"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "http_proto" name = "http_proto"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+2
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
use common::manager::application::Resource; use common::manager::application::Resource;
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "http" name = "http"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+6 -10
View File
@@ -230,14 +230,7 @@ async fn delivery_diagnose(
// Lookup MX // Lookup MX
let now = Instant::now(); let now = Instant::now();
let mxs = match server let mxs = match server.mx_lookup(domain.as_str()).await {
.core
.smtp
.resolvers
.dns
.mx_lookup(&domain, Some(&server.inner.cache.dns_mx))
.await
{
Ok(mxs) => mxs, Ok(mxs) => mxs,
Err(err) => { Err(err) => {
tx.send(DeliveryStage::MxLookupError { tx.send(DeliveryStage::MxLookupError {
@@ -419,7 +412,7 @@ async fn delivery_diagnose(
}) })
.await?; .await?;
None continue 'outer;
} }
Ok(TlsaResult::Missing) => { Ok(TlsaResult::Missing) => {
tx.send(DeliveryStage::TlsaNotFound { tx.send(DeliveryStage::TlsaNotFound {
@@ -440,14 +433,17 @@ async fn delivery_diagnose(
reason: "No TLSA records found for MX".to_string(), reason: "No TLSA records found for MX".to_string(),
}) })
.await?; .await?;
None
} else { } else {
tx.send(DeliveryStage::TlsaLookupError { tx.send(DeliveryStage::TlsaLookupError {
elapsed: now.elapsed_ms(), elapsed: now.elapsed_ms(),
reason: err.to_string(), reason: err.to_string(),
}) })
.await?; .await?;
continue 'outer;
} }
None
} }
}; };
+21
View File
@@ -562,6 +562,27 @@ impl ParseHttp for Server {
}) })
.into_http_response()); .into_http_response());
} }
// inbuxa: the cluster coordinator's connection, for
// monitoring. It stays out of live and ready on purpose:
// a node without its coordinator still serves mail, and
// failing those would have an orchestrator restart, or
// take out of service, every node at once when the
// coordinator goes down
"cluster" => {
let coordinator = &self.core.storage.coordinator;
let (status, state) = match coordinator.is_connected() {
Some(true) => (StatusCode::OK, "connected"),
Some(false) => (StatusCode::SERVICE_UNAVAILABLE, "disconnected"),
None if coordinator.is_none() => (StatusCode::OK, "none"),
None => (StatusCode::OK, "unknown"),
};
return Ok(http_proto::JsonResponse::with_status(
status,
serde_json::json!({ "coordinator": state }),
)
.no_cache()
.into_http_response());
}
_ => (), _ => (),
} }
} }
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "imap_proto" name = "imap_proto"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "imap" name = "imap"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+32
View File
@@ -390,6 +390,16 @@ impl<T: SessionStream> SessionData<T> {
.await .await
.imap_ctx(&arguments.tag, trc::location!())?; .imap_ctx(&arguments.tag, trc::location!())?;
let mut dest_cache = None; let mut dest_cache = None;
let train_spam = if dest_mailbox_id == JUNK_ID {
Some(true)
} else if src_mailbox.id.mailbox_id == JUNK_ID && dest_mailbox_id != TRASH_ID {
Some(false)
} else {
None
};
let mut train_batch = BatchBuilder::new();
let mut did_train = false;
train_batch.with_account_id(src_account_id);
for (id, imap_id) in ids { for (id, imap_id) in ids {
match self match self
.server .server
@@ -515,11 +525,33 @@ impl<T: SessionStream> SessionData<T> {
} }
}; };
if let Some(is_spam) = train_spam {
self.server
.add_account_spam_sample(
&mut train_batch,
src_account_id,
id,
is_spam,
self.session_id,
)
.await
.imap_ctx(&arguments.tag, trc::location!())?;
train_batch.commit_point();
did_train = true;
}
if is_move { if is_move {
destroy_ids.insert(id); destroy_ids.insert(id);
} }
} }
if did_train {
self.server
.commit_batch(train_batch)
.await
.imap_ctx(&arguments.tag, trc::location!())?;
}
// Untag or delete emails // Untag or delete emails
if !destroy_ids.is_empty() { if !destroy_ids.is_empty() {
let mut batch = BatchBuilder::new(); let mut batch = BatchBuilder::new();
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "jmap_proto" name = "jmap_proto"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+20
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
use super::ahash_is_empty; use super::ahash_is_empty;
@@ -71,6 +73,23 @@ pub struct SetResponse<T: JmapObject> {
#[serde(rename = "notDestroyed")] #[serde(rename = "notDestroyed")]
#[serde(skip_serializing_if = "VecMap::is_empty")] #[serde(skip_serializing_if = "VecMap::is_empty")]
pub not_destroyed: VecMap<MaybeInvalid<Id>, SetError<T::Property>>, pub not_destroyed: VecMap<MaybeInvalid<Id>, SetError<T::Property>>,
// inbuxa: on a registry write that changes the running settings, whether
// the server applied it
#[serde(rename = "x:settingsReload")]
#[serde(skip_serializing_if = "Option::is_none")]
pub settings_reload: Option<SettingsReload>,
}
/// inbuxa: the settings reload that followed a registry write.
#[derive(Debug, Clone, serde::Serialize)]
pub struct SettingsReload {
/// The running settings (here and, through the cluster, on every node)
/// include the write.
pub applied: bool,
/// Why they don't, when they don't.
#[serde(skip_serializing_if = "Option::is_none")]
pub description: Option<String>,
} }
impl<'de, T: JmapObject> DeserializeArguments<'de> for SetRequest<'de, T> { impl<'de, T: JmapObject> DeserializeArguments<'de> for SetRequest<'de, T> {
@@ -199,6 +218,7 @@ impl<T: JmapObject> SetResponse<T> {
not_created: VecMap::new(), not_created: VecMap::new(),
not_updated: VecMap::new(), not_updated: VecMap::new(),
not_destroyed: VecMap::new(), not_destroyed: VecMap::new(),
settings_reload: None,
}) })
} else { } else {
Err(trc::JmapEvent::RequestTooLarge.into_err()) Err(trc::JmapEvent::RequestTooLarge.into_err())
+3 -3
View File
@@ -91,7 +91,7 @@ pub enum Capability {
FileNode = 1 << 15, FileNode = 1 << 15,
#[serde(rename(serialize = "urn:ietf:params:jmap:mail:share"))] #[serde(rename(serialize = "urn:ietf:params:jmap:mail:share"))]
MailShare = 1 << 16, MailShare = 1 << 16,
#[serde(rename(serialize = "urn:stalwart:jmap"))] #[serde(rename(serialize = "urn:inbuxa:jmap:registry"))]
Stalwart = 1 << 17, Stalwart = 1 << 17,
#[serde(rename(serialize = "urn:ietf:params:jmap:webpush-vapid"))] #[serde(rename(serialize = "urn:ietf:params:jmap:webpush-vapid"))]
WebPushVapid = 1 << 18, WebPushVapid = 1 << 18,
@@ -353,7 +353,7 @@ impl Capability {
Capability::PrincipalsAvailability => "urn:ietf:params:jmap:principals:availability", Capability::PrincipalsAvailability => "urn:ietf:params:jmap:principals:availability",
Capability::FileNode => "urn:ietf:params:jmap:filenode", Capability::FileNode => "urn:ietf:params:jmap:filenode",
Capability::MailShare => "urn:ietf:params:jmap:mail:share", Capability::MailShare => "urn:ietf:params:jmap:mail:share",
Capability::Stalwart => "urn:stalwart:jmap", Capability::Stalwart => "urn:inbuxa:jmap:registry",
Capability::WebPushVapid => "urn:ietf:params:jmap:webpush-vapid", Capability::WebPushVapid => "urn:ietf:params:jmap:webpush-vapid",
Capability::EmailPush => "urn:ietf:params:jmap:emailpush", Capability::EmailPush => "urn:ietf:params:jmap:emailpush",
Capability::Inbuxa => "urn:inbuxa:jmap", Capability::Inbuxa => "urn:inbuxa:jmap",
@@ -501,7 +501,7 @@ impl Capability {
"urn:ietf:params:jmap:contacts:parse" => Capability::ContactsParse, "urn:ietf:params:jmap:contacts:parse" => Capability::ContactsParse,
"urn:ietf:params:jmap:calendars:parse" => Capability::CalendarsParse, "urn:ietf:params:jmap:calendars:parse" => Capability::CalendarsParse,
"urn:ietf:params:jmap:mail:share" => Capability::MailShare, "urn:ietf:params:jmap:mail:share" => Capability::MailShare,
"urn:stalwart:jmap" => Capability::Stalwart, "urn:inbuxa:jmap:registry" => Capability::Stalwart,
"urn:ietf:params:jmap:webpush-vapid" => Capability::WebPushVapid, "urn:ietf:params:jmap:webpush-vapid" => Capability::WebPushVapid,
"urn:ietf:params:jmap:emailpush" => Capability::EmailPush, "urn:ietf:params:jmap:emailpush" => Capability::EmailPush,
"urn:inbuxa:jmap" => Capability::Inbuxa, "urn:inbuxa:jmap" => Capability::Inbuxa,
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "jmap" name = "jmap"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+30 -1
View File
@@ -11,7 +11,11 @@ use crate::{
use common::{Server, auth::AccessToken}; use common::{Server, auth::AccessToken};
use email::{ use email::{
cache::{MessageCacheFetch, email::MessageCacheAccess, mailbox::MailboxCacheAccess}, cache::{MessageCacheFetch, email::MessageCacheAccess, mailbox::MailboxCacheAccess},
message::copy::{CopyMessageError, EmailCopy}, mailbox::JUNK_ID,
message::{
copy::{CopyMessageError, EmailCopy},
ingest::EmailIngest,
},
}; };
use http_proto::HttpSessionData; use http_proto::HttpSessionData;
use jmap_proto::{ use jmap_proto::{
@@ -29,6 +33,7 @@ use jmap_proto::{
}; };
use jmap_tools::{Key, Value}; use jmap_tools::{Key, Value};
use std::future::Future; use std::future::Future;
use store::write::BatchBuilder;
use trc::AddContext; use trc::AddContext;
use types::acl::Acl; use types::acl::Acl;
use utils::map::vec_map::VecMap; use utils::map::vec_map::VecMap;
@@ -87,6 +92,9 @@ impl JmapEmailCopy for Server {
}; };
let on_success_delete = request.on_success_destroy_original.unwrap_or(false); let on_success_delete = request.on_success_destroy_original.unwrap_or(false);
let mut destroy_ids = Vec::new(); let mut destroy_ids = Vec::new();
let mut train_batch = BatchBuilder::new();
let mut did_train = false;
train_batch.with_account_id(from_account_id);
'create: for (id, create) in request.create.into_valid() { 'create: for (id, create) in request.create.into_valid() {
let mut from_message_id = None; let mut from_message_id = None;
@@ -208,6 +216,7 @@ impl JmapEmailCopy for Server {
} }
// Add response // Add response
let train_spam = mailboxes.contains(&JUNK_ID);
match self match self
.copy_message( .copy_message(
from_account_id, from_account_id,
@@ -221,6 +230,20 @@ impl JmapEmailCopy for Server {
.await? .await?
{ {
Ok(email) => { Ok(email) => {
if train_spam {
self.add_account_spam_sample(
&mut train_batch,
from_account_id,
from_message_id.document_id(),
true,
session.session_id,
)
.await
.caused_by(trc::location!())?;
train_batch.commit_point();
did_train = true;
}
response response
.created .created
.append(id, ingested_into_object(email).into()); .append(id, ingested_into_object(email).into());
@@ -245,6 +268,12 @@ impl JmapEmailCopy for Server {
} }
} }
if did_train {
self.commit_batch(train_batch)
.await
.caused_by(trc::location!())?;
}
// Update state // Update state
if !response.created.is_empty() { if !response.created.is_empty() {
response.new_state = self.get_cached_messages(account_id).await?.get_state(false); response.new_state = self.get_cached_messages(account_id).await?.get_state(false);
+17 -2
View File
@@ -427,9 +427,24 @@ pub(crate) async fn trace_query(
} }
None => false, None => false,
}, },
Property::QueueId => match value.as_str() { // The queue id column is an integer on every search backend, and
// holds a trace's first queue id; the keywords carry all of them
Property::QueueId => match value
.as_str()
.and_then(|v| v.trim().parse::<u64>().ok())
.or_else(|| value.as_u64())
{
Some(queue_id) => { Some(queue_id) => {
search.push(SearchFilter::eq(TracingSearchField::QueueId, queue_id.to_string())); search.extend([
SearchFilter::Or,
SearchFilter::eq(TracingSearchField::QueueId, queue_id),
SearchFilter::has_text(
TracingSearchField::Keywords,
queue_id.to_string(),
nlp::language::Language::None,
),
SearchFilter::End,
]);
true true
} }
None => false, None => false,
+14 -1
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
use crate::registry::mapping::{RegistrySetResponse, map_bootstrap_error}; use crate::registry::mapping::{RegistrySetResponse, map_bootstrap_error};
@@ -99,7 +101,7 @@ pub(crate) async fn action_set(
} else { } else {
set.response set.response
.not_created .not_created
.append(id, map_bootstrap_error(result.errors)); .append(id, reload_refused(result.errors));
} }
} }
Action::InvalidateCaches => { Action::InvalidateCaches => {
@@ -573,3 +575,14 @@ async fn dmarc_troubleshoot(
Some(request) Some(request)
} }
/// inbuxa: a refused reload names the object that stopped it and says the
/// settings weren't applied; upstream passed on the first error's bare message
/// ("Invalid address: ..."), which read like a problem with the request.
fn reload_refused(errors: Vec<registry::types::error::Error>) -> SetError<Property> {
let description = format!(
"Settings were not reloaded. {}",
common::cache::reload::describe_reload_errors(&errors)
);
map_bootstrap_error(errors).with_description(description)
}
@@ -208,7 +208,7 @@ pub(crate) async fn bootstrap_set(
.with_description(concat!( .with_description(concat!(
"The selected data store contains information from an older version. ", "The selected data store contains information from an older version. ",
"Please follow the upgrade instructions at ", "Please follow the upgrade instructions at ",
"https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING/v0_16.md" "https://docs.inbuxa.org/install/migrating/"
)), )),
); );
break; break;
@@ -657,7 +657,7 @@ fn map_dns_server(dns_server: &DnsServerBootstrap) -> Option<registry::schema::s
// FreeBSD keeps variable application data under /var/db (hier(7)) // FreeBSD keeps variable application data under /var/db (hier(7))
// rather than FHS /var/lib. // rather than FHS /var/lib.
const DEFAULT_DATA_PATH: &str = if cfg!(target_os = "freebsd") { const DEFAULT_DATA_PATH: &str = if cfg!(target_os = "freebsd") {
"/var/db/stalwart/" "/var/db/inbuxa/"
} else { } else {
"/var/lib/inbuxa/" "/var/lib/inbuxa/"
}; };
@@ -679,7 +679,7 @@ fn build_default_bootstrap(server: &Server) -> Bootstrap {
directory: DirectoryBootstrap::Internal, directory: DirectoryBootstrap::Internal,
tracer: Tracer::Log(TracerLog { tracer: Tracer::Log(TracerLog {
path: "/var/log/inbuxa/".to_string(), path: "/var/log/inbuxa/".to_string(),
prefix: "stalwart".to_string(), prefix: "inbuxa".to_string(),
ansi: true, ansi: true,
enable: true, enable: true,
..Default::default() ..Default::default()
+19 -21
View File
@@ -38,7 +38,7 @@ use directory::core::secret::{hash_secret, is_password_hash};
use http_proto::HttpSessionData; use http_proto::HttpSessionData;
use jmap_proto::{ use jmap_proto::{
error::set::{SetError, SetErrorType}, error::set::{SetError, SetErrorType},
method::set::{SetRequest, SetResponse}, method::set::{SetRequest, SetResponse, SettingsReload},
object::registry::Registry, object::registry::Registry,
references::resolve::ResolveCreatedReference, references::resolve::ResolveCreatedReference,
request::{IntoValid, MaybeInvalid}, request::{IntoValid, MaybeInvalid},
@@ -931,30 +931,28 @@ impl RegistrySet for Server {
} }
}; };
// inbuxa: DIR-17: a directory or the server default applies on the // inbuxa: a write to an object the running settings are built from
// next request, here and on every node // applies at once, here and on every node (DIR-17 did this for
if matches!( // directories and the server default; now it covers every such object)
object_type, let mut result = result;
ObjectType::Directory | ObjectType::Authentication if let Ok(response) = &mut result
) && let Ok(response) = &result
&& (!response.created.is_empty() && (!response.created.is_empty()
|| !response.updated.is_empty() || !response.updated.is_empty()
|| !response.destroyed.is_empty()) || !response.destroyed.is_empty())
&& let Some(reload) = self.reload_after_write(object_type).await
{ {
let change = common::ipc::RegistryChange::Reload(ObjectType::Directory); response.settings_reload = Some(match reload {
match Box::pin(self.reload_registry(change)).await { Ok(()) => SettingsReload {
Ok(reload) if !reload.has_errors() => { applied: true,
self.cluster_broadcast(common::ipc::BroadcastEvent::RegistryChange(change)) description: None,
.await; },
} Err(reason) => SettingsReload {
Ok(_) => trc::event!( applied: false,
Registry(trc::RegistryEvent::BuildWarning), description: Some(format!(
Details = "Settings didn't reload after a directory change", "Saved, but the running settings were not reloaded. {reason}"
), )),
Err(err) => { },
trc::error!(err.details("Failed to reload directories")); });
}
}
} }
result result
} }
+3 -3
View File
@@ -1,13 +1,13 @@
[package] [package]
name = "inbuxa" name = "inbuxa"
description = "INBUXA Mail and Collaboration Server, a fork of Stalwart" description = "inbuxa mail and collaboration server, a fork of Stalwart"
authors = [ "Stalwart Labs LLC <[email protected]>"] authors = [ "Stalwart Labs LLC <[email protected]>"]
homepage = "https://inbuxa.org" homepage = "https://inbuxa.org"
keywords = ["imap", "jmap", "smtp", "email", "mail", "webdav", "server"] keywords = ["imap", "jmap", "smtp", "email", "mail", "webdav", "server"]
categories = ["email"] categories = ["email"]
# Upstream offers AGPL-3.0-only OR LicenseRef-SEL; INBUXA takes the AGPL only. # Upstream offers AGPL-3.0-only OR LicenseRef-SEL; inbuxa takes the AGPL only.
license = "AGPL-3.0-only" license = "AGPL-3.0-only"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[[bin]] [[bin]]
+6
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
#![warn(clippy::large_futures)] #![warn(clippy::large_futures)]
@@ -107,6 +109,10 @@ async fn main() -> std::io::Result<()> {
// Wait for shutdown signal // Wait for shutdown signal
wait_for_shutdown().await; wait_for_shutdown().await;
// inbuxa: hand back the task locks this node holds, so other nodes can
// run those tasks now rather than when the locks expire
services::task_manager::lock::release_task_locks(&inner.build_server()).await;
// Shutdown collector // Shutdown collector
Collector::shutdown(); Collector::shutdown();
+2 -2
View File
@@ -57,7 +57,7 @@ pub async fn insert_test_data(server: &Server) {
server.inner.data.queue_id_gen.generate(), server.inner.data.queue_id_gen.generate(),
QueueName::default(), QueueName::default(),
); );
assert!(qm.save_changes(server, None).await); assert!(qm.save_changes(server, None, None).await);
} }
for report in sample_tls_internal_reports() { for report in sample_tls_internal_reports() {
@@ -163,7 +163,7 @@ fn sample_queued_messages(blob_hashes: Vec<BlobHash>) -> Vec<Message> {
}, },
}), }),
flags: RCPT_DSN_SENT, flags: RCPT_DSN_SENT,
orcpt: Some("rfc822;[email protected]".into()), orcpt: Some("[email protected]".into()),
}, },
], ],
received_from_ip: std::net::IpAddr::V4(Ipv4Addr::new(192, 168, 1, 10)), received_from_ip: std::net::IpAddr::V4(Ipv4Addr::new(192, 168, 1, 10)),
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "managesieve" name = "managesieve"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "migration" name = "migration"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+53 -3
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
#![warn(clippy::large_futures)] #![warn(clippy::large_futures)]
@@ -19,6 +21,10 @@ pub mod destroy;
pub mod v016; pub mod v016;
pub async fn try_migrate(server: &Server) -> trc::Result<()> { pub async fn try_migrate(server: &Server) -> trc::Result<()> {
// inbuxa: before the version check, which returns early on a current
// store, and before migrate_v0_16, which reads the renamed key.
rename_spam_blobs(server).await?;
match server match server
.store() .store()
.get_value::<u32>(AnyKey { .get_value::<u32>(AnyKey {
@@ -36,14 +42,14 @@ pub async fn try_migrate(server: &Server) -> trc::Result<()> {
Some(0..=4) => { Some(0..=4) => {
abort(concat!( abort(concat!(
"You must first upgrade to version 0.15, please read ", "You must first upgrade to version 0.15, please read ",
"https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING/v0_16.md" "https://docs.inbuxa.org/install/migrating/"
)); ));
} }
Some(5) => { Some(5) => {
if !server.registry().is_recovery_mode() { if !server.registry().is_recovery_mode() {
abort(concat!( abort(concat!(
"Upgrading to version 0.16 is a multi-step process, please read ", "Upgrading to version 0.16 is a multi-step process, please read ",
"https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING/v0_16.md" "https://docs.inbuxa.org/install/migrating/"
)); ));
} }
} }
@@ -61,7 +67,7 @@ pub async fn try_migrate(server: &Server) -> trc::Result<()> {
} else { } else {
abort(concat!( abort(concat!(
"You must first upgrade to version 0.15, please read ", "You must first upgrade to version 0.15, please read ",
"https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING/v0_16.md" "https://docs.inbuxa.org/install/migrating/"
)); ));
} }
} }
@@ -134,3 +140,47 @@ async fn is_new_install(server: &Server) -> trc::Result<bool> {
Ok(true) Ok(true)
} }
/// inbuxa: the spam filter's trainer and model blobs, under the names they
/// had before the fork renamed them (SPEC §2.4), paired with the current ones.
const RENAMED_SPAM_BLOBS: [(&[u8], &[u8]); 2] = [
(b"STALWART_SPAM_TRAIN_DATA.lz4", common::manager::SPAM_TRAINER_KEY),
(
b"STALWART_SPAM_CLASSIFIER_MODEL.lz4",
common::manager::SPAM_CLASSIFIER_KEY,
),
];
/// Moves each spam blob from its pre-rename key to the current one, so a
/// trained model survives the rename. A blob already under the current key
/// wins and the old one is just removed; with neither, nothing happens.
async fn rename_spam_blobs(server: &Server) -> trc::Result<()> {
let blobs = server.blob_store();
for (old, new) in RENAMED_SPAM_BLOBS {
let Some(data) = blobs
.get_blob(old, 0..usize::MAX)
.await
.caused_by(trc::location!())?
else {
continue;
};
if blobs
.get_blob(new, 0..usize::MAX)
.await
.caused_by(trc::location!())?
.is_none()
{
blobs
.put_blob(new, &data, server.core.email.compression)
.await
.caused_by(trc::location!())?;
}
blobs.delete_blob(old).await.caused_by(trc::location!())?;
trc::event!(
Server(trc::ServerEvent::Startup),
Details = "Moved a spam filter blob to its renamed key",
Key = new,
);
}
Ok(())
}
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "nlp" name = "nlp"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "pop3" name = "pop3"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+2 -8
View File
@@ -64,14 +64,8 @@ impl<T: SessionStream> Session<T> {
) )
.get_full_range(); .get_full_range();
self.write_bytes( self.write_bytes(Response::Message::<u32> { bytes, lines }.serialize())
Response::Message::<u32> { .await
bytes,
lines: lines.unwrap_or(0),
}
.serialize(),
)
.await
} else { } else {
Err(trc::Pop3Event::Error Err(trc::Pop3Event::Error
.into_err() .into_err()
+86 -19
View File
@@ -16,7 +16,7 @@ pub enum Response<'x, T> {
List(Vec<T>), List(Vec<T>),
Message { Message {
bytes: SliceRange<'x>, bytes: SliceRange<'x>,
lines: u32, lines: Option<u32>,
}, },
Capability { Capability {
mechanisms: Vec<Mechanism>, mechanisms: Vec<Mechanism>,
@@ -54,40 +54,65 @@ impl<'x, T: Display> Response<'x, T> {
buf buf
} }
Response::Message { bytes, lines } => { Response::Message { bytes, lines } => {
let mut buf = Vec::with_capacity(bytes.len() + 10); let lines = *lines;
buf.extend_from_slice(b"+OK "); let mut message = Vec::with_capacity(bytes.len() + 16);
buf.extend_from_slice(bytes.len().to_string().as_bytes()); let mut octets = 0;
buf.extend_from_slice(b" octets\r\n"); let mut last_byte = b'\n';
let mut in_headers = lines.is_some();
let mut line_count = 0; let mut is_blank_line = true;
let mut last_byte = 0; let mut body_lines = 0;
// Transparency procedure // Transparency procedure
for &byte in bytes.into_iter() { for &byte in bytes.into_iter() {
// POP3 requires that lines end with CRLF, do this check to ensure that // POP3 requires that lines end with CRLF, do this check to ensure that
if byte == b'\n' && last_byte != b'\r' { if byte == b'\n' && last_byte != b'\r' {
buf.push(b'\r'); message.push(b'\r');
octets += 1;
} }
if byte == b'.' && last_byte == b'\n' { if byte == b'.' && last_byte == b'\n' {
buf.push(b'.'); message.push(b'.');
} }
buf.push(byte); message.push(byte);
octets += 1;
last_byte = byte; last_byte = byte;
if *lines > 0 && byte == b'\n' { match byte {
line_count += 1; b'\n' => {
if line_count == *lines { if in_headers {
break; in_headers = !is_blank_line;
} else {
body_lines += 1;
}
if !in_headers && lines.is_some_and(|lines| body_lines >= lines) {
break;
}
is_blank_line = true;
}
b'\r' => {}
_ => {
is_blank_line = false;
} }
} }
} }
if last_byte != b'\n' { if last_byte != b'\n' {
buf.extend_from_slice(b"\r\n"); message.extend_from_slice(b"\r\n");
octets += 2;
} }
buf.extend_from_slice(b".\r\n"); if in_headers {
message.extend_from_slice(b"\r\n");
octets += 2;
}
message.extend_from_slice(b".\r\n");
let mut buf = Vec::with_capacity(message.len() + 24);
buf.extend_from_slice(b"+OK ");
buf.extend_from_slice(octets.to_string().as_bytes());
buf.extend_from_slice(b" octets\r\n");
buf.extend_from_slice(&message);
buf buf
} }
Response::Capability { mechanisms, stls } => { Response::Capability { mechanisms, stls } => {
@@ -208,9 +233,51 @@ mod tests {
( (
Response::Message { Response::Message {
bytes: SliceRange::Split(b"Subject: test\r\n\r\n.\r\n", b"test.\r\n.test\r\na"), bytes: SliceRange::Split(b"Subject: test\r\n\r\n.\r\n", b"test.\r\n.test\r\na"),
lines: 0, lines: None,
}, },
"+OK 35 octets\r\nSubject: test\r\n\r\n..\r\ntest.\r\n..test\r\na\r\n.\r\n", "+OK 37 octets\r\nSubject: test\r\n\r\n..\r\ntest.\r\n..test\r\na\r\n.\r\n",
),
(
Response::Message {
bytes: SliceRange::Split(b"Subject: test\r\n\r\n.\r\n", b"test.\r\n.test\r\na"),
lines: Some(0),
},
"+OK 17 octets\r\nSubject: test\r\n\r\n.\r\n",
),
(
Response::Message {
bytes: SliceRange::Split(b"Subject: test\r\n\r\n.\r\n", b"test.\r\n.test\r\na"),
lines: Some(2),
},
"+OK 27 octets\r\nSubject: test\r\n\r\n..\r\ntest.\r\n.\r\n",
),
(
Response::Message {
bytes: SliceRange::Split(b"Subject: test\r\n\r\n.\r\n", b"test.\r\n.test\r\na"),
lines: Some(100),
},
"+OK 37 octets\r\nSubject: test\r\n\r\n..\r\ntest.\r\n..test\r\na\r\n.\r\n",
),
(
Response::Message {
bytes: SliceRange::Single(b"Subject: test\n\nbody\n"),
lines: None,
},
"+OK 23 octets\r\nSubject: test\r\n\r\nbody\r\n.\r\n",
),
(
Response::Message {
bytes: SliceRange::Single(b"Subject: test\n\n.leading dot\n"),
lines: Some(1),
},
"+OK 31 octets\r\nSubject: test\r\n\r\n..leading dot\r\n.\r\n",
),
(
Response::Message {
bytes: SliceRange::Single(b".dot\r\nSubject: test\r\n"),
lines: Some(3),
},
"+OK 23 octets\r\n..dot\r\nSubject: test\r\n\r\n.\r\n",
), ),
] { ] {
assert_eq!(expected, String::from_utf8(cmd.serialize()).unwrap()); assert_eq!(expected, String::from_utf8(cmd.serialize()).unwrap());
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "registry" name = "registry"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+6 -4
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
// This file is auto-generated. Do not edit directly. // This file is auto-generated. Do not edit directly.
@@ -10372,8 +10374,8 @@ impl EnumImpl for SieveCapability {
b"spamtest" => SieveCapability::Spamtest, b"spamtest" => SieveCapability::Spamtest,
b"spamtestplus" => SieveCapability::Spamtestplus, b"spamtestplus" => SieveCapability::Spamtestplus,
b"virustest" => SieveCapability::Virustest, b"virustest" => SieveCapability::Virustest,
b"vnd.stalwart.while" => SieveCapability::VndStalwartWhile, b"vnd.inbuxa.while" => SieveCapability::VndStalwartWhile,
b"vnd.stalwart.expressions" => SieveCapability::VndStalwartExpressions, b"vnd.inbuxa.expressions" => SieveCapability::VndStalwartExpressions,
} }
} }
@@ -10426,8 +10428,8 @@ impl EnumImpl for SieveCapability {
SieveCapability::Spamtest => "spamtest", SieveCapability::Spamtest => "spamtest",
SieveCapability::Spamtestplus => "spamtestplus", SieveCapability::Spamtestplus => "spamtestplus",
SieveCapability::Virustest => "virustest", SieveCapability::Virustest => "virustest",
SieveCapability::VndStalwartWhile => "vnd.stalwart.while", SieveCapability::VndStalwartWhile => "vnd.inbuxa.while",
SieveCapability::VndStalwartExpressions => "vnd.stalwart.expressions", SieveCapability::VndStalwartExpressions => "vnd.inbuxa.expressions",
} }
} }
+18 -18
View File
@@ -732,7 +732,7 @@ impl Pickle for AddressBook {
impl Default for AddressBook { impl Default for AddressBook {
fn default() -> Self { fn default() -> Self {
Self { Self {
default_display_name: Some("INBUXA Address Book".to_string()), default_display_name: Some("inbuxa Address Book".to_string()),
default_href_name: Some("default".to_string()), default_href_name: Some("default".to_string()),
max_v_card_size: 524288u64, max_v_card_size: 524288u64,
max_address_books: Some(250u64), max_address_books: Some(250u64),
@@ -4597,7 +4597,7 @@ impl Pickle for Calendar {
impl Default for Calendar { impl Default for Calendar {
fn default() -> Self { fn default() -> Self {
Self { Self {
default_display_name: Some("INBUXA Calendar".to_string()), default_display_name: Some("inbuxa Calendar".to_string()),
default_href_name: Some("default".to_string()), default_href_name: Some("default".to_string()),
max_attendees: 20u64, max_attendees: 20u64,
max_recurrence_expansions: 3000u64, max_recurrence_expansions: 3000u64,
@@ -4734,7 +4734,7 @@ impl Default for CalendarAlarm {
allow_external_rcpts: false, allow_external_rcpts: false,
enable: true, enable: true,
from_email: Default::default(), from_email: Default::default(),
from_name: "INBUXA Calendar".to_string(), from_name: "inbuxa Calendar".to_string(),
min_trigger_interval: Duration::from_millis(3600000), min_trigger_interval: Duration::from_millis(3600000),
template: Default::default(), template: Default::default(),
} }
@@ -28310,7 +28310,7 @@ impl MtaStageConnect {
ExpressionContext { ExpressionContext {
expr: &self.smtp_greeting, expr: &self.smtp_greeting,
default: Some(Expression { default: Some(Expression {
else_: "system('hostname') + ' INBUXA ESMTP at your service'".to_string(), else_: "system('hostname') + ' inbuxa ESMTP at your service'".to_string(),
..Default::default() ..Default::default()
}), }),
property: Property::SmtpGreeting, property: Property::SmtpGreeting,
@@ -28374,7 +28374,7 @@ impl Default for MtaStageConnect {
fn default() -> Self { fn default() -> Self {
Self { Self {
smtp_greeting: Expression { smtp_greeting: Expression {
else_: "system('hostname') + ' INBUXA ESMTP at your service'".to_string(), else_: "system('hostname') + ' inbuxa ESMTP at your service'".to_string(),
..Default::default() ..Default::default()
}, },
hostname: Expression { hostname: Expression {
@@ -29622,8 +29622,8 @@ impl Default for MySqlSettings {
Self { Self {
host: Default::default(), host: Default::default(),
port: 3306u64, port: 3306u64,
database: "stalwart".to_string(), database: "inbuxa".to_string(),
auth_username: Some("stalwart".to_string()), auth_username: Some("inbuxa".to_string()),
auth_secret: Default::default(), auth_secret: Default::default(),
} }
} }
@@ -29780,8 +29780,8 @@ impl Default for MySqlStore {
read_replicas: Default::default(), read_replicas: Default::default(),
host: Default::default(), host: Default::default(),
port: 3306u64, port: 3306u64,
database: "stalwart".to_string(), database: "inbuxa".to_string(),
auth_username: Some("stalwart".to_string()), auth_username: Some("inbuxa".to_string()),
auth_secret: Default::default(), auth_secret: Default::default(),
} }
} }
@@ -29942,7 +29942,7 @@ impl Default for NatsCoordinator {
no_echo: true, no_echo: true,
use_tls: false, use_tls: false,
auth_secret: Default::default(), auth_secret: Default::default(),
auth_username: Some("stalwart".to_string()), auth_username: Some("inbuxa".to_string()),
credentials: Default::default(), credentials: Default::default(),
} }
} }
@@ -31125,8 +31125,8 @@ impl Default for PostgreSqlSettings {
Self { Self {
host: Default::default(), host: Default::default(),
port: 5432u64, port: 5432u64,
database: "stalwart".to_string(), database: "inbuxa".to_string(),
auth_username: Some("stalwart".to_string()), auth_username: Some("inbuxa".to_string()),
auth_secret: Default::default(), auth_secret: Default::default(),
options: Default::default(), options: Default::default(),
} }
@@ -31270,8 +31270,8 @@ impl Default for PostgreSqlStore {
read_replicas: Default::default(), read_replicas: Default::default(),
host: Default::default(), host: Default::default(),
port: 5432u64, port: 5432u64,
database: "stalwart".to_string(), database: "inbuxa".to_string(),
auth_username: Some("stalwart".to_string()), auth_username: Some("inbuxa".to_string()),
auth_secret: Default::default(), auth_secret: Default::default(),
options: Default::default(), options: Default::default(),
} }
@@ -32576,7 +32576,7 @@ impl Default for RedisClusterStore {
Self { Self {
urls: Map::new(vec!["redis://127.0.0.1".to_string()]), urls: Map::new(vec!["redis://127.0.0.1".to_string()]),
timeout: Duration::from_millis(10000), timeout: Duration::from_millis(10000),
auth_username: Some("stalwart".to_string()), auth_username: Some("inbuxa".to_string()),
auth_secret: Default::default(), auth_secret: Default::default(),
max_retry_wait: Default::default(), max_retry_wait: Default::default(),
min_retry_wait: Default::default(), min_retry_wait: Default::default(),
@@ -32743,7 +32743,7 @@ impl Default for RedisSentinelStore {
urls: Map::new(vec!["redis://127.0.0.1:26379".to_string()]), urls: Map::new(vec!["redis://127.0.0.1:26379".to_string()]),
service_name: "mymaster".to_string(), service_name: "mymaster".to_string(),
timeout: Duration::from_millis(10000), timeout: Duration::from_millis(10000),
auth_username: Some("stalwart".to_string()), auth_username: Some("inbuxa".to_string()),
auth_secret: Default::default(), auth_secret: Default::default(),
sentinel_username: Default::default(), sentinel_username: Default::default(),
sentinel_secret: Default::default(), sentinel_secret: Default::default(),
@@ -40215,7 +40215,7 @@ impl Default for SpamSettings {
score_reject: Float::new(0.0f64), score_reject: Float::new(0.0f64),
score_spam: Float::new(5.0f64), score_spam: Float::new(5.0f64),
trust_replies: true, trust_replies: true,
spam_filter_rules_url: Some("https://github.com/stalwartlabs/spam-filter/releases/latest/download/spam-filter-rules.json.gz".to_string()), spam_filter_rules_url: None,
} }
} }
} }
@@ -46308,7 +46308,7 @@ impl Default for TracerLog {
fn default() -> Self { fn default() -> Self {
Self { Self {
path: Default::default(), path: Default::default(),
prefix: "stalwart".to_string(), prefix: "inbuxa".to_string(),
rotate: LogRotateFrequency::Daily, rotate: LogRotateFrequency::Daily,
ansi: true, ansi: true,
multiline: false, multiline: false,
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "scim-proto" name = "scim-proto"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "scim" name = "scim"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+9 -1
View File
@@ -4,6 +4,14 @@
* SPDX-License-Identifier: AGPL-3.0-only * SPDX-License-Identifier: AGPL-3.0-only
*/ */
// The release profile computes the layout of this crate's async fn bodies in
// one go, and the deepest of them -- writable_domain, which awaits through
// the directory, the store and the JMAP registry -- takes rustc past its
// default query depth. The dev profile does not get that far, so the failure
// only appears in a release build: CI was green and the tag that started a
// release was not.
#![recursion_limit = "256"]
//! SCIM 2.0 provisioning (`docs/spec/features/scim.md`). inbuxa-server is the //! SCIM 2.0 provisioning (`docs/spec/features/scim.md`). inbuxa-server is the
//! service provider: an identity provider pushes users and groups to //! service provider: an identity provider pushes users and groups to
//! `/scim/v2`, and each request becomes the same `x:Account` reads and //! `/scim/v2`, and each request becomes the same `x:Account` reads and
@@ -205,7 +213,7 @@ impl ScimResponse {
if error.status == 401 { if error.status == 401 {
response.headers.push(( response.headers.push((
"WWW-Authenticate", "WWW-Authenticate",
"Bearer realm=\"INBUXA SCIM\"".to_string(), "Bearer realm=\"inbuxa SCIM\"".to_string(),
)); ));
} }
response response
+1 -1
View File
@@ -1,6 +1,6 @@
[package] [package]
name = "services" name = "services"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+33 -9
View File
@@ -26,7 +26,7 @@ pub fn spawn_broadcast_subscriber(inner: Arc<Inner>, mut shutdown_rx: watch::Rec
}; };
tokio::spawn(async move { tokio::spawn(async move {
let mut retry_count = 0; let mut retry_count: u32 = 0;
trc::event!(Cluster(ClusterEvent::SubscriberStart)); trc::event!(Cluster(ClusterEvent::SubscriberStart));
@@ -53,7 +53,7 @@ pub fn spawn_broadcast_subscriber(inner: Arc<Inner>, mut shutdown_rx: watch::Rec
); );
match tokio::time::timeout( match tokio::time::timeout(
Duration::from_secs(1 << retry_count.max(6)), subscribe_retry_delay(retry_count),
shutdown_rx.changed(), shutdown_rx.changed(),
) )
.await .await
@@ -62,7 +62,7 @@ pub fn spawn_broadcast_subscriber(inner: Arc<Inner>, mut shutdown_rx: watch::Rec
break; break;
} }
Err(_) => { Err(_) => {
retry_count += 1; retry_count = retry_count.saturating_add(1);
continue; continue;
} }
} }
@@ -96,6 +96,13 @@ pub fn spawn_broadcast_subscriber(inner: Arc<Inner>, mut shutdown_rx: watch::Rec
} }
}; };
inner
.shared_core
.load()
.storage
.data
.invalidate_read_snapshot();
loop { loop {
match batch.next_event() { match batch.next_event() {
Ok(Some(event)) => { Ok(Some(event)) => {
@@ -174,9 +181,7 @@ pub fn spawn_broadcast_subscriber(inner: Arc<Inner>, mut shutdown_rx: watch::Rec
.await; .await;
} }
BroadcastEvent::QueueRefresh => { BroadcastEvent::QueueRefresh => {
let core = inner.shared_core.load_full(); if inner.shared_core.load().network.roles.outbound_mta {
if core.network.roles.outbound_mta {
core.storage.data.invalidate_read_snapshot();
let _ = inner let _ = inner
.ipc .ipc
.queue_tx .queue_tx
@@ -185,9 +190,7 @@ pub fn spawn_broadcast_subscriber(inner: Arc<Inner>, mut shutdown_rx: watch::Rec
} }
} }
BroadcastEvent::RegistryChange(change) => { BroadcastEvent::RegistryChange(change) => {
let server = inner.build_server(); match Box::pin(inner.build_server().reload_registry(change)).await {
server.store().invalidate_read_snapshot();
match Box::pin(server.reload_registry(change)).await {
Ok(result) => { Ok(result) => {
result.log(); result.log();
} }
@@ -231,6 +234,11 @@ pub fn spawn_broadcast_subscriber(inner: Arc<Inner>, mut shutdown_rx: watch::Rec
}); });
} }
/// Delay before the next subscribe attempt: 1 s, 2 s, 4 s ... capped at 64 s.
fn subscribe_retry_delay(retry_count: u32) -> Duration {
Duration::from_secs(1u64 << retry_count.min(6))
}
fn log_event(event: &BroadcastEvent) -> trc::Value { fn log_event(event: &BroadcastEvent) -> trc::Value {
match event { match event {
BroadcastEvent::PushNotification(notification) => match notification { BroadcastEvent::PushNotification(notification) => match notification {
@@ -293,3 +301,19 @@ fn log_event(event: &BroadcastEvent) -> trc::Value {
BroadcastEvent::QueueRefresh => "QueueRefresh".into(), BroadcastEvent::QueueRefresh => "QueueRefresh".into(),
} }
} }
#[cfg(test)]
mod tests {
use super::subscribe_retry_delay;
use std::time::Duration;
#[test]
fn subscribe_retry_backoff_grows_then_caps() {
let schedule: Vec<u64> = (0..10)
.map(|n| subscribe_retry_delay(n).as_secs())
.collect();
assert_eq!(schedule, vec![1, 2, 4, 8, 16, 32, 64, 64, 64, 64]);
// No shift overflow at the top of the range.
assert_eq!(subscribe_retry_delay(u32::MAX), Duration::from_secs(64));
}
}
@@ -6,7 +6,7 @@
* Modified by Coffey Labs in 2026 for INBUXA. * Modified by Coffey Labs in 2026 for INBUXA.
*/ */
use crate::task_manager::TaskResult; use crate::task_manager::{TaskResult, deferred_retry_time};
use common::Server; use common::Server;
use email::{message::metadata::MessageMetadata, sieve::SieveScript}; use email::{message::metadata::MessageMetadata, sieve::SieveScript};
use groupware::file::FileNode; use groupware::file::FileNode;
@@ -41,7 +41,7 @@ impl DestroyAccountTask for Server {
match destroy_account(self, task).await { match destroy_account(self, task).await {
Ok(result) => result, Ok(result) => result,
Err(err) => { Err(err) => {
let result = TaskResult::temporary(err.to_string()); let result = TaskResult::deferred(deferred_retry_time(&err), err.to_string());
trc::error!( trc::error!(
err.account_id(task.account_id.document_id()) err.account_id(task.account_id.document_id())
.details("Failed to destroy account") .details("Failed to destroy account")
+69 -40
View File
@@ -6,7 +6,7 @@
* Modified by Coffey Labs in 2026 for INBUXA. * Modified by Coffey Labs in 2026 for INBUXA.
*/ */
use crate::task_manager::{Task, TaskDetails, TaskFailureType, TaskResult}; use crate::task_manager::{Task, TaskDetails, TaskFailureType, TaskResult, deferred_retry_time};
use common::Server; use common::Server;
use email::{ use email::{
cache::MessageCacheFetch, cache::MessageCacheFetch,
@@ -91,7 +91,15 @@ impl SearchIndexTask for Server {
build_contact_document(self, account_id, document_id).await build_contact_document(self, account_id, document_id).await
} }
IndexDocumentType::File => { IndexDocumentType::File => {
// File indexing not implemented yet // File indexing not implemented yet. inbuxa: still
// one result per task: update_tasks pairs them by
// position, and a missing one shifts every result
// after it onto the wrong task
results.push(IndexTaskResult {
task_type: TaskType::Insert,
index: task.document_type,
result: TaskResult::Ignored,
});
continue; continue;
} }
}; };
@@ -274,7 +282,7 @@ impl SearchIndexTask for Server {
); );
for r in results.iter_mut() { for r in results.iter_mut() {
if r.task_type == TaskType::Insert && r.result.is_success() { if r.task_type == TaskType::Insert && r.result.is_success() {
r.result = search_store_failure(retry_at, "Failed to index documents"); r.result = TaskResult::deferred(retry_at, "Failed to index documents");
} }
} }
return results; return results;
@@ -331,7 +339,7 @@ impl SearchIndexTask for Server {
for r in results.iter_mut() { for r in results.iter_mut() {
if r.task_type == TaskType::Delete && r.result.is_success() { if r.task_type == TaskType::Delete && r.result.is_success() {
r.result = r.result =
search_store_failure(retry_at, "Failed to delete documents from index"); TaskResult::deferred(retry_at, "Failed to delete documents from index");
} }
} }
return results; return results;
@@ -445,22 +453,6 @@ pub(crate) async fn reindex_account(server: &Server, account_id: u32) -> trc::Re
Ok(()) Ok(())
} }
fn deferred_retry_time(err: &trc::Error) -> Option<u64> {
err.value(trc::Key::NextRetry)
.and_then(|value| value.to_uint())
}
fn search_store_failure(retry_at: Option<u64>, message: &'static str) -> TaskResult {
match retry_at {
Some(retry_at) => TaskResult::Failure {
typ: TaskFailureType::Retry(retry_at),
message: message.into(),
max_attempts: None,
},
None => TaskResult::temporary(message),
}
}
fn attempt_number(status: &TaskStatus) -> u64 { fn attempt_number(status: &TaskStatus) -> u64 {
match status { match status {
TaskStatus::Pending(_) => 0, TaskStatus::Pending(_) => 0,
@@ -583,20 +575,14 @@ async fn build_contact_document(
} }
// inbuxa: MON-16: a trace's search document, when trace search is on: // inbuxa: MON-16: a trace's search document, when trace search is on
// its event types, queue ids, and addresses, their domains, hosts, IPs,
// message ids and account names as keywords
async fn build_tracing_span_document( async fn build_tracing_span_document(
server: &Server, server: &Server,
span_id: u64, span_id: u64,
) -> trc::Result<Option<IndexDocument>> { ) -> trc::Result<Option<IndexDocument>> {
use common::telemetry::tracers::store::MaybeTrace; use common::telemetry::tracers::store::MaybeTrace;
use registry::schema::{enums::SearchTracingField, structs::Search}; use registry::schema::structs::Search;
use store::{ use store::write::{TelemetryClass, ValueClass};
search::TracingSearchField,
write::{TelemetryClass, ValueClass},
};
use trc::Key;
let settings = server let settings = server
.registry() .registry()
@@ -606,7 +592,6 @@ async fn build_tracing_span_document(
if !settings.index_telemetry { if !settings.index_telemetry {
return Ok(None); return Ok(None);
} }
let wants = |field: SearchTracingField| settings.index_tracing_fields.iter().any(|f| *f == field);
let Some(MaybeTrace(Some(trace))) = server let Some(MaybeTrace(Some(trace))) = server
.tracing_store() .tracing_store()
.get_value::<MaybeTrace>(ValueKey::from(ValueClass::Telemetry(TelemetryClass::Span( .get_value::<MaybeTrace>(ValueKey::from(ValueClass::Telemetry(TelemetryClass::Span(
@@ -617,23 +602,67 @@ async fn build_tracing_span_document(
return Ok(None); return Ok(None);
}; };
Ok(Some(trace_search_document(
span_id,
&trace,
&settings
.index_tracing_fields
.iter()
.copied()
.collect::<Vec<_>>(),
)))
}
/// inbuxa: MON-16: the search document for a stored trace.
///
/// The event type and queue id columns are integers on every search backend
/// (BIGINT on PostgreSQL and MySQL, long on Elasticsearch), and each holds a
/// single value per trace: the event type is the trace's opening event, the
/// one `x:Trace/query` filters on, and the queue id is the first queue id the
/// trace mentions. Every queue id also goes into the keywords, so a session
/// that queued several messages is found by any of them.
pub fn trace_search_document(
span_id: u64,
trace: &registry::schema::structs::Trace,
fields: &[registry::schema::enums::SearchTracingField],
) -> IndexDocument {
use registry::schema::{enums::SearchTracingField, structs::TraceValue};
use store::search::TracingSearchField;
use trc::Key;
let wants = |field: SearchTracingField| fields.contains(&field);
let mut document = IndexDocument::new(SearchIndex::Tracing).with_id(span_id); let mut document = IndexDocument::new(SearchIndex::Tracing).with_id(span_id);
if wants(SearchTracingField::EventType)
&& let Some(first) = trace.events.iter().next()
{
document.index_unsigned(TracingSearchField::EventType, first.event.to_id() as u64);
}
let mut seen = store::ahash::AHashSet::new(); let mut seen = store::ahash::AHashSet::new();
let mut queue_id_indexed = false;
for event in trace.events.iter() { for event in trace.events.iter() {
if wants(SearchTracingField::EventType) && seen.insert(event.event.as_str().to_string()) {
document.index_keyword(TracingSearchField::EventType, event.event.as_str());
}
for kv in event.key_values.iter() { for kv in event.key_values.iter() {
let text = match &kv.value { let text = match &kv.value {
registry::schema::structs::TraceValue::String(v) => v.value.clone(), TraceValue::String(v) => v.value.clone(),
registry::schema::structs::TraceValue::UnsignedInt(v) => v.value.to_string(), TraceValue::UnsignedInt(v) => v.value.to_string(),
registry::schema::structs::TraceValue::IpAddr(v) => v.value.to_string(), TraceValue::IpAddr(v) => v.value.to_string(),
_ => continue, _ => continue,
}; };
match kv.key { match kv.key {
Key::QueueId if wants(SearchTracingField::QueueId) => { Key::QueueId => {
if seen.insert(format!("q:{text}")) { let Ok(queue_id) = text.parse::<u64>() else {
document.index_keyword(TracingSearchField::QueueId, &text); continue;
};
if wants(SearchTracingField::QueueId) && !queue_id_indexed {
document.index_unsigned(TracingSearchField::QueueId, queue_id);
queue_id_indexed = true;
}
if wants(SearchTracingField::Keywords) && seen.insert(format!("k:{text}")) {
document.index_text(
TracingSearchField::Keywords,
&text,
nlp::language::Language::None,
);
} }
} }
Key::From Key::From
@@ -664,7 +693,7 @@ async fn build_tracing_span_document(
} }
} }
} }
Ok(Some(document)) document
} }
// inbuxa: UD-1, UD-4: archives a deleted file, event or contact noted at // inbuxa: UD-1, UD-4: archives a deleted file, event or contact noted at
+73 -2
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
use crate::task_manager::*; use crate::task_manager::*;
@@ -13,13 +15,21 @@ pub trait TaskLockManager: Sync + Send {
impl TaskLockManager for Server { impl TaskLockManager for Server {
async fn try_lock_task(&self, id: u64) -> bool { async fn try_lock_task(&self, id: u64) -> bool {
// inbuxa: a node that is stopping claims nothing new
let locks = &self.inner.ipc.task_locks;
if locks.is_stopping() {
return false;
}
match self match self
.in_memory_store() .in_memory_store()
.try_lock(KV_LOCK_TASK, &id.to_be_bytes(), DEFAULT_LOCK_EXPIRY) .try_lock(KV_LOCK_TASK, &id.to_be_bytes(), locks.expiry())
.await .await
{ {
Ok(result) => { Ok(result) => {
if !result { if result {
locks.insert(id);
} else {
trc::event!( trc::event!(
TaskManager(TaskManagerEvent::TaskLocked), TaskManager(TaskManagerEvent::TaskLocked),
Id = id, Id = id,
@@ -48,5 +58,66 @@ impl TaskLockManager for Server {
.caused_by(trc::location!()) .caused_by(trc::location!())
); );
} }
self.inner.ipc.task_locks.remove(id);
} }
} }
/// inbuxa: on a graceful stop, stops claiming tasks and releases every task
/// lock this node holds, so the rest of the cluster can pick the tasks up at
/// once instead of after the lock expires. Returns how many were released.
pub async fn release_task_locks(server: &Server) -> usize {
let ids = server.inner.ipc.task_locks.stop();
for id in &ids {
if let Err(err) = server
.in_memory_store()
.remove_lock(KV_LOCK_TASK, &id.to_be_bytes())
.await
{
trc::error!(
err.details("Failed to release task lock on shutdown")
.ctx(trc::Key::Id, *id)
.caused_by(trc::location!())
);
}
}
ids.len()
}
/// inbuxa: renews the lease on every task this node is running, so it stays
/// claimed for as long as it runs while a node that dies loses its claims
/// within one lock lifetime. Returns how many leases were renewed and how
/// many were found lost (expired, perhaps taken by another node).
pub async fn renew_task_locks(server: &Server) -> (usize, usize) {
let locks = &server.inner.ipc.task_locks;
let expiry = locks.expiry();
let (mut renewed, mut lost) = (0, 0);
for id in locks.held_ids() {
match server
.in_memory_store()
.renew_lock(KV_LOCK_TASK, &id.to_be_bytes(), expiry)
.await
{
Ok(true) => renewed += 1,
Ok(false) => {
// Still held here as far as this node knows; the task
// finishes and its lock is removed as usual
if locks.is_held(id) {
lost += 1;
trc::event!(
TaskManager(TaskManagerEvent::TaskLocked),
Id = id,
Details = "Task lock expired while the task was running",
);
}
}
Err(err) => {
trc::error!(
err.details("Failed to renew task lock")
.ctx(trc::Key::Id, id)
.caused_by(trc::location!())
);
}
}
}
(renewed, lost)
}
+273 -161
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
use crate::task_manager::acme::AcmeTask; use crate::task_manager::acme::AcmeTask;
@@ -11,17 +13,18 @@ use crate::task_manager::dkim::DkimManagementTask;
use crate::task_manager::dns::DnsManagementTask; use crate::task_manager::dns::DnsManagementTask;
use crate::task_manager::imip::SendImipTask; use crate::task_manager::imip::SendImipTask;
use crate::task_manager::index::SearchIndexTask; use crate::task_manager::index::SearchIndexTask;
use crate::task_manager::lock::TaskLockManager; use crate::task_manager::lock::{TaskLockManager, renew_task_locks};
use crate::task_manager::maintenance::MaintenanceTask; use crate::task_manager::maintenance::MaintenanceTask;
use crate::task_manager::merge_threads::MergeThreadsTask; use crate::task_manager::merge_threads::MergeThreadsTask;
use crate::task_manager::report::{self, SubmitReportTask}; use crate::task_manager::report::{self, SubmitReportTask};
use crate::task_manager::restore_item::RestoreItemTask; use crate::task_manager::restore_item::RestoreItemTask;
use crate::task_manager::spam_classifier::SpamFilterMaintenanceTask; use crate::task_manager::spam_classifier::SpamFilterMaintenanceTask;
use crate::task_manager::{ use crate::task_manager::{
DEFAULT_LOCK_EXPIRY, Locked, QUEUE_REFRESH_INTERVAL, TaskDetails, TaskFailureType, TaskInfo, CLAIM_RECHECK_INTERVAL, Locked, QUEUE_REFRESH_INTERVAL, TaskDetails, TaskFailureType, TaskInfo,
TaskJob, TaskManagerIpc, TaskResult, TaskJob, TaskManagerIpc, TaskResult,
}; };
use common::BuildServer; use common::BuildServer;
use common::config::network::ClusterRoles;
use common::config::server::{DEFAULT_TLS_TIMEOUT, ServerProtocol}; use common::config::server::{DEFAULT_TLS_TIMEOUT, ServerProtocol};
use common::network::limiter::ConcurrencyLimiter; use common::network::limiter::ConcurrencyLimiter;
use common::network::{ServerInstance, TcpAcceptor}; use common::network::{ServerInstance, TcpAcceptor};
@@ -56,10 +59,12 @@ pub fn spawn_task_manager(inner: Arc<Inner>) {
let server = inner.build_server(); let server = inner.build_server();
let roles = &server.core.network.roles; let roles = &server.core.network.roles;
// inbuxa: outbound_mta too, which now governs report tasks
if !roles.account_maintenance if !roles.account_maintenance
&& !roles.store_maintenance && !roles.store_maintenance
&& !roles.search_indexing && !roles.search_indexing
&& !roles.spam_training && !roles.spam_training
&& !roles.outbound_mta
&& !roles.task_manager && !roles.task_manager
{ {
return; return;
@@ -70,6 +75,28 @@ pub fn spawn_task_manager(inner: Arc<Inner>) {
trc::event!(TaskManager(TaskManagerEvent::ManagerStarted)); trc::event!(TaskManager(TaskManagerEvent::ManagerStarted));
// inbuxa: keep the leases of running tasks alive, every third of a lock
// lifetime, until the node stops
{
let inner = inner.clone();
tokio::spawn(async move {
let mut renewed_at = Instant::now();
loop {
tokio::time::sleep(Duration::from_secs(1)).await;
let locks = &inner.ipc.task_locks;
if locks.is_stopping() {
break;
}
if renewed_at.elapsed() >= Duration::from_secs((locks.expiry() / 3).max(1)) {
renewed_at = Instant::now();
if locks.held() > 0 {
renew_task_locks(&inner.build_server()).await;
}
}
}
});
}
// Create dummy server instance for alarms // Create dummy server instance for alarms
let server_instance = Arc::new(ServerInstance { let server_instance = Arc::new(ServerInstance {
id: "_local".to_string(), id: "_local".to_string(),
@@ -124,72 +151,47 @@ pub fn spawn_task_manager(inner: Arc<Inner>) {
let server = inner.build_server(); let server = inner.build_server();
let batch_size = server.core.email.index_batch_size; let batch_size = server.core.email.index_batch_size;
let mut batch = Vec::with_capacity(batch_size); let mut batch = Vec::with_capacity(batch_size);
match server if let Some(task) = fetch_task(&server, job).await {
.store() batch.push(task);
.get_value::<Task>(ValueKey::from(ValueClass::TaskQueue(
TaskQueueClass::Task { id: job.id },
)))
.await
{
Ok(Some(task)) => {
batch.push(TaskDetails { task, info: job });
}
Ok(None) => {
trc::event!(
TaskManager(TaskManagerEvent::TaskIgnored),
Id = job.id,
Reason = "Task not found in store, likely already processed.",
);
}
Err(err) => {
trc::error!(
err.id(job.id)
.details("Failed to retrieve task details.")
.caused_by(trc::location!())
);
}
} }
while batch.len() < batch_size { while batch.len() < batch_size {
match rx.try_recv() { match rx.try_recv() {
Ok(job) => { Ok(job) => {
match server if let Some(task) = fetch_task(&server, job).await {
.store() batch.push(task);
.get_value::<Task>(ValueKey::from(ValueClass::TaskQueue(
TaskQueueClass::Task { id: job.id },
)))
.await
{
Ok(Some(task)) => {
batch.push(TaskDetails { task, info: job });
}
Ok(None) => {
trc::event!(
TaskManager(TaskManagerEvent::TaskIgnored),
Id = job.id,
Reason = "Task not found in store, likely already processed.",
);
}
Err(err) => {
trc::error!(
err.id(job.id)
.details("Failed to retrieve task details.")
.caused_by(trc::location!())
);
}
} }
} }
Err(_) => break, Err(_) => break,
} }
} }
// Dispatch // Dispatch. inbuxa: on a task of its own, so a panic
// releases the batch's locks and leaves this worker
// running; a dead worker would keep claiming tasks it
// can never run
let mut refresh_queue = false; let mut refresh_queue = false;
let results = server.index(&batch).await.into_iter().map(|r| { let ids = batch.iter().map(|task| task.info.id).collect::<Vec<_>>();
refresh_queue |= r.result.is_retry(); let run = {
r.result let server = server.clone();
}); tokio::spawn(async move {
update_tasks(&server, &mut batch, results).await; let results = server.index(&batch).await;
(batch, results)
})
};
match run.await {
Ok((mut batch, results)) => {
let results = results.into_iter().map(|r| {
refresh_queue |= r.result.is_retry();
r.result
});
update_tasks(&server, &mut batch, results).await;
}
Err(err) => {
worker_failed(&server, &ids, err).await;
refresh_queue = true;
}
}
if refresh_queue || rx.is_empty() { if refresh_queue || rx.is_empty() {
server.notify_task_queue(); server.notify_task_queue();
@@ -203,83 +205,31 @@ pub fn spawn_task_manager(inner: Arc<Inner>) {
let server = inner.build_server(); let server = inner.build_server();
let mut refresh_queue = false; let mut refresh_queue = false;
match server if let Some(TaskDetails { task, info }) = fetch_task(&server, job).await {
.store() // inbuxa: on a task of its own, as above
.get_value::<Task>(ValueKey::from(ValueClass::TaskQueue( let run = {
TaskQueueClass::Task { id: job.id }, let server = server.clone();
))) let server_instance = server_instance.clone();
.await tokio::spawn(async move {
{ let result = run_task(&server, &task, server_instance).await;
Ok(Some(task)) => { (task, result)
let result = match &task { })
Task::CalendarAlarmEmail(task) => { };
server.send_email_alarm(task, server_instance.clone()).await match run.await {
} Ok((task, result)) => {
Task::CalendarAlarmNotification(task) => { refresh_queue = result.is_retry();
server.send_display_alarm(task).await
}
Task::CalendarItipMessage(task) => {
server.send_imip(task, server_instance.clone()).await
}
Task::MergeThreads(task) => server.merge_threads(task).await,
Task::DmarcReport(task) => {
server
.submit_report(report::ReportId::Dmarc(task.report_id.id()))
.await
}
Task::TlsReport(task) => {
server
.submit_report(report::ReportId::Tls(task.report_id.id()))
.await
}
Task::RestoreArchivedItem(task) => server.restore_item(task).await,
Task::DestroyAccount(task) => server.destroy_account(task).await,
Task::AccountMaintenance(task) => {
server.account_maintenance(task).await
}
Task::TenantMaintenance(task) => {
server.tenant_maintenance(task).await
}
Task::StoreMaintenance(task) => {
server.store_maintenance(task).await
}
Task::SpamFilterMaintenance(task) => {
Box::pin(server.spam_filter_maintenance(task)).await
}
Task::AcmeRenewal(task) => server.acme_management(task).await,
Task::DkimManagement(task_dkim_rotation) => {
server.dkim_management(task_dkim_rotation).await
}
Task::DnsManagement(task_dns_management) => {
server.dns_management(task_dns_management).await
}
Task::IndexDocument(_)
| Task::UnindexDocument(_)
| Task::IndexTrace(_) => unreachable!(),
};
refresh_queue = result.is_retry(); update_tasks(
&server,
update_tasks( &mut [TaskDetails { task, info }],
&server, vec![result],
&mut [TaskDetails { task, info: job }], )
vec![result], .await;
) }
.await; Err(err) => {
} worker_failed(&server, &[info.id], err).await;
Ok(None) => { refresh_queue = true;
trc::event!( }
TaskManager(TaskManagerEvent::TaskIgnored),
Id = job.id,
Reason = "Task not found in store, likely already processed.",
);
}
Err(err) => {
trc::error!(
err.id(job.id)
.details("Failed to retrieve task details.")
.caused_by(trc::location!())
);
} }
} }
@@ -318,6 +268,13 @@ pub(crate) trait TaskQueueManager: Sync + Send {
impl TaskQueueManager for Server { impl TaskQueueManager for Server {
async fn process_tasks(&self, ipc: &mut TaskManagerIpc) -> Duration { async fn process_tasks(&self, ipc: &mut TaskManagerIpc) -> Duration {
// inbuxa: a node that is stopping has released its locks and claims
// nothing new
let task_locks = &self.inner.ipc.task_locks;
if task_locks.is_stopping() {
return Duration::from_secs(QUEUE_REFRESH_INTERVAL);
}
let lock_expiry = task_locks.expiry();
let now_timestamp = now(); let now_timestamp = now();
let from_key = ValueKey::<ValueClass> { let from_key = ValueKey::<ValueClass> {
account_id: 0, account_id: 0,
@@ -357,26 +314,13 @@ impl TaskQueueManager for Server {
.caused_by(trc::location!()) .caused_by(trc::location!())
.ctx(trc::Key::Value, value) .ctx(trc::Key::Value, value)
})?; })?;
let enabled = match task_type { // inbuxa: running here under a lease this node
TaskType::IndexDocument // renews; don't hand it to a worker again
| TaskType::UnindexDocument if task_locks.is_held(task_id) {
| TaskType::IndexTrace => roles.search_indexing, return Ok(true);
TaskType::AccountMaintenance }
| TaskType::TenantMaintenance
| TaskType::DestroyAccount => roles.account_maintenance, let enabled = task_enabled(roles, task_type);
TaskType::StoreMaintenance => roles.store_maintenance,
TaskType::SpamFilterMaintenance => roles.spam_training,
TaskType::CalendarAlarmEmail
| TaskType::CalendarAlarmNotification
| TaskType::CalendarItipMessage
| TaskType::MergeThreads
| TaskType::DmarcReport
| TaskType::TlsReport
| TaskType::RestoreArchivedItem
| TaskType::AcmeRenewal
| TaskType::DkimManagement
| TaskType::DnsManagement => true,
};
if !enabled { if !enabled {
trc::event!( trc::event!(
@@ -393,9 +337,7 @@ impl TaskQueueManager for Server {
let locked = entry.get_mut(); let locked = entry.get_mut();
if locked.expires <= now || locked.due < task_due { if locked.expires <= now || locked.due < task_due {
locked.expires = Instant::now() locked.expires = Instant::now()
+ std::time::Duration::from_secs( + std::time::Duration::from_secs(lock_expiry + 1);
DEFAULT_LOCK_EXPIRY + 1,
);
locked.due = task_due; locked.due = task_due;
tasks.push(( tasks.push((
TaskJob { TaskJob {
@@ -411,9 +353,7 @@ impl TaskQueueManager for Server {
Entry::Vacant(entry) => { Entry::Vacant(entry) => {
entry.insert(Locked { entry.insert(Locked {
expires: Instant::now() expires: Instant::now()
+ std::time::Duration::from_secs( + std::time::Duration::from_secs(lock_expiry + 1),
DEFAULT_LOCK_EXPIRY + 1,
),
due: task_due, due: task_due,
revision: ipc.revision, revision: ipc.revision,
}); });
@@ -464,12 +404,26 @@ impl TaskQueueManager for Server {
let tx = &ipc.txs[task_type_idx as usize]; let tx = &ipc.txs[task_type_idx as usize];
if tx.capacity() > 0 { if tx.capacity() > 0 {
if self.try_lock_task(task_job.id).await && tx.send(task_job).await.is_err() { let id = task_job.id;
if !self.try_lock_task(id).await {
// inbuxa: another node holds the task. Look again after a
// short while rather than a full lock lifetime from now:
// the holder may have claimed it after this scan began,
// or run on a clock ahead of this one, and waiting the
// whole lifetime again would leave the task stuck for
// another hour past its lock if that holder died
if let Some(locked) = ipc.locked.get_mut(&id) {
locked.expires =
Instant::now() + Duration::from_secs(claim_recheck_interval(lock_expiry));
}
} else if tx.send(task_job).await.is_err() {
trc::event!( trc::event!(
Server(trc::ServerEvent::ThreadError), Server(trc::ServerEvent::ThreadError),
Details = "Error sending task.", Details = "Error sending task.",
CausedBy = trc::location!() CausedBy = trc::location!()
); );
// inbuxa: nothing will run it here, so don't hold it
self.remove_index_lock(id).await;
} }
} else { } else {
// If the channel is full, release the lock so it can be picked up in the next iteration // If the channel is full, release the lock so it can be picked up in the next iteration
@@ -481,9 +435,159 @@ impl TaskQueueManager for Server {
let now = Instant::now(); let now = Instant::now();
ipc.locked ipc.locked
.retain(|_, locked| locked.expires > now && locked.revision == ipc.revision); .retain(|_, locked| locked.expires > now && locked.revision == ipc.revision);
Duration::from_secs(next_event.map_or(QUEUE_REFRESH_INTERVAL, |timestamp| { let sleep_for = Duration::from_secs(next_event.map_or(QUEUE_REFRESH_INTERVAL, |timestamp| {
timestamp.saturating_sub(store::write::now()) timestamp.saturating_sub(store::write::now())
})) }));
// inbuxa: wake up when a claim held elsewhere is due to be tried
// again, rather than only on the next task or refresh
ipc.locked
.values()
.map(|locked| locked.expires.saturating_duration_since(now))
.min()
.map_or(sleep_for, |recheck| sleep_for.min(recheck.max(Duration::from_secs(1))))
}
}
/// inbuxa: whether this node's cluster role lets it run a task type. Upstream
/// checked the dedicated roles (search indexing, account and store
/// maintenance, spam training) and let every node with a task manager run
/// the rest, whatever its taskQueueProcessing setting. Every task type now
/// answers to one ClusterTaskType:
///
/// - IndexDocument, UnindexDocument, IndexTrace: searchIndexing
/// - AccountMaintenance, TenantMaintenance, DestroyAccount: accountMaintenance
/// - StoreMaintenance: storeMaintenance
/// - SpamFilterMaintenance: spamClassifierTraining
/// - DmarcReport, TlsReport: outboundMta. They build and send reports to
/// other domains (TLS reports can go straight to an HTTPS endpoint), which
/// is the outbound MTA's business.
/// - CalendarAlarmEmail, CalendarAlarmNotification, CalendarItipMessage,
/// MergeThreads, RestoreArchivedItem, AcmeRenewal, DkimManagement,
/// DnsManagement: taskQueueProcessing, the role for queue tasks with no
/// role of their own.
///
/// A node that may not run a task leaves it unclaimed, so a node that may
/// picks it up.
pub fn task_enabled(roles: &ClusterRoles, task_type: TaskType) -> bool {
match task_type {
TaskType::IndexDocument | TaskType::UnindexDocument | TaskType::IndexTrace => {
roles.search_indexing
}
TaskType::AccountMaintenance | TaskType::TenantMaintenance | TaskType::DestroyAccount => {
roles.account_maintenance
}
TaskType::StoreMaintenance => roles.store_maintenance,
TaskType::SpamFilterMaintenance => roles.spam_training,
TaskType::DmarcReport | TaskType::TlsReport => roles.outbound_mta,
TaskType::CalendarAlarmEmail
| TaskType::CalendarAlarmNotification
| TaskType::CalendarItipMessage
| TaskType::MergeThreads
| TaskType::RestoreArchivedItem
| TaskType::AcmeRenewal
| TaskType::DkimManagement
| TaskType::DnsManagement => roles.task_manager,
}
}
async fn run_task(
server: &Server,
task: &Task,
server_instance: Arc<ServerInstance>,
) -> TaskResult {
match task {
Task::CalendarAlarmEmail(task) => {
server.send_email_alarm(task, server_instance.clone()).await
}
Task::CalendarAlarmNotification(task) => {
server.send_display_alarm(task).await
}
Task::CalendarItipMessage(task) => {
server.send_imip(task, server_instance.clone()).await
}
Task::MergeThreads(task) => server.merge_threads(task).await,
Task::DmarcReport(task) => {
server
.submit_report(report::ReportId::Dmarc(task.report_id.id()))
.await
}
Task::TlsReport(task) => {
server
.submit_report(report::ReportId::Tls(task.report_id.id()))
.await
}
Task::RestoreArchivedItem(task) => server.restore_item(task).await,
Task::DestroyAccount(task) => server.destroy_account(task).await,
Task::AccountMaintenance(task) => {
server.account_maintenance(task).await
}
Task::TenantMaintenance(task) => {
server.tenant_maintenance(task).await
}
Task::StoreMaintenance(task) => {
server.store_maintenance(task).await
}
Task::SpamFilterMaintenance(task) => {
Box::pin(server.spam_filter_maintenance(task)).await
}
Task::AcmeRenewal(task) => server.acme_management(task).await,
Task::DkimManagement(task_dkim_rotation) => {
server.dkim_management(task_dkim_rotation).await
}
Task::DnsManagement(task_dns_management) => {
server.dns_management(task_dns_management).await
}
Task::IndexDocument(_)
| Task::UnindexDocument(_)
| Task::IndexTrace(_) => unreachable!(),
}
}
/// Reads a claimed task. When it is gone or can't be read, the claim is
/// released: inbuxa: holding it would block the task, everywhere, until
/// the lock expired.
async fn fetch_task(server: &Server, job: TaskJob) -> Option<TaskDetails> {
match server
.store()
.get_value::<Task>(ValueKey::from(ValueClass::TaskQueue(TaskQueueClass::Task {
id: job.id,
})))
.await
{
Ok(Some(task)) => Some(TaskDetails { task, info: job }),
Ok(None) => {
trc::event!(
TaskManager(TaskManagerEvent::TaskIgnored),
Id = job.id,
Reason = "Task not found in store, likely already processed.",
);
server.remove_index_lock(job.id).await;
None
}
Err(err) => {
trc::error!(
err.id(job.id)
.details("Failed to retrieve task details.")
.caused_by(trc::location!())
);
server.remove_index_lock(job.id).await;
None
}
}
}
/// inbuxa: a task panicked: its locks are released so it runs again, here or
/// on another node, and the worker carries on.
async fn worker_failed(server: &Server, ids: &[u64], err: tokio::task::JoinError) {
trc::event!(
Server(trc::ServerEvent::ThreadError),
Details = "Task worker failed",
Reason = err.to_string(),
CausedBy = trc::location!()
);
for id in ids {
server.remove_index_lock(*id).await;
} }
} }
@@ -614,6 +718,13 @@ async fn update_tasks(
} }
} }
/// inbuxa: how long to wait before trying again to claim a task another node
/// holds: a twelfth of the lock lifetime, so five minutes for the one-hour
/// lock, never more than that and never under a second.
pub(crate) fn claim_recheck_interval(lock_expiry: u64) -> u64 {
(lock_expiry / 12).clamp(1, CLAIM_RECHECK_INTERVAL)
}
pub fn perpetual_retry_time(typ: TaskType, attempt: u64) -> Option<u64> { pub fn perpetual_retry_time(typ: TaskType, attempt: u64) -> Option<u64> {
matches!( matches!(
typ, typ,
@@ -621,6 +732,7 @@ pub fn perpetual_retry_time(typ: TaskType, attempt: u64) -> Option<u64> {
| TaskType::DkimManagement | TaskType::DkimManagement
| TaskType::IndexDocument | TaskType::IndexDocument
| TaskType::UnindexDocument | TaskType::UnindexDocument
| TaskType::DestroyAccount
) )
.then(|| { .then(|| {
now().saturating_add( now().saturating_add(
+19 -1
View File
@@ -35,7 +35,9 @@ pub mod scheduler;
pub mod spam_classifier; pub mod spam_classifier;
const QUEUE_REFRESH_INTERVAL: u64 = 60 * 5; // 5 minutes const QUEUE_REFRESH_INTERVAL: u64 = 60 * 5; // 5 minutes
const DEFAULT_LOCK_EXPIRY: u64 = 60 * 60; // 1 hour // inbuxa: the lock lifetime (one hour) lives in common::ipc::TaskLocks, per
// server, so a graceful stop can release the locks and the tests can shorten it
const CLAIM_RECHECK_INTERVAL: u64 = 60 * 5; // 5 minutes
pub(crate) struct TaskManagerIpc { pub(crate) struct TaskManagerIpc {
txs: [mpsc::Sender<TaskJob>; TaskType::COUNT], txs: [mpsc::Sender<TaskJob>; TaskType::COUNT],
@@ -137,4 +139,20 @@ impl TaskResult {
max_attempts: None, max_attempts: None,
} }
} }
pub fn deferred(retry_at: Option<u64>, message: impl Into<String>) -> Self {
match retry_at {
Some(retry_at) => TaskResult::Failure {
typ: TaskFailureType::Retry(retry_at),
message: message.into(),
max_attempts: None,
},
None => TaskResult::temporary(message),
}
}
}
pub(crate) fn deferred_retry_time(err: &trc::Error) -> Option<u64> {
err.value(trc::Key::NextRetry)
.and_then(|value| value.to_uint())
} }
@@ -2,13 +2,15 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
use crate::task_manager::{TaskFailureType, TaskResult}; use crate::task_manager::{TaskFailureType, TaskResult};
use common::{ use common::{
Server, Server,
ipc::{BroadcastEvent, RegistryChange}, ipc::{BroadcastEvent, RegistryChange},
manager::{SPAM_CLASSIFIER_KEY, SPAM_TRAINER_KEY, fetch_resource}, manager::{SPAM_CLASSIFIER_KEY, SPAM_TRAINER_KEY, fetch_resource, spam_rules},
}; };
use registry::{ use registry::{
schema::{ schema::{
@@ -106,6 +108,7 @@ struct RuleUpdateResult {
async fn update_spam_rules(server: &Server) -> trc::Result<TaskResult> { async fn update_spam_rules(server: &Server) -> trc::Result<TaskResult> {
let started = Instant::now(); let started = Instant::now();
let bundled = server.core.spam.spam_rules_url.is_none();
let rules = match fetch_spam_rules(server).await { let rules = match fetch_spam_rules(server).await {
Ok(rules) => rules, Ok(rules) => rules,
Err(err) => { Err(err) => {
@@ -289,29 +292,36 @@ async fn update_spam_rules(server: &Server) -> trc::Result<TaskResult> {
Elapsed = started.elapsed(), Elapsed = started.elapsed(),
); );
// inbuxa: so the next start knows these bundled rules are in
if bundled {
spam_rules::set_applied_version(server.store(), spam_rules::BUNDLED_SPAM_RULES_VERSION)
.await?;
}
Ok(TaskResult::Success(vec![])) Ok(TaskResult::Success(vec![]))
} }
async fn fetch_spam_rules(server: &Server) -> Result<Rules, RuleUpdateError> { async fn fetch_spam_rules(server: &Server) -> Result<Rules, RuleUpdateError> {
let Some(rules_url) = server.core.spam.spam_rules_url.as_ref() else { // inbuxa: no URL means the rules bundled with the server
return Err(RuleUpdateError { let bytes = match server.core.spam.spam_rules_url.as_ref() {
typ: TaskFailureType::Permanent, Some(rules_url) => fetch_resource(rules_url, None, Duration::from_secs(60), 1024 * 500)
reason: "Spam rules resource URL not configured".to_string(),
});
};
let rules_json: AHashMap<String, Vec<serde_json::Value>> =
fetch_resource(rules_url, None, Duration::from_secs(60), 1024 * 500)
.await .await
.map_err(|reason| RuleUpdateError { .map_err(|reason| RuleUpdateError {
typ: TaskFailureType::Temporary, typ: TaskFailureType::Temporary,
reason, reason,
}),
None => spam_rules::bundled_rules().map_err(|reason| RuleUpdateError {
typ: TaskFailureType::Permanent,
reason,
}),
};
let rules_json: AHashMap<String, Vec<serde_json::Value>> =
bytes.and_then(|bytes| {
serde_json::from_slice(&bytes).map_err(|err| RuleUpdateError {
typ: TaskFailureType::Permanent,
reason: format!("Failed to parse spam rules JSON: {err}"),
}) })
.and_then(|bytes| { })?;
serde_json::from_slice(&bytes).map_err(|err| RuleUpdateError {
typ: TaskFailureType::Permanent,
reason: format!("Failed to parse spam rules JSON: {err}"),
})
})?;
let mut rules = Rules::default(); let mut rules = Rules::default();
for (object_type, values) in rules_json { for (object_type, values) in rules_json {
+3 -4
View File
@@ -1,13 +1,12 @@
[package] [package]
name = "smtp" name = "smtp"
description = "Stalwart SMTP Server" description = "inbuxa SMTP server"
authors = [ "Stalwart Labs LLC <[email protected]>"] authors = [ "Stalwart Labs LLC <[email protected]>"]
repository = "https://github.com/stalwartlabs/smtp-server" homepage = "https://inbuxa.org"
homepage = "https://stalw.art/smtp"
keywords = ["smtp", "email", "mail", "server"] keywords = ["smtp", "email", "mail", "server"]
categories = ["email"] categories = ["email"]
license = "AGPL-3.0-only OR LicenseRef-SEL" license = "AGPL-3.0-only OR LicenseRef-SEL"
version = "0.16.22" version = "0.16.23"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
+8 -4
View File
@@ -11,6 +11,7 @@ use common::{
config::smtp::auth::VerifyStrategy, config::smtp::auth::VerifyStrategy,
network::{ServerInstance, asn::AsnGeoLookupResult}, network::{ServerInstance, asn::AsnGeoLookupResult},
}; };
use email::message::delivery::ORCPT_ADDR_TYPE;
use mail_auth::{IprevOutput, SpfOutput}; use mail_auth::{IprevOutput, SpfOutput};
use smtp_proto::request::receiver::{ use smtp_proto::request::receiver::{
BdatReceiver, DataReceiver, DummyDataReceiver, DummyLineReceiver, LineReceiver, RequestReceiver, BdatReceiver, DataReceiver, DummyDataReceiver, DummyLineReceiver, LineReceiver, RequestReceiver,
@@ -306,10 +307,13 @@ impl SessionAddress {
} }
} }
pub fn report_address(&self) -> &str { pub fn orig_address(&self) -> &str {
self.dsn_info.as_deref().unwrap_or(&self.address_lcase)
}
pub fn orcpt_parameter(&self) -> Option<String> {
self.dsn_info self.dsn_info
.as_ref() .as_deref()
.and_then(|v| v.strip_prefix("rfc822;")) .map(|orcpt| format!("{ORCPT_ADDR_TYPE}{}", orcpt.to_lowercase()))
.unwrap_or(&self.address_lcase)
} }
} }
+1 -1
View File
@@ -443,7 +443,7 @@ impl<T: SessionStream> Session<T> {
if !rc.analysis.forward { if !rc.analysis.forward {
self.data self.data
.rcpt_to .rcpt_to
.retain(|rcpt| !rc.analysis.is_report_address(rcpt.report_address())); .retain(|rcpt| !rc.analysis.is_report_address(rcpt.orig_address()));
} }
if self.data.rcpt_to.is_empty() { if self.data.rcpt_to.is_empty() {
+15 -1
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art> * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
* *
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/ */
use common::config::smtp::session::Milter; use common::config::smtp::session::Milter;
@@ -25,7 +27,19 @@ impl MilterClient<TcpStream> {
pub async fn connect(config: &Milter, session_id: u64) -> Result<Self> { pub async fn connect(config: &Milter, session_id: u64) -> Result<Self> {
tokio::time::timeout(config.timeout_command, async { tokio::time::timeout(config.timeout_command, async {
let mut last_err = Error::Disconnected; let mut last_err = Error::Disconnected;
for addr in &config.addrs { // inbuxa: a hostname is resolved here, per connection, rather
// than while the settings are built
let resolved;
let addrs = if config.addrs.is_empty() {
resolved = tokio::net::lookup_host((config.hostname.as_str(), config.port))
.await
.map_err(Error::Io)?
.collect::<Vec<_>>();
&resolved
} else {
&config.addrs
};
for addr in addrs {
match TcpStream::connect(addr).await { match TcpStream::connect(addr).await {
Ok(stream) => { Ok(stream) => {
return Ok(MilterClient { return Ok(MilterClient {
+2 -3
View File
@@ -202,8 +202,8 @@ impl<T: SessionStream> Session<T> {
let mut new_addr = SessionAddress::new(address); let mut new_addr = SessionAddress::new(address);
if !self.data.rcpt_to.contains(&new_addr) { if !self.data.rcpt_to.contains(&new_addr) {
new_addr.dsn_info = format!("rfc822;{}", orig_addr.address_lcase).into();
new_addr.flags = orig_addr.flags; new_addr.flags = orig_addr.flags;
new_addr.dsn_info = orig_addr.address_lcase.into();
self.data.rcpt_to.push(new_addr); self.data.rcpt_to.push(new_addr);
} else { } else {
trc::event!( trc::event!(
@@ -353,7 +353,6 @@ impl<T: SessionStream> Session<T> {
// Expand list // Expand list
if let Some(members) = rcpt_members { if let Some(members) = rcpt_members {
let list_addr = self.data.rcpt_to.pop().unwrap(); let list_addr = self.data.rcpt_to.pop().unwrap();
let orcpt = format!("rfc822;{}", list_addr.address_lcase);
for member in members.as_ref() { for member in members.as_ref() {
let member_lcase = member.to_lowercase(); let member_lcase = member.to_lowercase();
let is_local = match self let is_local = match self
@@ -399,7 +398,7 @@ impl<T: SessionStream> Session<T> {
if !self.data.rcpt_to.contains(&member_addr) if !self.data.rcpt_to.contains(&member_addr)
&& member_addr.address_lcase != list_addr.address_lcase && member_addr.address_lcase != list_addr.address_lcase
{ {
member_addr.dsn_info = orcpt.clone().into(); member_addr.dsn_info = list_addr.address_lcase.clone().into();
member_addr.flags = list_addr.flags; member_addr.flags = list_addr.flags;
self.data.rcpt_to.push(member_addr); self.data.rcpt_to.push(member_addr);
} }

Some files were not shown because too many files have changed in this diff Show More