7ba9ec9fa0d79dbffe6b676eb1d8048dc362e5ca
71
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 jcoffey-dev
|
538ae107d7 |
Legal holds, step 6: what each hold keeps
inbuxa:LegalHold/get answers accountsCovered, itemsHeld and sizeHeld when asked: the accounts a hold reaches now (deleted ones it keeps included) and the archived items it keeps, with their size. Worked out in one pass over accounts and archive, only for requests that name them. Held items stay out of the user's quota, as all archived copies do (LH-9). |
||
|
jcoffey-dev
|
39707cd2e8 |
Legal holds, step 5: held accounts can't be destroyed
Destroying a held account removes the login, as offboarding needs, but keeps its data as a deleted account with no expiry, whether or not undelete keeps accounts; its addresses stay reserved and its holds name it from then on. Destroy-now refuses it, and its DestroyAccount task defers itself while it's held or its time hasn't come. Holds placed or released later freeze or free kept accounts in the same settle pass, with 30 days' grace after the last release (LH-8, LH-10). |
||
|
jcoffey-dev
|
8d3e99bc00 |
Legal holds, step 4: freezing, release, and the audit log
Placing or widening a hold freezes what's already archived in its scope and range, its old deadline noted; releasing one gives each item no other hold covers that deadline back, or release plus 30 days if later. One pass over the archive does both and changes nothing twice (LH-6, LH-10, LH-11). A held archived item can't be destroyed; restoring still can, and the hold is named only to callers who may see holds (LH-7). Audit records about a held account survive the purge (AU-7). Fixes the daily clean-up of expired archived items (UD-13), which never found any: the registry's unfiltered query reads an all-ids index that archived items aren't in. Items are now walked account by account, kept deleted accounts included. Expired items were still removed whenever their account's archive was read. |
||
|
jcoffey-dev
|
7b97efbb7f |
Legal holds, step 3: deleted items in a held account are kept
Every way of deleting mail (JMAP, IMAP EXPUNGE, POP3, mailbox removal, Trash emptying) and Sieve scripts, events, contacts and files now asks how the account's deletions are kept: a hold keeps them with no expiry (archivedUntil 9999-12-31), even with undelete off; otherwise undelete's period applies as before (LH-4). A hold's date range decides by the item's own date (LH-3). Mail is noted as held at deletion and settled when it's archived, once its received date is known; outside the range it gets undelete's deadline or isn't kept. Events go by their start, with a day's slack for time zones; recurring events, contacts, files and scripts are held whole. A groupware item's note now stays until its archive succeeds, and a failure retries the task instead of being logged and lost (LH-5). |
||
|
jcoffey-dev
|
318783f444 |
Legal holds, step 2: who a hold covers
A hold reaches an account by name, through any of its addresses' domains, its groups or its tenant, as they are now, so an account added to a held domain later is held too. An account that leaves a held domain, group or tenant stays held: the registry write hook adds it to the hold by name on every account change, whoever makes it (LH-2). Server::holds_on answers for the deletion paths, from the store each time so a hold binds every node at once. |
||
|
jcoffey-dev
|
5d2e35b2dc |
Legal holds, step 1: the hold itself
inbuxa:LegalHold get/set places a hold on accounts, groups, domains, tenants or the whole server, with an optional date range. A hold's range and scope can only widen, a released hold is read-only, and none is ever deleted. Placing, changing and releasing each need a reason and are audited (LH-1, LH-3, LH-10, AU-12). Permissions 669-672 (see, place, widen or release, export held data) go to server administrators only; the tenant ceiling always strips them, as it does Impersonate (LH-13). Schema: Compliance > Legal Holds. What a hold keeps comes next, through the undelete hooks. Also moves the lock expiry helpers below the lock module's imports. |
||
|
jcoffey-dev
|
9f6761c9dd |
Writing delegates may add at the top of a locked account's Files
A shared account refuses top-level folders, so an organize or full delegate couldn't add anything to a locked account with no folders. A delegate who may write now can, as the owner could; the reconcile after the create grants it the new folder. Read delegates still can't (AL-6, AL-7). |
||
|
jcoffey-dev
|
d4d127fa7d |
Delegates reach the whole locked account
A delegate's token listed the locked account only for kinds of data it held grants on, so one with no files (or no calendar) was refused to the delegate outright: "You do not have access to account". The token now lists the locked account for mail, calendars, contacts and files alike, so an empty kind reads as empty. What the delegate may see or change is still each container's grant (AL-7). |
||
|
jcoffey-dev
|
447229f871 |
Lock accounts: keep receiving mail, no sign-in, hand to delegates
A locked account can't sign in (it fails as a wrong password does), its sessions end on every node, refresh tokens stop working, and its Sieve scripts forward and reply to nothing. Mail keeps arriving. Delegates get real ACL grants on the account's mailboxes, calendars, address books and files at read, organize or full, with the rights they replaced restored on unlock. Folders made later are granted after the create and in a daily sweep. Organize delegates can't destroy; send-as needs organize or full. The JMAP session marks delegated accounts in urn:inbuxa:jmap. New inbuxa:AccountLock object with get/set, permissions 665-668, and a Compliance > Locked Accounts entry in the schema. Lock, unlock and delegate changes need a reason and are audited; delegate access and writes are audited too (audit-hold-lock spec AL-1 to AL-12). |
||
|
jcoffey-dev
|
86d7ebd982 |
Audit log: a permanent, tamper-evident record of admin actions
What administrators and the server itself do to the control plane is now recorded, from inbuxa-drafts/specs/audit-hold-lock.md (AU-1 to AU-12): settings, accounts, domains, roles and every other registry change, with each field's before and after (secrets only as "changed"); the fork's own settings objects; administrator sign-ins (and failed ones to administrator accounts), master-user and recovery-admin sign-ins, once an hour per account, method and address; access to another account's data through impersonation or FetchAnyBlob, once an hour; exports and tamper checks; and registry writes the server makes on its own, named by subsystem (system:AcmeRenewal, system:auto-ban, system:directory-sync, ...), with a spam rules update as one summary record. No change without its record (AU-3): before a set method changes anything, a pending record per requested create, update and destroy is written; if that fails, the method is refused with serverFail. Its outcome follows as a later entry. A change interrupted by a crash stays "unfinished". Records live in the fork's subspace under L, as one SHA-256 hash chain per node. The chain's head is stored, never cached, and every append asserts it, so two writers can't take the same place. Nothing can edit or delete a record; the daily purge removes the oldest past the retention (default 730 days, minimum 90) and records where the chain now starts, so verification still passes. security.audit-recorded (647) copies each record to webhooks, OpenTelemetry and the log; security.audit-write-failed (648) reports a failed write. New JMAP objects under urn:inbuxa:jmap: inbuxa:AuditEvent/get and /query (filters: time, actor, action, target, account, tenant, outcome, address, text), inbuxa:AuditSettings, inbuxa:AuditExport (CSV or JSON Lines built on the server, each line with its chain hash, ending in a manifest; the created object names the blob and its SHA-256) and inbuxa:AuditVerification. New permissions sysAuditGet, sysAuditExport and sysAuditSettingsUpdate: the Administrator role gets all three, the Tenant Administrator role gets read and export, once, on existing installs too. A tenant administrator sees records whose actor or target is in its tenant, including a server administrator's changes there. Sign-in method on the session: access tokens now remember how they signed in (password, app password, API key, OAuth client, directory, master user, recovery admin), including across the HTTP credential cache. New OAuth access tokens carry their client id in the sealed claims; older ones show as client "unknown" until they expire. The schema gains the permissions, the two events and a Management > Compliance > Audit Log link. Stack: the request layer boxes every inner future where it's made. Without that, a debug build overflowed the default 2 MB worker stack on a registry set; measured with the same request, the branch and main now overflow at the same stack size (between 1856 and 1920 KiB, debug), so the layer adds nothing measurable. Tests: unit tests in inbuxa-features and jmap; system::audit::audit_log_tests (run with --ignored) passes on RocksDB, SQLite, PostgreSQL, PostgreSQL with a read replica, MySQL, MySQL with a replica and FoundationDB. The system, JMAP and SCIM suites pass. authorization.rs skipped fork permissions that guard no registry object; the audit suite checks a plain user is refused instead. |
||
|
jcoffey-dev
|
7e7eca0883 |
Explain: shorter answers, streamed, remembered, and prepared for settings
ai-explain spec, amendment 1 (EX-22 to EX-28): - answers are three or four sentences, max_tokens 160, cut at 700 chars; - POST /api/explain streams the answer as server-sent events; - each node remembers answers in memory (1,000, 24 h), keyed by the facts, prompt version and model, shared by server-level administrators; - resources/explain/settings.json.gz ships answers for settings at their defaults, generated with prepare_setting_explanations (717 for 2026.9.27); - the system prompt no longer carries the per-request marker, so a model server can reuse it; - inbuxa:Explanation gains source, answeredAt and preparedFor. |
||
|
jcoffey-dev
|
b6f943a77c |
Don't listen on a socket whose bind failed
When a listener couldn't bind its address (a port below 1024 without root, a port already in use, or the legacy-protocols switch putting a listener back after privileges were dropped), the bind error was reported but the socket was still passed to listen(). The kernel then bound it itself, to a random port on every interface, and the server logged the listener as started on the port it was configured with. listen() now refuses a socket that isn't bound, so the listener is reported with a listen error and skipped, and nothing opens anywhere unexpected. |
||
|
jcoffey-dev
|
d9a6db025b |
Explain this: the local model reads delivery failures, verdicts, logs and settings
A new method, inbuxa:Explanation/set, asks the node's local model for a short plain-words reading of one thing an administrator is looking at: a failed recipient in the queue, a Classify verdict, a log line or trace event, or one setting with its saved value. The server builds the prompt itself from stored data and the registry schema, never from text the console sends, and grounds SMTP replies in RFC 3463 and RFC 5321. What the model is never shown: secrets (including ones nested inside a setting, like an AI model's HTTP auth), raw protocol events, and the contents of any other event. A tag name that doesn't have a tag's shape is refused before a model is asked. Calls share the AI gate with spam classification, but mail always keeps its slot, and Explain has its own hourly count per account and its own on/off switch in inbuxa:AiLimits. The permission is sysAiExplain, superuser only; tenant administrators can't use it. The session carries an aiExplain flag so a console knows when to offer the button. An install whose roles were stored before the permission existed gets it added once, at start-up, to the roles that are administrators' alone, not the User role their defaults share with every account. An operator who removes it later isn't overruled. Tests: unit tests in inbuxa-features and jmap, and ai_explain_tests (run with --ignored) covering the acceptance tests and the upgrade. |
||
|
jcoffey-dev
|
71ce11c57d |
Settings writes: wait for a burst to settle before reloading
A cluster rehearsal sent ten x:<Object>/set requests at once and got ten full reloads on every node. #39's coalescing only joined writes that queued behind a running reload, but the requests reached the server about 33 ms apart and a reload takes tens of milliseconds, so none overlapped one. A full reload after a registry write now waits for writes to settle: 75 ms after the last one, and at most 250 ms after the first it covers, so a steady stream still reloads at least four times a second. 75 ms is a little over twice the gap the rehearsal saw between requests. A single write pays it once: in the tests a settings write takes about 140 ms instead of 60. The reload runs in a task of its own, so a request that goes away doesn't cancel it for the others. Each write takes the result of the first reload that started after it was stored (the gate keeps the last 64 results), so applied true or false still describes the reload that covered that write. The 33 ms gap was a queue on the server, not password hashing: Basic credentials are cached per Authorization header, so they are checked once. Every authenticated HTTP request counted itself against the account's rate limit by incrementing one counter per account in the in-memory store, so parallel requests from one account queued on that key: a row lock on PostgreSQL (a few round trips to the database each) and conflict retries with a 50-300 ms backoff on RocksDB. An account with the unlimitedRequests permission (administrators, by default) passes the rate and concurrency limits anyway, so its requests are no longer counted. Ten parallel Core/echo calls as the admin now finish in 1-4 ms; before, they finished one after another over 20 ms on a local PostgreSQL and 300-450 ms on RocksDB. Other accounts still count every request. system::auto_reload::settings_reload_tests: ten concurrent writes now take one reload (the gate counts them; at most two allowed), all are applied: true and in the running settings, and a single write takes exactly one reload. RocksDB and PostgreSQL, 1 reload in 141-196 ms. With the old behavior (no wait, requests counted) the same writes took 5 reloads; without the wait but with the rate fix, 2. cluster::broadcast (3 nodes, PostgreSQL + NATS) and system::reload still pass. |
||
|
jcoffey-dev
|
a891667149 |
Tracers whose settings change start over on reload
A cluster rehearsal moved a Log tracer to another directory: the write was reported x:settingsReload applied:true, but the tracer kept writing to the old file until a restart. Telemetry::update only refreshed each running tracer's events, level and lossiness; a tracer's own settings (path, prefix, rotation, format, endpoint, headers, ...) stayed as built. Each tracer now carries a hash of the registry object it was built from, less the fields that change in place. The reload compares it with the running tracer's: unchanged ones are updated in place as before, changed ones are started over, new ones started and removed ones stopped. Only tracers this server started are removed; upstream removed every subscriber not in the settings, which also cut off live-tracing streams on each reload. Starting over is a swap in the collector, so no event is lost or written twice: a subscriber registered under a running one's id replaces it between two collection passes. The old one's batch is sent first (what its full channel can't take moves to the new one), and dropping it closes its channel, so its task writes what is queued and ends. Per tracer kind: - Log: a tracer started over on the same files (rotation or format changed) waits for the old one to finish, so lines don't interleave. - Webhook: the task held a sender of its own channel for retries, so it never ended; retries now use a weak sender, and pending events are posted when the channel closes. - OpenTelemetry: pending logs and spans are exported when the channel closes instead of dropped, and a span that was open across the swap is exported by the new tracer with the events it saw. - Console and journal: nothing kept between batches. - Trace history: built from the tracing store, which takes a restart, so it is never started over. No kind needs a restart, so x:settingsReload doesn't gain one. system::tracer_reload::tracer_reload_tests (new): a Log tracer created over JMAP writes to its directory; its path is changed over JMAP while 2000 numbered events are emitted; after the reload, events land in the new file and not the old one, each numbered event is in exactly one of the two files, and a destroyed tracer writes nothing. On main the new file never appears. |
||
|
jcoffey-dev
|
e00978c0b4 |
Cluster role changes apply to delivery and tasks without a restart
In cluster rehearsal 3, turning outboundMta off on node1's role was
reported applied (x:settingsReload applied: true), yet node1 kept
delivering mail, a report message included, until it was restarted.
The queue and report managers were started at boot only when the
node's role included outboundMta (crates/smtp/src/lib.rs), and the task
manager only when the role had some task type (spawn_task_manager).
After that nothing looked at the role again: a queue manager that was
running kept claiming and delivering, and one that wasn't never
started.
They now start on every node (outside recovery mode) and follow the
role live:
- Queue manager: before each scan it reads the role from the running
settings. Without outboundMta it claims nothing new; deliveries
already running finish and report back as usual, which releases
their locks. When the role comes back (a reload wakes the manager
with ReloadSettings, and it looks again every 30 s regardless) it
logs queue.started and scans the whole queue at once.
- Report scheduler: DMARC and TLS report events are handled only while
the role has outboundMta, as at boot; events arriving without it are
dropped, as they were on a node started without the role.
- Task manager: task_enabled already read the current role on every
scan. It now also runs on nodes whose role has no task type (the
scan returns at once until one is added), a job claimed before a
role change is handed back at once rather than run or held until
its lease lapses, and a settings reload wakes the manager so a role
that gained task types starts claiming them straight away.
Starting the queue manager on every node also drains the queue channel
on nodes without outboundMta. Upstream left that channel unread, so
each message queued there parked a refresh in it, and by the code,
queueing would block once 1024 had piled up (not reproduced here).
A role object edit reaches the nodes that name that role in
INBUXA_ROLE. Moving a node to another role still means changing its
environment, and so a restart. Listener changes in a role still need a
restart too (listeners bind at boot); this change is about tasks and
delivery.
cluster::live_roles::live_role_tests (new; PostgreSQL, two nodes over
one store):
1. A node started with outboundMta delivers and runs a TLS report
task; after its role loses outboundMta and the settings reload, a
new message isn't attempted and a new report task stays pending;
with the role back, both are taken up.
2. A node started with no task type at all gains outboundMta: a
waiting message is attempted and a report task runs.
On main the test fails at step 1 ("delivery attempted without
outboundMta"); with step 1 bypassed, step 2 fails (nothing picked the
message up in 20 s).
|
||
|
jcoffey-dev
|
08f29926d4 |
SQL queries time out; readiness follows the data store
Cluster rehearsal 3: with PostgreSQL paused (docker pause, so its kernel still answered TCP keepalives), requests on connections already checked out hung until it came back, and /healthz/ready stayed 200 through the outage. #41 bounded getting a connection, not using one. Client-side query limits (store::backend::query_timeout). Every operation on a PostgreSQL or MySQL connection now runs under a time limit. A server-side statement_timeout (or MySQL's MAX_EXECUTION_TIME, which covers SELECTs only) can't do this: the server that would enforce it is the one not answering. When an operation runs out, its connection is closed instead of pooled, since a query may still be in flight on it or a transaction open: deadpool's Object::take on PostgreSQL; Conn::disconnect on MySQL, which marks the connection closed before it sends anything, so the pool discards it even when the server never answers. - query, 2 minutes: reads, writes (the whole transaction with its retries), blobs, SQL lookups, search queries and indexing. These take milliseconds; two minutes leaves room for a large blob over a slow link and still ends a hang. - maintenance, 30 minutes: range deletes (account removal, purges), unindexing, purge_store, and creating tables and indexes at startup, which can legitimately run long in one statement. Their existing chunked fallback for server-side statement timeouts is unchanged. - iterate (exports, reindexing, maintenance scans) can run for hours, so the query limit bounds each wait for the database (preparing, the query starting, the next row) rather than the whole scan. The limits are fixed, like the pool timeouts; the DataStore schema has no field for them. Tests set them with Store::with_query_timeouts (test_mode only). Readiness. /healthz/ready answered 200 whenever a data store was configured. It now reads one key from the data store with a 2 s limit and reuses the answer for 2 s, so probes can't load the database; while one probe runs, others get the last answer. The first failed probe of an outage is logged. /healthz/live stays 200: restarting a node doesn't bring its database back, and an orchestrator restarting on failed liveness would restart every node at once. The container HEALTHCHECK already uses /healthz/live. Tests, store::pool_timeout (a proxy that stops forwarding while keeping connections open plays the paused database): - postgres_query_timeout, mysql_query_timeout (new): with four pooled connections open, a read, a scan and a write each fail with "Query timed out" 2.0 s after the pause (2 s test limit); once the proxy forwards again the store answers. With the limits set to an hour (upstream's behavior), the read was still waiting at the test's 20 s limit. - postgres_readiness (new, STORE=PostgreSql): a node's data store goes through the proxy; /healthz/ready is 200, 503 about 4 s after the pause while /healthz/live stays 200, and 200 again about 2 s after it ends. - postgres_pool_timeout, mysql_pool_timeout: pass as before. store::store_tests (PostgreSql, MySql, including the MariaDB statement timeout step) and store::task_locks (PostgreSql) pass; store::search_tests (PostgreSql) fails at the same ordering assertion (query.rs:684) as on main. |
||
|
jcoffey-dev
|
fcef4b1c3f |
Allowed IPs take the full settings reload after a write
write_reload_target sent AllowedIp writes to the blocked-IP reload, but
that reload rebuilds only BlockedIps. Allowed IPs are parsed into the
core's security settings (Security::parse), which only a full reload
rebuilds, so an AllowedIp write reported x:settingsReload applied: true
while the change wasn't live until the next full reload.
AllowedIp now maps to the full reload, like the other settings objects;
BlockedIp keeps its targeted reload.
system::auto_reload::settings_reload_tests now creates an allowed IP
over JMAP and checks that is_ip_allowed sees it with no ReloadSettings,
and that destroying it takes it out again. On main it fails ("allowed
IP not in the running settings").
|
||
|
jcoffey-dev
|
6e50ba25a9 |
SQL pools time out; task locks are a renewed five-minute lease
A 3-node rehearsal (PostgreSQL + NATS + Garage) found two ways a crash leaves work stuck: Pool hangs. The PostgreSQL pool (deadpool) was built with no timeouts, so a request waited for a free connection, and for one to be opened or recycled, for as long as it took: forever when the server stopped answering. MySQL's pool (mysql_async) has no wait timeout at all. - PostgreSQL: wait 30 s (or the store's timeout if longer), create the store's timeout or 15 s (it bounds the whole handshake, where tokio-postgres's connect_timeout covers only the TCP connect), recycle 10 s. The pool config is now always set, not only with poolMaxConnections. - MySQL: every connection is taken through MysqlStore::conn(), which gives up after 30 s. - Both: TCP keepalive after 60 s idle, so a server that vanished without closing the connection is noticed in minutes rather than the two-hour system default. The DataStore schema has no pool timeout settings, so these are fixed defaults; the store's own timeout bounds connecting on PostgreSQL. Task locks. A task lock lasted an hour, so after a hard crash the dead node's tasks waited up to an hour and five minutes. The lock is now a five-minute lease: while this node runs a task, the task manager renews its lock every third of the lifetime (InMemoryStore::renew_lock, a compare-and-set on the store backends and SET XX EX on Redis, which leaves a lock that already expired alone). A killed node's tasks run elsewhere within about five minutes plus the claim recheck. A task this node holds isn't handed to a worker again by the scan. store::pool_timeout (new): a local listener that accepts connections and never answers plays a hung server; a PostgreSQL store with a 2 s timeout returns an error in about 4 s, and a MySQL store in 30 s. Without the timeouts both wait for good. store::task_locks gains a task held for 1.5 lock lifetimes: its lease is still held, and released when the task ends. |
||
|
jcoffey-dev
|
2c684be5c9 |
Registry writes apply to the running settings without ReloadSettings
A 3-node rehearsal found that saving an MtaDeliverySchedule left it
unknown to the queue ("Queue strategy not found") until someone ran
x:Action ReloadSettings; only Directory and Authentication writes
reloaded (DIR-17). The admin UI has to remember a separate reload after
every save, and a script or API client that doesn't gets a server
running stale settings.
x:<Object>/set now reloads the running settings when it created,
updated or destroyed an object they are built from, and broadcasts the
same RegistryChange::Reload over the coordinator as ReloadSettings, so
every node applies it:
- Settings objects (MTA, spam filter, listeners, tracers, Sieve system
scripts, cluster roles, directories, ...: the object types the core,
telemetry, listener and directory builders read) get a full reload.
- Certificates, lookup stores and blocked/allowed IPs get their own
targeted reloads.
- Accounts, domains, roles and other data read as needed, stores (they
take a restart) and applications (their own reload action) get none.
Full reloads are coalesced: a write waits for a reload that started
after it was stored and joins one if it can, so a burst of writes, or
a request with many objects, costs one or two reloads, not one each.
The write itself is never undone. When the reload is refused (build
errors in objects that were working, the rule from the previous
commit), the set response says so in a new x:settingsReload field,
{"applied": false, "description": "Saved, but the running settings
were not reloaded. <object>: <error>"}; {"applied": true} otherwise.
The field is absent when the write needs no reload. The description
helper is shared with ReloadSettings' refusal.
Each reload sends the queue a ReloadSettings event, so the SMTP test
harness's read_event, try_read_event and assert_no_events now pass over
those; expect_reload_settings still waits for one.
system::auto_reload::settings_reload_tests (new): an MtaVirtualQueue
and an MtaDeliverySchedule created over JMAP are in the running
settings with no ReloadSettings, and gone once destroyed; eight
concurrent creates all land; a write whose reload fails is stored and
reported applied: false with the error; a domain write carries no
x:settingsReload. On main the new schedule is missing. The cluster
broadcast test (three nodes, PostgreSQL + NATS) now checks that every
node has a schedule created on node 0 without a reload.
|
||
|
jcoffey-dev
|
999ae12cc7 |
Settings reload: no DNS at build time, don't refuse over old failures
A 3-node rehearsal found every settings reload refused, cluster-wide,
because one node couldn't resolve the Pyzor server:
- PyzorConfig::parse resolved the host while building the settings and
made a failed lookup a build error. It now keeps the host and port and
resolves when a message is checked (an IP address is used as is, a
name is reused for five minutes, the lookup counts against the Pyzor
timeout). A failure there is a Pyzor error for that message.
- A milter's hostname was resolved the same way, with a blocking
to_socket_addrs in async code. An IP address is kept; a name is now
resolved on each connection.
Other build-time I/O is already non-fatal: directories that can't
connect become unavailable with a warning (DIR-21), and the AI model
locality check only warns.
reload_registry swapped the core only when the whole build was free of
errors, while boot runs with whatever built. One failing object thus
refused every later reload, and the running settings went stale. Now a
reload is refused only for errors in objects that built when the
running settings were built (at boot or by the last applied reload):
applying it would lose those. Objects that already failed then are
missing from the running settings anyway, as at boot, so their errors
are logged and returned as known_errors but don't hold the reload back.
Refusing on new errors keeps a bad edit from taking a working object
out of service; the admin gets the error instead.
ReloadSettings now says "Settings were not reloaded." and names the
object and its error ("Tracer with id ...: Only one console tracer is
allowed"), with a count of any further errors. A refused reload after a
directory change logs its errors too.
system::reload::reload_tests (new): with Pyzor enabled on an
unresolvable host, ReloadSettings succeeds (on main it fails with
"Invalid address: failed to lookup address information"); an IP host
needs no lookup; a new build error refuses the reload, names the object
and leaves the running settings unchanged; the same error, once known
from the running settings' build, no longer blocks; once fixed, a new
error there blocks again. smtp::inbound::milter's session test now
names its milter "localhost", so the connect-time lookup is exercised.
|
||
|
jcoffey-dev
|
7c80a12d75 |
Task manager: release task locks on stop, recheck claims held elsewhere
A cluster rehearsal (PostgreSQL + NATS) left index tasks pending well past the one-hour task lock after the node that claimed them was stopped or killed. The exact cause there isn't confirmed; this closes every path found in the task manager that stretches a takeover past the lock, or keeps a task claimed without running it: - A graceful stop never released the locks it held, so every task the node had claimed stayed blocked for an hour. The server now tracks the locks it holds (common::ipc::TaskLocks) and, once the shutdown signal arrives, stops claiming and releases them before exiting. - A node that failed to claim a task (another node held it) set its own local hold for a full lock lifetime from that scan. If the holder claimed it just after the scan began, or ran on a clock ahead, that hold ran out a moment before the lock did and was set for another hour: two hours in all. Such claims are now tried again every five minutes (a twelfth of the lock lifetime), and the task manager wakes up for them: before, a node without a coordinator could sleep up to five minutes past the recheck, or until something else woke it. - A worker that panicked took its task type down on that node for good, while the scan kept claiming that type's tasks and failing to hand them over, re-taking each lock as it expired and so starving every other node of them. Each batch now runs on a task of its own; a panic is logged, the batch's locks are released and the worker carries on. A failed hand-over releases the lock too. - A claimed task the worker couldn't read, or found gone, kept its lock for the hour. It is released. - An IndexDocument task for a file (not indexed) returned no result, which shifted every later result in the batch onto the wrong task in update_tasks. It returns Ignored. Nothing queues such a task today. The lock lifetime stays one hour; it now lives per server so the tests can shorten it. store::task_locks::task_lock_tests plays a second node by writing its locks straight into the in-memory store: tasks it claimed and abandoned run here once its locks expire, including locks that outlive this node's view of them, and a graceful stop hands this node's locks back at once and claims nothing more. It passes on RocksDB, SQLite and PostgreSQL. With the old recheck it fails. |
||
|
jcoffey-dev
|
212cd77cd3 |
Export/import: keep archived items, spam samples and the spam model
--export skipped three things, so a move from one database to another
(RocksDB to PostgreSQL, say) lost them without a word:
- archived items (subspace j), the records behind undelete;
- spam training samples (subspace w);
- the trained spam classifier and its trainer state, blobs stored under
fixed names that no blob link points at, so the walk over links never
reached them.
j and w now travel with the registry family, where their indexes and id
counters already were, so EXPORT_TYPES=registry keeps them consistent.
The two named blobs travel with the blob family. The file format is
unchanged and import reads any subspace it is given, so an export made
by an older binary still imports.
The full-text index (subspace z) stays out, on purpose. It belongs to one
search backend: PostgreSQL and MySQL index into their own tables and have
no z table at all, and external engines keep the index themselves. So
--import now returns the subspaces it wrote, and boot queues the
reindexAccounts and reindexTelemetry store maintenance tasks, the same
ones an administrator can queue by hand, to rebuild the index for
whichever search store the server runs with once it starts.
The round trip also turned up a loss in import itself: the SQL stores
add a negative amount with an UPDATE, which does nothing to a row that
isn't there yet, so every negative counter or quota vanished on import
into PostgreSQL, MySQL or SQLite. Import now creates the row first.
The in-memory subspaces (m, y) stay out: rate limits, locks, greylisting,
ACME challenge tokens and OAuth codes, all short-lived. Issued
certificates are registry objects and travel.
The store test now writes archived items, spam samples, directory
entries, the fork's own subspace and the named blobs, checks they come
back in place, then imports the same export into a fresh store of the
other local backend (RocksDB to SQLite, or SQLite to RocksDB), compares
it key for key and counter for counter, and checks the queued reindex.
It fails on the old export code ("Subspace j was not exported").
--help now says what an export holds.
|
||
|
jcoffey-dev
|
17426f6d60 |
Bundle the spam filter rules with the server
The server fetched upstream's latest published rules from GitHub at run time: a version nobody here tested, code-like expressions from an account we don't control, and the upstream name as a default in the admin form. The published rules of spam-filter v3.0.2 are now embedded (resources/spam-filter/, MIT, in THIRD-PARTY.md) and used whenever no other source is configured. An empty setting and upstream's old default both mean the bundled rules, so existing installs switch without a settings change; the URL stays an operator override (https:// or file://). The schema default is dropped and its description says what empty means, and the strip's rename pass does the same to each import. Rules load on first boot as before, and again whenever the bundled version differs from the last one loaded, which only adds missing rules and tags. That brings the AI classifier's LLM_* scores to installs that predate them: production has none today. upstream-watch now also opens an issue when spam-filter publishes a newer release; resources/spam-filter/README.md says how to take it. The antispam test now runs on the bundled rules, the path production takes; SPAM_RULES_URL tests another set. Unit tests cover the URL handling and that the bundled rules parse and score the AI tags as the AI spec says. |
||
|
jcoffey-dev
|
a993f9ab01 |
Write the brand in lowercase where people see it
The name is inbuxa, lowercase, like the wordmark; INBUXA reads as an acronym. The admin and webmail already changed. Here that's everything the server shows people: the brand macro behind the protocol greetings, the HTTP and SCIM realms, the startup banner and the calendar and contact PRODID; the first-party OAuth client descriptions; the legacy-protocol refusals; the default calendar and address book names and the SMTP greeting default, in the code and the schema served to the admin (checksum regenerated); startup and shutdown events; the User-Agent; the sign-in and RSVP pages; the service units; the OpenAPI realm; the crate descriptions and the README, where it's set in bold. Identifiers that are uppercase for their own reasons stay: INBUXA_* settings, SUBSPACE_INBUXA. So do code comments and the AGPL 5(a) notice lines. Tests follow: the IMAP ID name, the default collection names, the PRODID in the iTIP fixtures and the CalDAV free-busy expectations, and the e2e legacy-protocol refusals. The webdav, imap and jmap suites pass, so do the unit tests of every crate touched, and 73 of 75 SMTP tests; of the other two, antispam fails on main too, and queue_retry is a timing flake that passes on its own. |
||
|
jcoffey-dev
|
cc6f1eb298 |
Rename the identifiers that carried the upstream name
Everything clients, users and operators meet now carries the fork's name, with no aliases (SPEC.md §2.4, changed here from "protocol identifiers stay"): - JMAP: upstream's registry capability is urn:inbuxa:jmap:registry, beside the fork's own urn:inbuxa:jmap. - WebDAV lock and sync tokens are urn:inbuxa:dav*; clients resync once. - Sieve: vnd.inbuxa.while and vnd.inbuxa.expressions. sieve-rs spells these into its compiler, so it's vendored (vendor/sieve-rs, 0.7.3) and patched in; a unit test fails if Cargo.lock ever moves past the vendored copy. The trusted runtime now names itself too, rather than answering sieve-rs's default. - The web interface's OAuth client is inbuxa-webui. On every start the old stalwart-webui client is removed and any application naming it is moved over. - The spam filter's blobs are INBUXA_SPAM_*; every start moves any left under the old keys, so a trained model survives. - SQL stores and log files default to inbuxa, in the code and in the schema served to the admin (checksum regenerated). - Settings are INBUXA_* only. A STALWART_* variable that's set where its INBUXA_* one isn't stops the server at startup, naming it. - The version-upgrade messages link docs.inbuxa.org's migration page, and the OpenAPI description, smtp crate metadata and web-push test fixtures lose the name. Kept on purpose, allowlisted with reasons: the OAuth key-derivation contexts (renaming them would end every session and invalidate every sealed client id) and the hashed application prefix. Also fixes a latent start-up failure: ensure_client updated an existing first-party client with a revision of 0, which the registry's assertion never matches, so adding a redirect URI or changing the webmail secret failed start-up. And the principal session test now expects legacyProtocols (C-1, added 2026-09-21), which it had missed. Tested: the server builds without warnings; common's 106 unit tests, including the vendoring check; a new integration test for the two start-up migrations; and the webdav, jmap, imap and SMTP Sieve suites. |
||
|
jcoffey-dev
|
c240946248 |
Drop upstream's issuer-routing test and an import it left unused
tests/src/directory/issuer.rs, new in v0.16.23, tests routing a bearer token to a directory by its issuer. That routing is Enterprise-only upstream (the body of get_directory_for_issuer), and the fork doesn't build it: a token naming no address gets the server default (DIR-2). The test also calls a helper from upstream's Enterprise-only OIDC test, so it can't compile here. mta.rs imported types::id::Id for code inside an Enterprise snippet; the stripped tree leaves it unused, upstream's as well as ours. |
||
|
jcoffey-dev
|
ee4988e00d |
Mark eight more changed files (AGPL section 5(a))
These upstream files were changed after the fork marked the files it had modified, and never got the notice: six by the listener and schema-cache work on 2026-09-20, two by the name check. Found by diffing against the upstream snapshot branch, as before. |
||
|
jcoffey-dev
|
b2ded0a776 |
Merge upstream v0.16.23
Five conflicts, resolved: - crates/common/src/auth/authentication.rs: upstream's get_directory_for_token and JwtClaims replace extract_jwt_domain; the per-domain directory code (DIR-1, DIR-5 to DIR-7) is kept, and the token lookup routes through it. The release's one new Enterprise snippet was the body of get_directory_for_issuer, which stays returning None: a token naming no address gets the server default, as DIR-2 specifies and as v0.16.22 did. - crates/common/src/manager/application.rs: upstream's rewrite of the tests, with the temp directory names renamed again, and the 5(a) notice the name-purge change should have added. - crates/common/src/network/mta.rs: both sides' imports. - crates/main/Cargo.toml: the AGPL-only license kept, version 0.16.23. - Cargo.lock: upstream's, with the fork's crates added by Cargo. |
||
|
jcoffey-dev
|
3a272096c0 |
Import upstream v0.16.23, stripped
trivy / Check (pull_request) Canceled after 0s
Upstream commit: 9d1c75ab68435e4417337f768291e5f947686203 Enterprise-only files removed or emptied: 63 Enterprise-only snippets removed: 118 in 50 files Dangling module declarations removed: 5 Edits turning enterprise off: 25 Third-party code: 14 files, 0 not in THIRD-PARTY.md Verification: clean One snippet more than v0.16.22, in crates/common/src/auth/authentication.rs (3, was 2). |
||
|
jcoffey-dev
|
7f14992e81 |
Keep the upstream project's name out of user-visible strings
ci / build (pull_request) Successful in 3m46s
The first-party application descriptions and the telemetry service name and instrumentation scope are shown to operators, and the unpacked-application temp directory carried the name too. Left alone deliberately: the OAuth key-derivation contexts (renaming them would invalidate every sealed token and client id), the migration defaults that read an upstream installation, links to upstream's upgrade guide, the wire-protocol identifiers, and upstream's own license and templates. |
||
|
jcoffey-dev
|
3f40b36032 |
The switch knows who still uses legacy mail apps (LP-15, server)
The impact panel's data. Every successful sign-in over IMAP, POP3,
ManageSieve or SMTP AUTH records, per account and per protocol, one
timestamp -- nothing else: no address, no IP, no client. It is written at
most once an hour per account and protocol, so a mail app polling every
minute costs a read per sign-in and a write an hour. A record that can't be
written is logged and the sign-in goes ahead.
Both switches serve it as a read-only property, recentLegacyUse, as
wouldClose serves the confirmation: a list of {accountId, name, protocol,
lastUsedAt} for sign-ins in the last 30 days, most recent first.
inbuxa:ProtocolPolicy lists every account; inbuxa:TenantProtocolPolicy
lists only its tenant's own (MT-1). Accounts since deleted are left out. It
is computed only when the property is asked for.
The recording sits where the tenant check already runs once the account is
known, which becomes admit_legacy_session: refuse if the account's tenant
has legacy protocols off, otherwise record. A refused sign-in is never
recorded.
The spec leaves the interface to the implementation; a property on each
switch keeps the panel's data behind the same permission as the switch
itself, with no new object.
Unit tests hold the 30-day window to acceptance test 11 (three days ago
listed, forty not), the hourly throttle and the keys. The e2e proves on a
running server that the admin's IMAP and submission sign-ins are listed
with their time, that a second sign-in within the hour isn't written again,
and that a tenant's list holds its own user and nobody outside the tenant.
All 70 checks pass.
|
||
|
jcoffey-dev
|
cd99037ca4 |
The session says whether legacy protocols are off for the account
The urn:inbuxa:jmap capability on the signed-in principal's own account gains legacyProtocols: "enabled" or "disabled", the stricter of the server's switch and the account's tenant's (legacy-protocols spec, Interfaces). It is what the webmail needs to tell someone why their phone's mail app won't connect (LP-19), and it closes acceptance test 13. contract.md's C-1 gains the line. It is an optional field added, which C-3 says doesn't bump the contract version. tests/e2e/legacy_protocols.py reads it back from the session on a running server: enabled for the tenant's user while both switches are on, disabled once its tenant turns legacy protocols off while an account outside the tenant still reads enabled, disabled for everyone while the server switch is off, and enabled again at the end. All 67 checks pass. |
||
|
jcoffey-dev
|
b65afb66f9 |
A tenant can turn legacy protocols off for itself (LP-9 to LP-14a)
The tenant switch. A tenant's administrator turns legacy mail protocols off for its own tenant, and from then on sign-in over IMAP, POP3, ManageSieve and SMTP AUTH is refused for every address on the tenant's domains, while every other domain on the server carries on. No port closes, since other tenants share them (LP-13): it is one stored fact per tenant, read at sign-in and when client configuration is answered. inbuxa:TenantProtocolPolicy/get and /set, one per tenant, id the tenant's: - Inside a tenant, a principal reaches only its own tenant's switch (MT-1): /get with no ids answers with it, another tenant's is notFound and can't be changed. At server level /get with no ids lists every tenant's. - Turning it off is always allowed. Turning it back on is refused with forbidden, naming inbuxa:ProtocolPolicy, while the server has legacy protocols off (LP-9). - A change raises security.legacy-protocols-changed with policy = tenant, the tenant's id, the new value and who made it (LP-14). - It takes sysDomainGet and sysDomainUpdate, not the two new permissions the spec names. The switch governs sign-in on the tenant's domains, so whoever manages those domains may turn it -- and the default Tenant Administrator role already holds both, where new permissions would reach no role already stored on a server (MT-12's note), leaving today's tenant administrators without the switch until someone edited their role by hand. The same trade inbuxa:AiLimits and inbuxa:ProtocolPolicy made. /query is not built yet; /get with no ids covers listing. Sign-in (LP-10 to LP-12). Before the credentials are looked at, the name given is resolved to its domain and the domain to its tenant, so a real account and a made-up address on the domain get the same refusal, with a right password or a wrong one, counted as no failed sign-in (LP-11). The words are the spec's: "Your organization allows only INBUXA webmail and JMAP apps...", in each protocol's form. A bearer token needn't name an account, so after authentication the account's own tenant is checked too; a token that named nobody can't slip past. The refusal carries policy = tenant and the domain, not the tenant's id: IMAP answers a command's tag from the Id key, so an error holding one was sent under the wrong tag and the mail app hung waiting for its reply. The first live run found that; a unit test now holds the refusal to it. Client configuration (LP-14a). Autoconfig, autodiscover, PACC and the suggested DNS records now ask whether legacy services are off for the domain being answered for -- the server's switch, or the domain's tenant's -- so a tenant's domains stop offering IMAP, POP3 and submission while others still do. tests/e2e/legacy_protocols.py builds a tenant with its own domain, a user and a tenant administrator, and a second tenant, and proves on a running server: the admin sees and changes only its own tenant's switch (test 10); turning it off is an event (test 14); the tenant's user is refused over IMAP with the right password and a wrong one, a made-up address on the domain the same (tests 6, 7); POP3 and submission refuse in their own forms and JMAP still works (test 8); an account on another domain signs in normally (test 6); autoconfig drops IMAP for the tenant's domain only; with the server off, the tenant can't turn it back on (test 9); and once back on, the user signs in again. All 62 checks pass. |
||
|
jcoffey-dev
|
64cddc9246 |
The switch reports every change as an event (LP-8)
Turning legacy mail protocols off or back on raises security.legacy-protocols-changed (id 643, info level, also in the packaged schema), with the scope (policy = server), the new value, who made the change (accountId), whether listeners closed or reopened (details), which ones (listenerId), and -- only when a listener could not be put back -- which and why (reason). It is raised in Server::set_protocol_policy rather than by the JMAP method, so whatever turns the switch is reported. A /set that changes nothing -- the switch already where it was asked to be, nothing to close or reopen -- is not a change and raises nothing. The event is never an error, but jmap's exhaustive map from security events to HTTP errors has to name it; it joins the other two that can't occur there. rustfmt now also wraps LP-6's two over-long lines in enums_impl.rs, which it flagged along with this change's. tests/e2e/legacy_protocols.py now gives the server a stdout tracer and reads events from the container's log: turning the switch off is exactly one event naming the scope, value, author and listeners closed; setting it off again raises none; turning it on is one event naming the listeners reopened. It also proves LP-6's side: seven refused submission sign-ins are seven auth.legacy-protocol-refused events, and there is no auth.failed or auth.too-many-attempts among them. All checks pass. |
||
|
jcoffey-dev
|
4b585905d7 |
Nothing advertises the legacy protocols while they are off (LP-7)
While the switch is off, the answers that tell a mail app where to connect stop offering what the switch closed, so a new phone or desktop app is not sent to a port that is shut or a sign-in that will be refused: - Thunderbird-style autoconfig (/mail/config-v1.1.xml and its other paths) and Outlook autodiscover leave out IMAP, POP3 and SMTP submission. - PACC (/.well-known/user-agent-configuration.json) offers JMAP, CalDAV, CardDAV and WebDAV, and no IMAP, POP3, SMTP or ManageSieve. The document is rendered once per configuration load, so the JMAP-only version is rendered beside it and chosen per request; the _ua-auto-config digest in the suggested zone follows, since it hashes the same document. - The suggested zone publishes _imap, _imaps, _pop3, _pop3s, _submission and _submissions with target "." -- "not offered", RFC 6186 section 3.4 -- the spec's decision, rather than dropping them: a client that looks is told, and an automatically managed zone replaces the old records instead of leaving them behind. - It also drops the TLSA records for ports 993 and 995. A TLS pin for a port the switch has closed advertises a service that is not there. Submission's 465 keeps its record: the SMTP lock keeps that port open. The switch is read per answer, as sign-in reads it, so every node agrees the moment it turns. Inbound mail, MX records and the JMAP, CalDAV and CardDAV answers are untouched. tests/e2e/legacy_protocols.py checks all four on a running server: with the switch on they offer IMAP, POP3 and SMTP (the control); while it is off they offer none of them and every legacy SRV name has target "."; and once it is back on, autoconfig and the zone read as they did before. All checks pass. |
||
|
jcoffey-dev
|
04252000da |
Legacy sign-in is refused while the switch is off (LP-6)
The second lock. While legacy mail protocols are off, a sign-in over IMAP,
POP3, ManageSieve or SMTP AUTH is refused for every account, so a listener
that exists by mistake -- or submission, which the SMTP lock keeps open --
still lets nobody in.
The check sits at the top of each protocol's sign-in, before the
credentials are looked at. So the answer is the same for a right password,
a wrong one and an account that doesn't exist; it isn't auth.failed, so it
counts nothing against the account and never feeds the auto-ban; and the
session stays open, since the mail app is being told, not thrown off.
Mail apps read the spec's words (LP-12, at server scope):
IMAP NO [ALERT] This server allows only INBUXA webmail and JMAP
apps. This mail app can't sign in.
POP3 -ERR [AUTH] ...the same...
ManageSieve NO "This server allows only INBUXA webmail and JMAP apps."
SMTP 535 5.7.0 This server allows only INBUXA webmail and JMAP
apps. This mail app can't send.
SMTP AUTH is refused on every SMTP listener, port 25 included: only mail
apps authenticate, so inbound delivery is untouched. LMTP is left alone.
The policy is read from the store on each sign-in rather than cached, so
every node of a cluster answers the same the moment the switch turns.
Each refusal raises a new event, auth.legacy-protocol-refused (id 642, info
level, also in the packaged schema), with the protocol as source, the
policy's scope and the domain -- never the account. The session adds the
listener and remote IP.
tests/e2e/legacy_protocols.py now also proves, on a running server: a
normal IMAP and submission sign-in works with the switch on, before and
after; while off, submission refuses the right password and six wrong ones
with the same words and without hanging up; and an IMAP listener created by
mistake while off refuses the right password, a wrong one and an account
that doesn't exist. All 33 checks pass. SMTP sign-ins in the script wait
out a second first: every connection arrives from Docker's gateway, and the
stock inbound throttle takes five a second from one IP.
|
||
|
jcoffey-dev
|
3f689529c7 |
A listener put back has to be bound, or it never comes up
Found while setting up the live check, which is the only place it could have shown: every unit test passes without it. spawn_restored_listeners re-parsed the listeners and spawned them, but never bound their sockets. Binding is not part of parsing -- it happens in bind_and_drop_priv, once, at startup -- so listen() would have failed on an unbound socket and the port would have stayed shut while the policy recorded it as reopened. LP-5 would have been a promise the server did not keep, and the operator's only clue a log line. bind() is now split out of bind_and_drop_priv and called on its own here. It cannot be the whole of bind_and_drop_priv, because that also drops privileges, which must happen once at startup and never again. That split has a consequence worth stating: a listener on a port below 1024 cannot be bound again once privileges are gone. Ports 143 and 110 are the realistic cases. Rather than leave such a listener parsed, spawned and silently dead, the bind errors are read back and those listeners are reported as needing a restart -- which is the "cannot be recreated" case LP-5 already anticipated, and it stays saved for another try. Re-parsing is also narrowed to the listeners being restored, so putting one back cannot bind a port another listener already holds. |
||
|
jcoffey-dev
|
3b29ca3571 |
The switch now reaches the running server
The join: the policy decides, features owns the listener objects, ListenerControl owns the running sockets, and only Server has both. Server::set_protocol_policy is what a click performs. It applies the locks to what was asked before storing anything (LP-21), so what is recorded is what the server allows. Closing removes each listener object and then stops its socket; opening puts the object back and then spawns it. The order is the point in both directions -- a socket stopped while its object remains returns on the next restart, and a socket spawned before its object exists has nothing to come back to. saved_listeners is carried over from the stored policy rather than taken from the request. A client never sets it, and a /set that omitted it would otherwise lose the listeners still waiting to come back. Putting a listener back has to bind a fresh socket, so it re-parses from the registry -- the objects are already back by then -- rather than trying to revive the saved one. Only main knows which session manager a protocol wants, so it leaves a spawner behind at startup and spawn_listener is now shared between that and the initial spawn. Without a spawner a restored listener is reported as pending a restart rather than promised, which is what the test servers will see. A listener that cannot be put back does not stop the others and stays saved for another try (LP-5). Still nothing an operator can reach: no JMAP method calls this yet, and no sign-in is refused. What it does do is close and reopen a port on a running server, which is the part that did not exist this morning. |
||
|
jcoffey-dev
|
fee6b74e79 |
Listeners can be stopped one at a time, which LP-2 needs
The legacy-protocols switch has to close the IMAP, POP3 and ManageSieve ports and leave everything else accepting. The server could not do that. Two findings from the source, both now recorded in the spec. A settings reload never closes a port: cache/reload.rs parses the listeners only to collect configuration errors and drops the result, and sockets are bound once at startup through init.servers.spawn in main.rs. And there is only one shutdown signal -- Listeners::spawn makes a single watch channel and hands every listener a clone -- so the one thing the server could do was stop all of them at once, port 25 included. That answers the spec's open question 1, and the answer was neither of the two it offered. So each listener gets its own channel. ListenerControl holds the sending ends keyed by listener id; firing one breaks that accept loop, which drops its TcpListener and closes the socket. The accept loop itself is unchanged -- it already did the right thing, it just had no way to be told about one listener. stop_matching takes a predicate and a keep list, because the inbound listener shares its protocol with submission and telling them apart is the caller's job (LP-3), not this registry's. spawn_with_control is a second method rather than a change to spawn. The registry owns the senders, so a dropped registry would stop every listener at once; the four test callers pass no registry and keep the old shared channel exactly as it was. Whole-server shutdown now fires the per-listener channels too, since the returned sender no longer reaches them. No policy, no JMAP and no screen yet: this is only the mechanism, with seven tests over stopping one, stopping many, sparing port 25 and sparing submission. It closes no port on its own, and it does not touch the host's firewall or any port-forward -- that is LP-20, and stays the operator's. |
||
|
jcoffey-dev
|
6a53d47106 |
Mark the files this fork changed (AGPL section 5(a))
The AGPL asks a modified version to carry prominent notices saying it was modified, and giving a date. Publishing the source is the conveyance that asks for it, so it wants doing before the repository is public rather than at the release. Every upstream file the fork changed now says so in its header, beneath the notice it came with: 164 files, found by diffing against the upstream snapshot branch rather than by guessing, so the list is what actually differs. Files the fork wrote itself already carry their own copyright and need nothing. Upstream's notices are untouched, which its licence requires and which was already true. The README says the same thing in prose, since the obligation is on the work as a whole and not only its Rust files. Builds unchanged: the server and the test binary both compile. |
||
|
jcoffey-dev
|
29d9263071 |
One edition: the last enterprise gates come out of shared code (SPEC 2.3)
Every one of the 14 was `#[cfg(not(feature = "enterprise"))]` on the arm the fork always compiles: the Enterprise arms went with the import, and nothing turns the feature on. Removing the attribute leaves the same code, now unconditional, in 11 files. Two of them looked like behavior worth checking before touching: the `validate_tenant_quota` stub that always passes, and the refusal to cancel a pending DestroyAccount task. The stub is vestigial — the rebuilt multi-tenancy enforces quotas in `crates/features/src/tenancy/quota.rs` for those objects and more — and the refusal is undelete's open question, which this change leaves exactly as it was. The binary builds with no new warnings, and `system_tests` and `jmap_tests`, which cover the touched registry, task-manager and auth paths, both pass. The feature definitions stay in the manifests, inert: taking them out would widen every sync's diff for nothing. |
||
|
jcoffey-dev
|
309835be1d |
ACME: post a challenge once, then poll the authorization
The renewal loop re-posted the challenge every time it polled and found the authorization still pending. RFC 8555 section 7.5.1 has the client post a challenge once to say it is ready and then poll; a server that has already moved the challenge to "processing" refuses a second post, and pebble answers 400 malformed, which failed the whole renewal. Verified against pebble: the 400s are gone and the client polls. The suite still can't finish on this machine, because pebble never reaches the test server to validate the challenge; that path is the environment, and the runbook now says so. |
||
|
jcoffey-dev
|
5cc28784df |
SCIM: turning a domain's SCIM authority off takes effect at once (SCIM-60)
`allowScimProvisioning` is cached with the domain as DOMAIN_FLAG_SCIM, but it wasn't among the fields whose change drops the cached entry, so flipping the flag changed nothing until something else evicted the domain. SCIM-60 says the change takes effect without a restart. Found by the two acceptance checks that were written but never called from the driver, so neither had ever run: `authority` (SCIM-58 to SCIM-60, through `synchronize_account` itself) and `rate_limits` (SCIM-14). Both are wired in now, and `scim_tests` passes with them. |
||
|
jcoffey-dev
|
0755fad51e |
Scale-out storage: IMAP CONDSTORE raises the mark, and the two-node check (ST-7, test 11)
A FETCH with CHANGEDSINCE presents its mod-sequence to the read scope, so a replica must have that change before it answers, as a JMAP sinceState already did. replica_cluster_tests covers test 11: with the replica's replay paused, a write on one node is read back through a second store with its own marks, sharing through Redis. Composite stores nest store futures deeply enough to pass rustc's default query depth once both postgres and redis are compiled in, so the server crates raise their recursion limit. |
||
|
jcoffey-dev
|
1518c69033 |
Scale-out storage: PostgreSQL and MySQL read replicas (ST-5 to ST-15)
A data store with readReplicas becomes a replicated store. Writes, operator-written SQL and everything outside a read scope go to the primary. JMAP reads before a request's first write, IMAP LIST, STATUS, SEARCH, SORT and FETCH, POP3 RETR and TOP, DAV GET, PROPFIND and REPORT, and blob downloads run in a read scope. Only account data (properties, indexes, change logs, counters, ACLs, blobs, the search index) is read from a replica; the registry, in-memory values, the task queue and the rest stay on the primary. In a scope, the first read picks a replica round-robin among those up and under the lag limit, and only if it has every change this node has written or heard of for the scope's accounts: marks come from write results, the cluster's state-change broadcasts, a sinceState the client presents, and, with more than one node, Redis. A write inside the scope sends the rest of it to the primary. A miss on a replica is looked up on the primary, and a replica error retries the read there and marks the replica down. Each node samples lag every second (WAL positions on PostgreSQL; GTID sets or Seconds_Behind_Source on MySQL), stops reading from a replica over 5 s and starts again under 2.5 s, and probes a down replica every 10 s. At startup a replica is left out if it's the primary, isn't read-only, applies out of commit order, or doesn't show a marker written to the primary within six tries. replica_tests (postgres, STORE=PostgreSqlReplicated) runs a primary and a streaming hot standby in containers: tests 9, 10, 12, 13, 14 and 15 pass. |
||
|
jcoffey-dev
|
f72e3bb85c |
Per-domain directories: each domain signs in against its own directory (DIR-1 to DIR-15)
The two lookups every caller uses now honor Domain.directoryId, then the server default, then the internal directory, so sign-in, bearer routing, recipient lookup, discovery, the PACC record and the refusal of password changes on external accounts all follow the domain. A directoryId, or a server default, naming a directory that doesn't exist is unavailable, never the internal directory. A directory speaks only for the domains it serves: an account it returns on another directory's domain is refused, for sign-in and recipients alike, and aliases and group claims on such domains are dropped with a warning. A bearer token must belong to the user the client names, or the name must be one of its aliases with alias sign-in allowed. Accounts and groups that sync creates pass the tenant checks, limits included. |
||
|
jcoffey-dev
|
3ffaee0695 |
SCIM: a suspended account's open sessions end, and no credential it holds still works (SCIM-52)
The push router records which account each subscription belongs to, and a new Revoke event drops every subscription the account holds, so its IMAP IDLE, JMAP event streams and WebSockets on this node end. Other users' subscriptions to its shared mailboxes stay. Test 28 now checks an open IDLE is ended, and that the account's password over HTTP and its own API key, both already cached, are refused on the next request. |
||
|
jcoffey-dev
|
940b42f3b4 |
SCIM: just-in-time directory sync is read-only on domains open to SCIM (SCIM-58 to SCIM-60)
On such a domain, sign-in sync creates no account (the person gets an ordinary authentication failure), changes nothing on an existing one, and creates no group from a groups claim. Without the flag sync works as before, and turning it off hands accounts back to sync with no restart. Checked through synchronize_account itself; acceptance test 5 does the same over OIDC once per-domain directories exist. |
||
|
jcoffey-dev
|
0ca26070d7 |
SCIM: users, groups, queries, PATCH, Bulk and cursors at /scim/v2, over x:Account (SCIM-1 to SCIM-57)
Every SCIM operation becomes the x:Account get, query or set JMAP makes, as the service principal, so permissions, tenant scope and limits, address uniqueness and account destruction are enforced in one place. Discovery is anonymous; everything else takes an API key as a bearer token and nothing else. Domains open to SCIM carry a flag in the domain cache. Filters take eq and and, answered from the account indexes, with unindexed attributes checked on at most 200 candidates. Cursors are stateless, HMAC-sealed under the server key. PATCH applies to the resource in memory and saves it as a PUT, so it is all or nothing. Groups get an address from their display name on the principal's domain; membership is written on each user. Every write emits one of five new scim.* events (ids 637 to 641), also added to the packaged schema. The helpers the surviving SCIM suites import are rebuilt from the spec; scim_tests runs the new acceptance suite and the surviving tenant isolation suite, and both pass. |