80d6c09c5933492816d5d020fb1c127dd212f1fe
14
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 jcoffey-dev
|
b2453d066b |
Merge upstream v0.16.24
Eight conflicted files resolved, plus the lock file and the schema:
- crates/services/src/task_manager/spam_classifier.rs: upstream's rules
update now replaces existing rules, DNSBL servers, lookups and file
extensions, keeping only whether each is on. Taken, with one difference:
an object an admin edited is kept as it is. Every object an update writes
is fingerprinted (content without `enable`, SHA-256, stored under
SUBSPACE_INBUXA "Sf"), and only one that still matches is replaced.
Scores are never replaced, as upstream has it. The AU-1.10 summary record
now names what was added, replaced and kept, and the bundled rules are
marked applied only when the update fully succeeded, so a failure runs
again on the next start. The marker becomes "3.0.2+2", which runs the
update once on upgrade to fingerprint every rule still as bundled.
- crates/common/src/network/autoconfig/autodiscover.rs: upstream's rewrite
(implicit TLS first, labeled SSL), with the per-protocol switches (LP-7,
LP-14a) passed in as a filter.
- crates/store/src/backend/mysql/{search,write}.rs: upstream's chunked
deletes (no unbounded first DELETE, stop on a short chunk, halve the
chunk on the new chunk-too-large errors) inside the fork's query timeout.
- crates/smtp/src/lib.rs: the fork's queue spawn kept. It already fixed the
stall upstream fixes here (a node without outboundMta stops accepting
mail at about 1024 queued messages), and follows role changes live.
- crates/jmap/src/registry/mapping/bootstrap.rs: the log path stays
/var/log/inbuxa/; upstream's PowerDNS mapping taken.
- crates/main/Cargo.toml: the AGPL-only license kept, version 0.16.24.
- tests/src/jmap/principal/get.rs: the fork's capabilities kept.
- resources/schema/schema.json.gz: merged as JSON; upstream relabeled the
vendor Sieve extensions "(Stalwart)", kept as "(vnd.inbuxa)".
- Cargo.lock: upstream's, with the fork's crates added by Cargo.
Also:
- tests/src/smtp/inbound/spam_rules_kept.rs: an edited rule survives an
update, an unedited one is updated, rules from before fingerprints are
handled, and the audit summary says so. Upstream's own spam_rules test
passes unchanged.
- tests/src/smtp/reporting/reschedule.rs moves to port 19058; upstream's
new spam_rules test took 19057.
- tools/fork/renames.py renames the "(Stalwart)" labels and the default
log path, so neither conflicts again.
- tools/fork/notice-check.py compares against the newest snapshot in the
checked-out history instead of the upstream branch head, so moving the
branch no longer fails other open pull requests.
- tests/src/directory/issuer.rs (since v0.16.23) stays out, and is on the
build check's known list: it tests issuer-based directory routing, which
the fork doesn't have (DIR-2).
- Strip report: docs/fork/strip-reports/v0.16.24.{md,json}.
|
||
|
jcoffey-dev
|
f59b084ce5 |
Import upstream v0.16.24, stripped
Upstream commit: af37a234981722493b74623a983581691d2b70b6 Enterprise-only files removed or emptied: 63 Enterprise-only snippets removed: 118 in 50 files Dangling module declarations removed: 5 Edits turning enterprise off: 25 Third-party code: 14 files, 0 not in THIRD-PARTY.md Renamed identifiers: 62 in 18 files Verification: clean The same Enterprise footprint as v0.16.23. The build check fails only on tests/src/directory/issuer.rs, unchanged since v0.16.23: it calls a helper from upstream's Enterprise-only OIDC test, and tests issuer-based directory routing, an Enterprise feature. main has never carried it. |
||
|
jcoffey-dev
|
68dd749291 |
Export what a legal hold keeps as a ZIP (LH-12)
inbuxa:HoldExport/set takes a hold, optionally some of the accounts it covers, and a reason; the collection runs in the background and get says when it's ready. The ZIP has, per account, mail as .eml under its folders, calendars as .ics, contacts as .vcf, files as stored, and the archived items the hold keeps under archived/; a manifest.csv gives each entry's account, kind, folder, date, whether it was archived, size and SHA-256, and manifest.sha256 hashes the manifest. Accounts the hold doesn't cover are left out, and items outside its date range are too: live mail by arrival, events by start, and archived items the same way, so an export doesn't carry deleted items that only another hold keeps. The finished file is a blob of whoever started the export, so only they download it, and it lasts as long as any upload (uploadTtl). Exports are records under the hold (SUBSPACE_INBUXA H/e): never changed or destroyed, each with its status, counts, size and checksum. Starting one needs sysLegalHoldExport, an active hold and a reason, and is recorded in the audit log like the audit log's own export. The build is in memory and capped at 2 GB; bigger holds fail with a message saying so, and are split by picking accounts. Tested: unit tests for safe ZIP names and the manifest and its hash; the legal_hold system test, on RocksDB, PostgreSQL and MySQL, exports a hold end to end (live and archived mail, the manifest's hash, an asked- for account the hold doesn't cover left out) and checks the refusals (no reason, a user without the permission, a released hold) and the audit record; and by hand from the console on a local server. Not covered by a test: the archived-item date range with two holds of different ranges over one account. |
||
|
jcoffey-dev
|
5d2e35b2dc |
Legal holds, step 1: the hold itself
inbuxa:LegalHold get/set places a hold on accounts, groups, domains, tenants or the whole server, with an optional date range. A hold's range and scope can only widen, a released hold is read-only, and none is ever deleted. Placing, changing and releasing each need a reason and are audited (LH-1, LH-3, LH-10, AU-12). Permissions 669-672 (see, place, widen or release, export held data) go to server administrators only; the tenant ceiling always strips them, as it does Impersonate (LH-13). Schema: Compliance > Legal Holds. What a hold keeps comes next, through the undelete hooks. Also moves the lock expiry helpers below the lock module's imports. |
||
|
jcoffey-dev
|
447229f871 |
Lock accounts: keep receiving mail, no sign-in, hand to delegates
A locked account can't sign in (it fails as a wrong password does), its sessions end on every node, refresh tokens stop working, and its Sieve scripts forward and reply to nothing. Mail keeps arriving. Delegates get real ACL grants on the account's mailboxes, calendars, address books and files at read, organize or full, with the rights they replaced restored on unlock. Folders made later are granted after the create and in a daily sweep. Organize delegates can't destroy; send-as needs organize or full. The JMAP session marks delegated accounts in urn:inbuxa:jmap. New inbuxa:AccountLock object with get/set, permissions 665-668, and a Compliance > Locked Accounts entry in the schema. Lock, unlock and delegate changes need a reason and are audited; delegate access and writes are audited too (audit-hold-lock spec AL-1 to AL-12). |
||
|
jcoffey-dev
|
86d7ebd982 |
Audit log: a permanent, tamper-evident record of admin actions
What administrators and the server itself do to the control plane is now recorded, from inbuxa-drafts/specs/audit-hold-lock.md (AU-1 to AU-12): settings, accounts, domains, roles and every other registry change, with each field's before and after (secrets only as "changed"); the fork's own settings objects; administrator sign-ins (and failed ones to administrator accounts), master-user and recovery-admin sign-ins, once an hour per account, method and address; access to another account's data through impersonation or FetchAnyBlob, once an hour; exports and tamper checks; and registry writes the server makes on its own, named by subsystem (system:AcmeRenewal, system:auto-ban, system:directory-sync, ...), with a spam rules update as one summary record. No change without its record (AU-3): before a set method changes anything, a pending record per requested create, update and destroy is written; if that fails, the method is refused with serverFail. Its outcome follows as a later entry. A change interrupted by a crash stays "unfinished". Records live in the fork's subspace under L, as one SHA-256 hash chain per node. The chain's head is stored, never cached, and every append asserts it, so two writers can't take the same place. Nothing can edit or delete a record; the daily purge removes the oldest past the retention (default 730 days, minimum 90) and records where the chain now starts, so verification still passes. security.audit-recorded (647) copies each record to webhooks, OpenTelemetry and the log; security.audit-write-failed (648) reports a failed write. New JMAP objects under urn:inbuxa:jmap: inbuxa:AuditEvent/get and /query (filters: time, actor, action, target, account, tenant, outcome, address, text), inbuxa:AuditSettings, inbuxa:AuditExport (CSV or JSON Lines built on the server, each line with its chain hash, ending in a manifest; the created object names the blob and its SHA-256) and inbuxa:AuditVerification. New permissions sysAuditGet, sysAuditExport and sysAuditSettingsUpdate: the Administrator role gets all three, the Tenant Administrator role gets read and export, once, on existing installs too. A tenant administrator sees records whose actor or target is in its tenant, including a server administrator's changes there. Sign-in method on the session: access tokens now remember how they signed in (password, app password, API key, OAuth client, directory, master user, recovery admin), including across the HTTP credential cache. New OAuth access tokens carry their client id in the sealed claims; older ones show as client "unknown" until they expire. The schema gains the permissions, the two events and a Management > Compliance > Audit Log link. Stack: the request layer boxes every inner future where it's made. Without that, a debug build overflowed the default 2 MB worker stack on a registry set; measured with the same request, the branch and main now overflow at the same stack size (between 1856 and 1920 KiB, debug), so the layer adds nothing measurable. Tests: unit tests in inbuxa-features and jmap; system::audit::audit_log_tests (run with --ignored) passes on RocksDB, SQLite, PostgreSQL, PostgreSQL with a read replica, MySQL, MySQL with a replica and FoundationDB. The system, JMAP and SCIM suites pass. authorization.rs skipped fork permissions that guard no registry object; the audit suite checks a plain user is refused instead. |
||
|
jcoffey-dev
|
d9a6db025b |
Explain this: the local model reads delivery failures, verdicts, logs and settings
A new method, inbuxa:Explanation/set, asks the node's local model for a short plain-words reading of one thing an administrator is looking at: a failed recipient in the queue, a Classify verdict, a log line or trace event, or one setting with its saved value. The server builds the prompt itself from stored data and the registry schema, never from text the console sends, and grounds SMTP replies in RFC 3463 and RFC 5321. What the model is never shown: secrets (including ones nested inside a setting, like an AI model's HTTP auth), raw protocol events, and the contents of any other event. A tag name that doesn't have a tag's shape is refused before a model is asked. Calls share the AI gate with spam classification, but mail always keeps its slot, and Explain has its own hourly count per account and its own on/off switch in inbuxa:AiLimits. The permission is sysAiExplain, superuser only; tenant administrators can't use it. The session carries an aiExplain flag so a console knows when to offer the button. An install whose roles were stored before the permission existed gets it added once, at start-up, to the roles that are administrators' alone, not the User role their defaults share with every account. An operator who removes it later isn't overruled. Tests: unit tests in inbuxa-features and jmap, and ai_explain_tests (run with --ignored) covering the acceptance tests and the upgrade. |
||
|
jcoffey-dev
|
b65afb66f9 |
A tenant can turn legacy protocols off for itself (LP-9 to LP-14a)
The tenant switch. A tenant's administrator turns legacy mail protocols off for its own tenant, and from then on sign-in over IMAP, POP3, ManageSieve and SMTP AUTH is refused for every address on the tenant's domains, while every other domain on the server carries on. No port closes, since other tenants share them (LP-13): it is one stored fact per tenant, read at sign-in and when client configuration is answered. inbuxa:TenantProtocolPolicy/get and /set, one per tenant, id the tenant's: - Inside a tenant, a principal reaches only its own tenant's switch (MT-1): /get with no ids answers with it, another tenant's is notFound and can't be changed. At server level /get with no ids lists every tenant's. - Turning it off is always allowed. Turning it back on is refused with forbidden, naming inbuxa:ProtocolPolicy, while the server has legacy protocols off (LP-9). - A change raises security.legacy-protocols-changed with policy = tenant, the tenant's id, the new value and who made it (LP-14). - It takes sysDomainGet and sysDomainUpdate, not the two new permissions the spec names. The switch governs sign-in on the tenant's domains, so whoever manages those domains may turn it -- and the default Tenant Administrator role already holds both, where new permissions would reach no role already stored on a server (MT-12's note), leaving today's tenant administrators without the switch until someone edited their role by hand. The same trade inbuxa:AiLimits and inbuxa:ProtocolPolicy made. /query is not built yet; /get with no ids covers listing. Sign-in (LP-10 to LP-12). Before the credentials are looked at, the name given is resolved to its domain and the domain to its tenant, so a real account and a made-up address on the domain get the same refusal, with a right password or a wrong one, counted as no failed sign-in (LP-11). The words are the spec's: "Your organization allows only INBUXA webmail and JMAP apps...", in each protocol's form. A bearer token needn't name an account, so after authentication the account's own tenant is checked too; a token that named nobody can't slip past. The refusal carries policy = tenant and the domain, not the tenant's id: IMAP answers a command's tag from the Id key, so an error holding one was sent under the wrong tag and the mail app hung waiting for its reply. The first live run found that; a unit test now holds the refusal to it. Client configuration (LP-14a). Autoconfig, autodiscover, PACC and the suggested DNS records now ask whether legacy services are off for the domain being answered for -- the server's switch, or the domain's tenant's -- so a tenant's domains stop offering IMAP, POP3 and submission while others still do. tests/e2e/legacy_protocols.py builds a tenant with its own domain, a user and a tenant administrator, and a second tenant, and proves on a running server: the admin sees and changes only its own tenant's switch (test 10); turning it off is an event (test 14); the tenant's user is refused over IMAP with the right password and a wrong one, a made-up address on the domain the same (tests 6, 7); POP3 and submission refuse in their own forms and JMAP still works (test 8); an account on another domain signs in normally (test 6); autoconfig drops IMAP for the tenant's domain only; with the server off, the tenant can't turn it back on (test 9); and once back on, the user signs in again. All 62 checks pass. |
||
|
jcoffey-dev
|
1b3ec64862 |
inbuxa:ProtocolPolicy over JMAP
The switch is now reachable. /get and /set on a server-level singleton, wired through jmap-proto the way inbuxa:AiLimits is: object, method names, request and response variants, reference resolution and evaluation. /set does not write the policy. It hands what was asked to Server::set_protocol_policy, which applies the locks, moves the listener objects and opens or closes their sockets, and reports what happened. So the method cannot drift from what the switch actually does. Two properties exist for the screen rather than the server. lockedProtocols serves LP-21's locked set, so the selector renders SMTP and JMAP locked from what the server says instead of a list the front end carries -- and unlocking later needs no admin release. wouldClose answers LP-16: exactly which listeners turning the switch on would close, by name and port, before anything happens. It is computed against a hypothetical disabled policy, so it reads the same whichever way the switch is set, and the registry is only asked when the property was requested. savedListeners, changedAt, changedBy and both of those are the server's to say; a client that sets one gets invalidProperties naming it. closeSubmission is different: locked, not immutable, so it is overruled rather than refused and the response hands back what was really stored (false). JMAP already has the place for that, the value beside an updated id. Permissions reuse SysNetworkListenerGet and SysNetworkListenerUpdate rather than adding to a schema-generated enum -- the same choice AiLimits made with the classifier's. It also reads right: this takes listeners away and puts them back, so whoever may edit a listener may turn the switch. changedBy stores the account id, not the name, which survives a rename. Still no screen, no sign-in refusal (LP-6) and no event (LP-8). |
||
|
jcoffey-dev
|
6a53d47106 |
Mark the files this fork changed (AGPL section 5(a))
The AGPL asks a modified version to carry prominent notices saying it was modified, and giving a date. Publishing the source is the conveyance that asks for it, so it wants doing before the repository is public rather than at the release. Every upstream file the fork changed now says so in its header, beneath the notice it came with: 164 files, found by diffing against the upstream snapshot branch rather than by guessing, so the list is what actually differs. Files the fork wrote itself already carry their own copyright and need nothing. Upstream's notices are untouched, which its licence requires and which was already true. The README says the same thing in prose, since the obligation is on the work as a whole and not only its Rust files. Builds unchanged: the server and the test binary both compile. |
||
|
jcoffey-dev
|
9490fc4677 |
AI spam classification: the model's opinion as one bounded spam signal, and the llm_prompt Sieve function (AI-1 to AI-28)
The classifier sends only the subject and text, between unforgeable markers after the operator's prompt, to an OpenAI-compatible endpoint the operator configured; nothing is preset. Its answer maps to an LLM_ tag whose score is clamped (+5.0, -1.0 by default) and can never discard or reject on its own; X-Spam-LLM is sanitized, encoded and folded, and a planted one is removed. Failures, timeouts past the ceiling, a full slot or a paused model leave mail flowing untagged. llm_prompt answers trusted scripts, and accounts holding interactAi within an hourly limit. Redirects aren't followed and no content or secret is logged. The limits live in inbuxa:AiLimits. Acceptance tests 1 and 3 to 21; test 2 as the re-enabled shared llm case, whose setup no longer waits on a rules file from a developer's own path; test 22 written as the ignored ai_compat. |
||
|
jcoffey-dev
|
ecbdfd533b |
Undelete: deleted accounts are kept for their period, hold their addresses, and are restored or destroyed through inbuxa:DeletedAccount (UD-15 to UD-17a)
With archiveDeletedAccountsFor set, a destroyed account's record is kept in the fork subspace with its id, its DestroyAccount task is due at the end of the period, and its shares are suspended both ways. Its addresses can't be taken by new accounts, aliases, lists or masks. inbuxa:DeletedAccount/get lists kept accounts to server and tenant administrators; /set restores one with a new password (same id, task cancelled, shares reinstated) or destroys it now. The destroy task also clears undelete's own records. Acceptance test 14; test 16 written as the ignored undelete_compat. |
||
|
jcoffey-dev
|
d04aafd3d7 |
Masked email: Fastmail's Masked Email API, MaskedEmail/get and /set (ME-1, ME-7a, ME-16)
Advertised as https://www.fastmail.com/dev/maskedemail in the session and on every account that may hold masks. Masks created through it start pending unless the create sets a state; pending can't be set again once left; state and the other mutable fields map onto the same records the x: API uses. |
||
|
jcoffey-dev
|
7dae9b29fd |
Import upstream v0.16.22, stripped
Upstream commit: 474dd0229cb20cf513036619781ed97bd8073c3f Enterprise-only files removed or emptied: 63 Enterprise-only snippets removed: 117 in 50 files Dangling module declarations removed: 5 Cargo edits turning enterprise off: 14 Verification: clean Enterprise feature gates left for rebuilt features: 19 in 18 files Produced by tools/fork/strip.py. The full report is in docs/fork/strip-reports/ on main. |