Commit Graph
52 Commits
Author SHA1 Message Date
jcoffey-dev f4061f542c Merge pull request 'Menus: Held Mail, Data Loss Prevention and Mail Flow Rules' (#109) from feature/dlp-console-nav into main
ci / fork-checks (push) Successful in 55s
ci / build (push) Canceled after 5m54s
2026-09-29 01:48:07 +00:00
jcoffey-dev 9653219c53 Menus: Held Mail, Data Loss Prevention and Mail Flow Rules
ci / fork-checks (pull_request) Successful in 1m31s
ci / build (pull_request) Successful in 13m30s
Schema layout entries for the console pages of the DLP and mail flow rules spec (§3): Held Mail and Data Loss Prevention under Management > Compliance after Legal Holds, and Mail Flow Rules beside the server Sieve scripts (the console's nine-group Settings bar places it under Mail flow). An older console shows these as unknown pages, so this ships with the console that has them.
2026-09-28 18:34:18 -07:00
jcoffey-dev f44382fb09 DLP phase 3: hold for review
ci / fork-checks (pull_request) Successful in 33s
ci / build (pull_request) Successful in 9m58s
The hold action now holds (dlp-and-mail-flow-rules spec, §2.6), where
until now it blocked.

- At DATA a hold decision queues the message with its release a century
  off (the queue's future-release mechanism, so the stored format is
  unchanged and an older node just never sends it), transport rules
  still applied, and replies 250 Held for review. A review record under
  R/h + queue id keeps the sender, recipients, subject, size, rules and
  detector counts. The sender is told when the rule asks.
- smtp/queue/held.rs: release (each recipient due now, its next notice
  as far off as it was, its lifetime counted from the release), reject
  (removed from the queue, the sender told, with the reviewer's note),
  and expiry: the daily clean-up rejects what nobody reviewed in 7 days,
  recorded as the server's doing.
- inbuxa:HeldMessage get/set: the review queue, sysDlpReviewGet to list
  and read (preview, 64 KB of text, only when asked for and recorded as
  blobAccess), sysDlpReviewUpdate to release or reject, a reason
  required and audited by the request layer; no create or destroy;
  server-level only.
- Guards: Emails > Queue refuses to change or delete held mail; the
  sender can't unsend it.
- Privacy catalog entry for inbuxa:HeldMessage; spec §2.6 as built.

Tests: mail_rules_tests gains the whole flow (held and listed with
counts, sender notified and nothing delivered, queue and unsend
refused, preview recorded, reject needs a reason and tells the sender
the note, release delivers, expiry returns it, decisions audited with
reasons). smtp inbound, system_tests (after one BlobNotFound in
antispam, the known flake, then clean), features and common unit tests.
2026-09-28 18:33:12 -07:00
jcoffey-dev 8afaee7d21 DLP and mail flow rules: inbuxa:MailRule over JMAP, and its permissions
ci / fork-checks (pull_request) Successful in 15s
ci / build (pull_request) Successful in 4m41s
Phase 2e of the DLP and mail flow rules spec, the API half.

- inbuxa:MailRule/get and /set under urn:inbuxa:jmap. Rules convert
  through serde, so what a client sends is the stored format. A create
  or change is validated whole (Rule::validate) and refused with the
  property at fault; id, createdBy, createdAt and updatedAt are the
  server's. Every change goes through the request layer's audit record.
- Six permissions, ids 674-679 (enum and schema labels): mail flow rules
  (sysMailRuleGet/Update), DLP rules (sysDlpPolicyGet/Update) and held
  mail (sysDlpReviewGet/Update, for phase 3). Either kind's permission
  gets through the gate; the handler shows and changes each rule only
  with its own kind's. All server-level: a tenant is refused (settled
  answer 3).
- Administrators get all six; the server-level Compliance Officer gets
  DLP rules to see and held mail to review (settled answer 4), added
  once to an existing server's officer role by the grant mechanism,
  which gains an officer audience.
- Privacy catalog entry for inbuxa:MailRule.

tests/src/system/mail_rules.rs: create, list in order, validation,
server-set properties refused, update, kind-separated permissions for
an officer, destroy, audit records.
2026-09-28 17:29:35 -07:00
jcoffey-dev 3199a6f1fb Release 2026.9.28.5
ci / fork-checks (pull_request) Successful in 47s
ci / build (pull_request) Successful in 7m37s
2026-09-28 16:57:47 -07:00
jcoffey-dev 32b22d0828 Merge pull request 'Schema: each expression field says which values and variables it accepts' (#91) from feature/expression-schema into main
ci / fork-checks (push) Canceled after 20s
ci / build (push) Canceled after 20s
2026-09-28 21:51:34 +00:00
jcoffey-dev e61a475859 Schema: each expression field says which values and variables it accepts
ci / fork-checks (pull_request) Successful in 52s
ci / build (pull_request) Successful in 5m24s
The registry knows, for every expression field, the constants it may
evaluate to and the variables its conditions may read, and enforces both.
The schema served to INBUXA Admin described every one as a bare
x:Expression, so the console could offer nothing better than free text.

tools/fork/expr-schema.py reads those contexts from the generated registry
code and writes them onto each field's type as
expression: {constants, variables}. All 124 expression fields are covered.
CI runs it with --check so the schema can't drift from the registry.
2026-09-28 12:32:28 -07:00
jcoffey-dev 6c862e4971 Release 2026.9.28.4
ci / fork-checks (pull_request) Successful in 15s
ci / build (pull_request) Successful in 23m29s
The personal-data catalog and what it feeds: the data inventory and its
snapshots (#83, #89), the Compliance Officer roles (#88), the Compliance
Overview and Data Inventory menu entries (#92). Log file retention
(#87), privacy defaults for new installs (#85, #90), webhooks that send
only the events they name (#82), and upstream v0.16.24 (#84), whose
spam rules updates keep what an admin edited.

Explain: 12 settings asked about (upstream's new createdAt fields, the
certificate dates and the webhook events policy), with the release's
recommended model built locally; 705 answers carry over.
2026-09-28 12:22:34 -07:00
jcoffey-dev 728586998b Catalog: what the Overview showed wrong
ci / fork-checks (pull_request) Successful in 19s
ci / build (pull_request) Successful in 44m19s
Seen in the console's first Overview. x:DmarcTroubleshoot and
x:SpamClassify are one-off actions whose results come back in the
response, not kept: object-life, not unbounded. Tasks go when done
(only a failed one's status may stay, still unconfirmed): object-life.
x:Log reads the log files, so it follows inbuxa:LogSettings.keepForDays.
x:TracerLog, x:WebHook and the OpenTelemetry tracers are configuration:
their credential fields stay classified, but they are no longer listed
in the inventory, where they counted as always sent off the server
even with none configured; the log-file, webhooks and otel-tracer
sources carry what they send.
2026-09-28 11:03:55 -07:00
jcoffey-dev cca49de92c Compliance menu: Overview and Data Inventory first
ci / fork-checks (pull_request) Successful in 16s
ci / build (pull_request) Canceled after 9m28s
Personal-data catalog spec, §8: the Compliance section in the console
reads Overview, Data Inventory, Legal Holds, Audit Log, Locked
Accounts. The two new entries are hand-built console pages
(CustomComponent/ComplianceOverview, CustomComponent/DataInventory),
shown to people with sysComplianceGet.

A console from before these pages shows the links and answers
"Unknown component", so the console that has them should be deployed
with the server release that carries this. Schema edited as the fork's
earlier Compliance entries were, hash updated.
2026-09-28 10:54:32 -07:00
jcoffey-dev a8fb10458b Evaluate the personal-data catalog: the data inventory and its history
ci / fork-checks (pull_request) Successful in 57s
ci / build (pull_request) Successful in 15m6s
Personal-data catalog spec, §6 (Phase 3c).

inbuxa:DataInventory/get evaluates the catalog against the server's
live settings and says what this server holds: for each source and each
object that can hold personal data, its categories and whose data it
is, whether it is collected here at all, what bounds its retention (the
live value of the setting that does, or unbounded), whether it leaves
the host and to which endpoints, and a summary. Every host that
receives something is listed once as a candidate processor with what it
receives. Inside a tenant it answers with the tenant's slice and none
of the server's processors. Read-only, with sysComplianceGet.

inbuxa:InventorySnapshot/get is the history: a dated copy of the
evaluated inventory, recorded when it changes -- after a registry write
to an object the inventory reads, after inbuxa's log, audit or AI
settings change, and on the daily clean-up -- and kept as long as the
audit log's records. ids: null lists every snapshot, newest first; the
full inventory only when asked for.

The catalog is embedded and parsed at start (new dependency: toml,
MIT/Apache); the evaluation is a pure function of it and the live
facts, so each configuration is tested without a server. Loopback
endpoints stay on the host; any other configured endpoint leaves it.

Tested: unit tests for the evaluation (a new install's defaults, an
external blob store, a hosted AI endpoint, telemetry off, a tenant's
slice, hosts from URLs, loopback), snapshots, and the fact gathering's
store and duration rules; the compliance system test, extended (the
officer reads the inventory, a plain user is refused, a tenant's
officer sees its slice and no processors, a webhook to another host
becomes a processor and a snapshot names x:WebHook, a retention change
reads through); the system, audit, legal hold and account lock suites;
fork checks. The system suite failed once of three runs with an email
import's blob not found, in antispam.rs; the same happened once in
purge.rs on the previous branch. Nothing here touches uploads; noted
for a separate look.
2026-09-28 09:59:48 -07:00
jcoffey-dev 7285b3e38a Merge main (upstream v0.16.24) into feature/compliance-roles
ci / fork-checks (pull_request) Successful in 15s
ci / build (pull_request) Successful in 7m33s
The schema conflicted as a binary file: taken from main and the one
edit here re-applied (sysComplianceGet after sysLegalHoldExport). The
import kept the permission count at 673, so the new id stays 673.

Retested on the merged tree in its own target directory: the
compliance and system suites pass. One earlier system run failed in
purge.rs (an imported blob not found) and didn't recur.
2026-09-28 09:26:27 -07:00
jcoffey-dev 9893452ca2 Merge pull request 'Merge upstream v0.16.24' (#84) from merge/upstream-v0.16.24 into main
ci / fork-checks (push) Successful in 54s
ci / build (push) Canceled after 39m0s
2026-09-28 15:55:16 +00:00
jcoffey-dev 63adb4e2b8 Add the compliance permission and the Compliance Officer roles
ci / fork-checks (pull_request) Successful in 31s
ci / build (pull_request) Successful in 11m31s
Personal-data catalog spec, §7 (settled 2026-09-28).

sysComplianceGet (673) sees the data inventory and compliance
overview: superusers and, for their tenant's slice, tenant
administrators, by default and through the one-time grants on servers
that already have their roles stored.

A Compliance Officer role at server level holds it with reading and
exporting the audit log, placing, widening, releasing and exporting
legal holds, seeing account locks, and reading accounts, lists,
domains, tenants and roles. It changes no server setting, creates or
deletes no account, and can't shorten audit retention.

A tenant's accounts can hold only roles of their own tenant (MT-3), so
the tenant role is one "Compliance Officer" role per tenant, without
holds (LH-13): made once for every tenant a server has, and whenever a
tenant is created. While nobody holds it, it is removed with its tenant
so it doesn't block the delete, and put back if the delete is refused
for another reason. Both roles carry a user's own permissions too,
since roles given to a person replace the default user role, which a
tenant's accounts can't hold anyway.

Every server makes these once, new or existing -- the built-in roles
are only made on a server with none -- and records each under P c, so a
role an administrator deletes stays deleted.

Tested: unit tests (neither role changes a setting beyond a user's
own; holds for the server's officer only; per-place records); a new
compliance system test (one server-level role; an officer reads the
audit log, places and releases a hold, and is refused a setting, an
account and audit retention; a tenant gets its role, whose holder reads
the tenant's audit log and no holds; a tenant with an unused role is
deleted and the role goes with it); the system, audit, legal hold,
account lock and SCIM suites; fork checks. The directory suite needs
its LDAP container and wasn't run here.
2026-09-28 08:50:17 -07:00
jcoffey-dev 1d5a49409f Keep rotated log files for a set number of days
ci / fork-checks (pull_request) Successful in 2m28s
ci / build (pull_request) Successful in 3m47s
Personal-data catalog spec, default D1 (settled 2026-09-28): log files
were never deleted. inbuxa:LogSettings.keepForDays says how many days
rotated log files are kept; unset (null) keeps every file, as before,
and a new install sets 30 days.

It is a fork-owned setting, stored under T + l as audit retention is,
not a field on x:TracerLog: that object is also stored inside
x:Bootstrap with a field after it, so a new field would change
x:Bootstrap's stored format. Server-level, with the tracers'
permissions (sysTracerGet, sysTracerUpdate); changes are in the audit
log, before and after.

Log files are local, so every node deletes its own: hourly, and at once
when the setting changes on that node. Only regular files named
<prefix>.<something> in each enabled log tracer's directory, last
changed more than the limit ago, are removed; the file being written is
never that old, and nothing else in the directory is touched. Minimum
one day. The catalog classifies inbuxa:LogSettings and points the log
file's retention at it.

Tested: unit tests for the file rule (only this log's old files; the
current file, other files and directories stay) and a purge on disk;
the system suite, which reads, sets, refuses zero, restores null and
checks the audit records; fork checks.
2026-09-28 08:23:36 -07:00
jcoffey-dev 80d6c09c59 Merge main into merge/upstream-v0.16.24
ci / fork-checks (pull_request) Successful in 21s
ci / build (pull_request) Successful in 35m18s
The schema, which both sides changed, merged as JSON with no conflicts.
The personal-data catalog (#83) gains upstream's new x:DnsServerPowerDns:
nothing personal but its API key, like the other DNS providers.
2026-09-28 08:19:24 -07:00
jcoffey-dev a0ffdb8071 New-install privacy defaults, and expired bans purged daily
ci / fork-checks (pull_request) Successful in 48s
ci / build (pull_request) Successful in 6m52s
Personal-data catalog spec, defaults D2, D3, D4, D6 and D7 (settled
2026-09-28, new installs only):

- D2: automatic IP bans expire after 30 days instead of never; D3:
  spam training samples, whole messages, are kept 90 days instead of
  180; D4: Pyzor, which sends a digest of each message's text to a
  public server, is off; D6: delivery history is kept 14 days instead
  of 30. Written on the first boot of a new install only -- one with no
  roles yet, the same test the built-in roles use -- by reading each
  singleton, setting these fields and writing it back whole. A server
  with roles keeps its settings, saved or default.
- D7: a webhook created from now on starts with the include policy and
  no events, so it sends nothing until events are chosen (Rust default
  and schema default, marked). The registry stores every field, so
  existing webhooks keep their policy.
- Expired bans are also removed by the daily data clean-up. They
  already stopped blocking and were deleted when settings next loaded;
  a server that seldom reloads kept them.

D1 (log retention) and D5 (the hashed-address blocklist off) are held,
and the spec says why: x:TracerLog is stored inside x:Bootstrap with a
field after it, so adding one changes that object's stored format; and
the spam-rules loader D5 touches is being reworked by the v0.16.24
import. The spec also corrects finding 3: expired bans were deleted on
settings load; bans were permanent only because no period is set.

Tested: unit tests for the new-install values and that everything else
in each singleton stays; the system suite, whose security test now
purges an expired ban and checks its record is gone; the telemetry
test; common's unit tests; fork checks.
2026-09-28 07:43:59 -07:00
jcoffey-dev a8dde68800 Add the personal-data catalog and the check that keeps it true
ci / fork-checks (pull_request) Successful in 52s
ci / build (pull_request) Successful in 37m38s
Phase 2 of the personal-data catalog spec.

resources/privacy/catalog.toml classifies every object in the schema
(316) and inbuxa's own JMAP objects (12): each property that can hold
personal data, with its categories, and for objects that hold any,
whose data it is, where it lives, its scope and what bounds its
retention (a named setting where there is one). Twenty sources that
are no object -- the log file, exporters, webhooks, spam lookups, the
Explain cache, relays and hooks, push, legacy-use records -- carry the
same facts plus the settings that turn them on, whether the data
leaves the host, and the code that writes it. Classifications of
objects that hold data about people are from the spec's source map;
the rest are typed from the schema alone (address, IP, secret).

tools/fork/privacy-check.py fails CI when an object or inbuxa object
has no entry, when a property the schema types as an address, IP or
secret is left to its object's default, when an entry names an
object, property, setting or code path that is gone, or when it uses
a word outside the catalog's vocabulary. --unlisted prints starting
entries. strip.py's report gains "Unclassified in the privacy
catalog": objects and fields new in an import and not classified,
informational like the Enterprise flags.

Tested: 13 unit tests (tools/fork/tests): the check passes on this
tree; fails on an unclassified object, an address hidden behind a
default, a secret in a set or object reference, stale properties,
objects, settings and code paths, an unlisted inbuxa object and a
word outside the vocabulary; --unlisted's entries; and the strip
report on a synthetic import. The check and the tests run in the
fork-checks job.
2026-09-28 06:54:34 -07:00
jcoffey-dev b2453d066b Merge upstream v0.16.24
Eight conflicted files resolved, plus the lock file and the schema:

- crates/services/src/task_manager/spam_classifier.rs: upstream's rules
  update now replaces existing rules, DNSBL servers, lookups and file
  extensions, keeping only whether each is on. Taken, with one difference:
  an object an admin edited is kept as it is. Every object an update writes
  is fingerprinted (content without `enable`, SHA-256, stored under
  SUBSPACE_INBUXA "Sf"), and only one that still matches is replaced.
  Scores are never replaced, as upstream has it. The AU-1.10 summary record
  now names what was added, replaced and kept, and the bundled rules are
  marked applied only when the update fully succeeded, so a failure runs
  again on the next start. The marker becomes "3.0.2+2", which runs the
  update once on upgrade to fingerprint every rule still as bundled.
- crates/common/src/network/autoconfig/autodiscover.rs: upstream's rewrite
  (implicit TLS first, labeled SSL), with the per-protocol switches (LP-7,
  LP-14a) passed in as a filter.
- crates/store/src/backend/mysql/{search,write}.rs: upstream's chunked
  deletes (no unbounded first DELETE, stop on a short chunk, halve the
  chunk on the new chunk-too-large errors) inside the fork's query timeout.
- crates/smtp/src/lib.rs: the fork's queue spawn kept. It already fixed the
  stall upstream fixes here (a node without outboundMta stops accepting
  mail at about 1024 queued messages), and follows role changes live.
- crates/jmap/src/registry/mapping/bootstrap.rs: the log path stays
  /var/log/inbuxa/; upstream's PowerDNS mapping taken.
- crates/main/Cargo.toml: the AGPL-only license kept, version 0.16.24.
- tests/src/jmap/principal/get.rs: the fork's capabilities kept.
- resources/schema/schema.json.gz: merged as JSON; upstream relabeled the
  vendor Sieve extensions "(Stalwart)", kept as "(vnd.inbuxa)".
- Cargo.lock: upstream's, with the fork's crates added by Cargo.

Also:

- tests/src/smtp/inbound/spam_rules_kept.rs: an edited rule survives an
  update, an unedited one is updated, rules from before fingerprints are
  handled, and the audit summary says so. Upstream's own spam_rules test
  passes unchanged.
- tests/src/smtp/reporting/reschedule.rs moves to port 19058; upstream's
  new spam_rules test took 19057.
- tools/fork/renames.py renames the "(Stalwart)" labels and the default
  log path, so neither conflicts again.
- tools/fork/notice-check.py compares against the newest snapshot in the
  checked-out history instead of the upstream branch head, so moving the
  branch no longer fails other open pull requests.
- tests/src/directory/issuer.rs (since v0.16.23) stays out, and is on the
  build check's known list: it tests issuer-based directory routing, which
  the fork doesn't have (DIR-2).
- Strip report: docs/fork/strip-reports/v0.16.24.{md,json}.
2026-09-28 06:30:20 -07:00
jcoffey-dev f59b084ce5 Import upstream v0.16.24, stripped
Upstream commit: af37a234981722493b74623a983581691d2b70b6
Enterprise-only files removed or emptied: 63
Enterprise-only snippets removed: 118 in 50 files
Dangling module declarations removed: 5
Edits turning enterprise off: 25
Third-party code: 14 files, 0 not in THIRD-PARTY.md
Renamed identifiers: 62 in 18 files
Verification: clean

The same Enterprise footprint as v0.16.23. The build check fails only on
tests/src/directory/issuer.rs, unchanged since v0.16.23: it calls a helper
from upstream's Enterprise-only OIDC test, and tests issuer-based directory
routing, an Enterprise feature. main has never carried it.
2026-09-28 06:29:38 -07:00
jcoffey-dev 305406a331 Release 2026.9.28.3
ci / fork-checks (pull_request) Successful in 14s
ci / build (pull_request) Successful in 7m25s
Per-protocol legacy switches (#79) and the hold export's exceptions
list (#75). The prepared Explain answers are relabeled for this
release; 706 carry over unchanged.
2026-09-28 00:15:33 -07:00
jcoffey-dev b7e3a765ca Release 2026.9.28.2
ci / fork-checks (pull_request) Successful in 56s
ci / build (pull_request) Successful in 5m22s
2026-09-27 22:54:18 -07:00
jcoffey-dev 815a642cc4 Release 2026.9.28.1
ci / fork-checks (pull_request) Successful in 16s
ci / build (pull_request) Successful in 7m29s
Legal hold exports (LH-12, #73). The prepared Explain answers are
relabeled for this release; 706 carry over unchanged.
2026-09-27 21:55:55 -07:00
jcoffey-dev 3046c418cd Release 2026.9.28
ci / fork-checks (pull_request) Successful in 50s
ci / build (pull_request) Successful in 13m2s
Legal holds (#70): a hold on people, groups, domains, tenants or the
whole server keeps everything it covers from being destroyed, by anyone,
until it's released; deleted accounts keep their data. Audit records
name accounts by their full address and holds by their case name. The
daily clean-up of expired archived items works again.

Prepared Explain answers relabeled for this release; no setting changed
since 2026.9.27.2, so all 706 carry over.
2026-09-27 20:04:44 -07:00
jcoffey-dev 5d2e35b2dc Legal holds, step 1: the hold itself
inbuxa:LegalHold get/set places a hold on accounts, groups, domains,
tenants or the whole server, with an optional date range. A hold's range
and scope can only widen, a released hold is read-only, and none is ever
deleted. Placing, changing and releasing each need a reason and are
audited (LH-1, LH-3, LH-10, AU-12).

Permissions 669-672 (see, place, widen or release, export held data)
go to server administrators only; the tenant ceiling always strips them,
as it does Impersonate (LH-13). Schema: Compliance > Legal Holds.

What a hold keeps comes next, through the undelete hooks.

Also moves the lock expiry helpers below the lock module's imports.
2026-09-27 19:33:06 -07:00
jcoffey-dev 355bd3a40e Release 2026.9.27.2
ci / fork-checks (pull_request) Successful in 1m23s
ci / build (pull_request) Successful in 7m16s
Delegates reach the whole locked account (#68): its calendars, contacts
and files as well as its mail, even a kind it holds none of yet, and
writing delegates may add at the top of its Files.

Prepared Explain answers relabeled for this release; no setting changed
since 2026.9.27.1, so all 706 carry over.
2026-09-27 18:27:33 -07:00
jcoffey-dev 30ea43d019 Release 2026.9.27.1
ci / fork-checks (pull_request) Successful in 13s
ci / build (pull_request) Successful in 7m33s
The audit log (#64): every administrator change, admin sign-in and look
into someone else's data, recorded before it happens, chained per node
and checkable for tampering, exportable with a manifest, kept 2 years.

Locked accounts (#65, #66): an account that keeps receiving mail but
can't sign in and sends nothing on its own, handed to delegates at read,
organize or full, ending at a date when one is set.

Prepared Explain answers relabeled for this release; no setting changed
since 2026.9.27, so all 706 carry over.
2026-09-27 16:35:42 -07:00
jcoffey-dev 447229f871 Lock accounts: keep receiving mail, no sign-in, hand to delegates
ci / fork-checks (pull_request) Successful in 1m4s
ci / build (pull_request) Successful in 8m47s
A locked account can't sign in (it fails as a wrong password does), its
sessions end on every node, refresh tokens stop working, and its Sieve
scripts forward and reply to nothing. Mail keeps arriving.

Delegates get real ACL grants on the account's mailboxes, calendars,
address books and files at read, organize or full, with the rights they
replaced restored on unlock. Folders made later are granted after the
create and in a daily sweep. Organize delegates can't destroy; send-as
needs organize or full. The JMAP session marks delegated accounts in
urn:inbuxa:jmap.

New inbuxa:AccountLock object with get/set, permissions 665-668, and a
Compliance > Locked Accounts entry in the schema. Lock, unlock and
delegate changes need a reason and are audited; delegate access and
writes are audited too (audit-hold-lock spec AL-1 to AL-12).
2026-09-27 14:46:06 -07:00
jcoffey-dev 86d7ebd982 Audit log: a permanent, tamper-evident record of admin actions
ci / fork-checks (pull_request) Successful in 52s
ci / build (pull_request) Successful in 1h4m15s
What administrators and the server itself do to the control plane is now
recorded, from inbuxa-drafts/specs/audit-hold-lock.md (AU-1 to AU-12):
settings, accounts, domains, roles and every other registry change, with
each field's before and after (secrets only as "changed"); the fork's own
settings objects; administrator sign-ins (and failed ones to administrator
accounts), master-user and recovery-admin sign-ins, once an hour per
account, method and address; access to another account's data through
impersonation or FetchAnyBlob, once an hour; exports and tamper checks;
and registry writes the server makes on its own, named by subsystem
(system:AcmeRenewal, system:auto-ban, system:directory-sync, ...), with a
spam rules update as one summary record.

No change without its record (AU-3): before a set method changes anything,
a pending record per requested create, update and destroy is written; if
that fails, the method is refused with serverFail. Its outcome follows as
a later entry. A change interrupted by a crash stays "unfinished".

Records live in the fork's subspace under L, as one SHA-256 hash chain per
node. The chain's head is stored, never cached, and every append asserts
it, so two writers can't take the same place. Nothing can edit or delete
a record; the daily purge removes the oldest past the retention (default
730 days, minimum 90) and records where the chain now starts, so
verification still passes. security.audit-recorded (647) copies each
record to webhooks, OpenTelemetry and the log; security.audit-write-failed
(648) reports a failed write.

New JMAP objects under urn:inbuxa:jmap: inbuxa:AuditEvent/get and /query
(filters: time, actor, action, target, account, tenant, outcome, address,
text), inbuxa:AuditSettings, inbuxa:AuditExport (CSV or JSON Lines built
on the server, each line with its chain hash, ending in a manifest; the
created object names the blob and its SHA-256) and
inbuxa:AuditVerification. New permissions sysAuditGet, sysAuditExport and
sysAuditSettingsUpdate: the Administrator role gets all three, the Tenant
Administrator role gets read and export, once, on existing installs too.
A tenant administrator sees records whose actor or target is in its
tenant, including a server administrator's changes there.

Sign-in method on the session: access tokens now remember how they signed
in (password, app password, API key, OAuth client, directory, master user,
recovery admin), including across the HTTP credential cache. New OAuth
access tokens carry their client id in the sealed claims; older ones show
as client "unknown" until they expire.

The schema gains the permissions, the two events and a Management >
Compliance > Audit Log link.

Stack: the request layer boxes every inner future where it's made. Without
that, a debug build overflowed the default 2 MB worker stack on a registry
set; measured with the same request, the branch and main now overflow at
the same stack size (between 1856 and 1920 KiB, debug), so the layer adds
nothing measurable.

Tests: unit tests in inbuxa-features and jmap; system::audit::audit_log_tests
(run with --ignored) passes on RocksDB, SQLite, PostgreSQL, PostgreSQL with a
read replica, MySQL, MySQL with a replica and FoundationDB. The system, JMAP
and SCIM suites pass. authorization.rs skipped fork permissions that guard
no registry object; the audit suite checks a plain user is refused instead.
2026-09-27 13:40:12 -07:00
jcoffey-dev 833e6871f7 Release 2026.9.27
ci / fork-checks (pull_request) Successful in 45s
ci / build (pull_request) Successful in 4m51s
inbuxa's own mark (#62): the kitten over a server with a bay for each
piece of the suite, on the built-in sign-in and RSVP pages, the web
logo and the email logo.

Prepared Explain answers relabeled for this release; no setting changed
since 2026.9.26.1, so all 706 carry over.
2026-09-26 22:13:54 -07:00
jcoffey-dev d7182f4511 Brand: inbuxa's own kitten replaces ihasmail's cat
ci / fork-checks (pull_request) Successful in 43s
ci / build (pull_request) Successful in 3m18s
inbuxa's mark was ihasmail's cat-and-envelope reused unchanged. The new
one keeps the family's face, paws and colors, over a server with a bay
for each piece of the suite: the letter (webmail), a prompt (console),
status lights (server).

- The built-in sign-in and calendar RSVP pages, and the web logo, drew
  the old cat as an embedded PNG. They now draw the mark as vector in
  the same slot, keeping class="symbol"; each page is about 31 KB
  lighter. The .min copies are updated the same way and the .min.gz
  regenerated with gzip -9 -n, as minify_html.sh does.
- resources/branding: email-logo.png (the compact lockup at 380x80 on
  white, as before) with its .b64 regenerated byte-for-byte in the old
  76-column form, and favicon-64.png.
- img/brand: the logo bundle, now pure vector, with its README.
2026-09-26 22:05:17 -07:00
jcoffey-dev bed0d72e3f Release 2026.9.26.1
ci / fork-checks (pull_request) Successful in 51s
ci / build (pull_request) Successful in 11m54s
Prepared Explain answers relabeled for this release; 11 settings whose
default is the time of creation drop out, since their answers could never
match.
2026-09-26 16:31:09 -07:00
jcoffey-dev 7e7eca0883 Explain: shorter answers, streamed, remembered, and prepared for settings
ci / fork-checks (pull_request) Successful in 1m31s
ci / build (pull_request) Failing after 5m18s
ai-explain spec, amendment 1 (EX-22 to EX-28):
- answers are three or four sentences, max_tokens 160, cut at 700 chars;
- POST /api/explain streams the answer as server-sent events;
- each node remembers answers in memory (1,000, 24 h), keyed by the facts,
  prompt version and model, shared by server-level administrators;
- resources/explain/settings.json.gz ships answers for settings at their
  defaults, generated with prepare_setting_explanations (717 for 2026.9.27);
- the system prompt no longer carries the per-request marker, so a model
  server can reuse it;
- inbuxa:Explanation gains source, answeredAt and preparedFor.
2026-09-26 16:11:43 -07:00
jcoffey-dev d9a6db025b Explain this: the local model reads delivery failures, verdicts, logs and settings
ci / fork-checks (pull_request) Successful in 48s
ci / build (pull_request) Successful in 9m4s
A new method, inbuxa:Explanation/set, asks the node's local model for a
short plain-words reading of one thing an administrator is looking at:
a failed recipient in the queue, a Classify verdict, a log line or trace
event, or one setting with its saved value. The server builds the prompt
itself from stored data and the registry schema, never from text the
console sends, and grounds SMTP replies in RFC 3463 and RFC 5321.

What the model is never shown: secrets (including ones nested inside a
setting, like an AI model's HTTP auth), raw protocol events, and the
contents of any other event. A tag name that doesn't have a tag's shape is
refused before a model is asked.

Calls share the AI gate with spam classification, but mail always keeps
its slot, and Explain has its own hourly count per account and its own
on/off switch in inbuxa:AiLimits. The permission is sysAiExplain,
superuser only; tenant administrators can't use it. The session carries
an aiExplain flag so a console knows when to offer the button.

An install whose roles were stored before the permission existed gets it
added once, at start-up, to the roles that are administrators' alone,
not the User role their defaults share with every account. An operator
who removes it later isn't overruled.

Tests: unit tests in inbuxa-features and jmap, and ai_explain_tests
(run with --ignored) covering the acceptance tests and the upgrade.
2026-09-26 00:52:51 -07:00
jcoffey-dev 95f0445d83 Coordinator: join the cluster when NATS comes up, report the connection
ci / fork-checks (pull_request) Successful in 46s
ci / build (pull_request) Successful in 11m8s
A node that started while NATS was down never got a coordinator. The
connect failed at boot, bootstrap recorded a build error and the node ran
with Coordinator::None until restarted. It had no broadcast subscriber
or publisher, so cross-node push and cache invalidation to it stayed
broken, and its healthcheck said nothing about it. Losing NATS after
startup was silent too.

- The NATS client now connects in the background
  (retry_on_initial_connect): startup never waits on NATS or fails over
  it, the node gets its coordinator, subscriber and publisher at once,
  and the client keeps trying (async-nats's backoff, at most 4 s apart)
  until NATS answers. Subscriptions made meanwhile start delivering when
  it does. A configured maxReconnects still ends the attempts.
- Three new events report the connection: cluster.coordinator-connected
  (info), cluster.coordinator-disconnected (warn: lost, closed, gave up,
  or not connected within the connection timeout at startup) and
  cluster.coordinator-error (warn: a failed attempt, reported once per
  outage rather than every retry, and server errors, slow consumers and
  lame duck mode). They are in the packaged schema, ids 644 to 646.
- GET /healthz/cluster reports the coordinator: 200
  {"coordinator":"connected"}, 503 {"coordinator":"disconnected"}, or
  200 with "none" (no coordinator) or "unknown" (a backend that doesn't
  track its connection). /healthz/live and /healthz/ready are unchanged
  on purpose: a node without its coordinator still serves mail, and
  failing those would have orchestrators restart, or pull out of
  service, every node at once whenever NATS is down.

Only NATS connects lazily; the other coordinator backends still fail at
boot as before.

cluster::coordinator::coordinator_reconnect_tests starts a node against a
NATS port with nothing behind it, checks it boots with a coordinator and
reports it disconnected, subscribes, then starts NATS on that port: the
node connects on its own and the subscription receives a message from a
second client. Stopping and restarting NATS shows disconnected, then
connected, and the same subscription keeps working.
2026-09-24 08:38:59 -07:00
jcoffey-dev eba4c7a32e Settings › Spam Filter gains Local AI, the AI spam filtering setup page
ci / fork-checks (pull_request) Successful in 14s
ci / build (pull_request) Successful in 4m23s
Adds a link to CustomComponent/LocalAi in the packaged schema's Settings ›
Spam Filter, above LLM Classifier, and updates the schema hash so admins
fetch the new layout rather than a cached one.

INBUXA Admin draws the page (feature/local-ai-setup); this makes it
reachable. An admin from before that page would show "Unknown component"
here, so this lands after the admin release that carries it.
2026-09-22 22:08:50 -07:00
jcoffey-dev 17426f6d60 Bundle the spam filter rules with the server
The server fetched upstream's latest published rules from GitHub at run
time: a version nobody here tested, code-like expressions from an account
we don't control, and the upstream name as a default in the admin form.

The published rules of spam-filter v3.0.2 are now embedded
(resources/spam-filter/, MIT, in THIRD-PARTY.md) and used whenever no other
source is configured. An empty setting and upstream's old default both mean
the bundled rules, so existing installs switch without a settings change;
the URL stays an operator override (https:// or file://). The schema default
is dropped and its description says what empty means, and the strip's
rename pass does the same to each import.

Rules load on first boot as before, and again whenever the bundled version
differs from the last one loaded, which only adds missing rules and tags.
That brings the AI classifier's LLM_* scores to installs that predate them:
production has none today.

upstream-watch now also opens an issue when spam-filter publishes a newer
release; resources/spam-filter/README.md says how to take it.

The antispam test now runs on the bundled rules, the path production
takes; SPAM_RULES_URL tests another set. Unit tests cover the URL handling
and that the bundled rules parse and score the AI tags as the AI spec says.
2026-09-22 22:01:30 -07:00
jcoffey-dev a993f9ab01 Write the brand in lowercase where people see it
ci / fork-checks (pull_request) Successful in 47s
ci / build (pull_request) Successful in 5m4s
The name is inbuxa, lowercase, like the wordmark; INBUXA reads as an
acronym. The admin and webmail already changed. Here that's everything
the server shows people: the brand macro behind the protocol greetings,
the HTTP and SCIM realms, the startup banner and the calendar and contact
PRODID; the first-party OAuth client descriptions; the legacy-protocol
refusals; the default calendar and address book names and the SMTP
greeting default, in the code and the schema served to the admin
(checksum regenerated); startup and shutdown events; the User-Agent;
the sign-in and RSVP pages; the service units; the OpenAPI realm; the
crate descriptions and the README, where it's set in bold.

Identifiers that are uppercase for their own reasons stay: INBUXA_*
settings, SUBSPACE_INBUXA. So do code comments and the AGPL 5(a) notice
lines.

Tests follow: the IMAP ID name, the default collection names, the PRODID
in the iTIP fixtures and the CalDAV free-busy expectations, and the e2e
legacy-protocol refusals. The webdav, imap and jmap suites pass, so do
the unit tests of every crate touched, and 73 of 75 SMTP tests; of the
other two, antispam fails on main too, and queue_retry is a timing flake
that passes on its own.
2026-09-22 20:08:10 -07:00
jcoffey-dev cc6f1eb298 Rename the identifiers that carried the upstream name
ci / fork-checks (pull_request) Successful in 16s
ci / build (pull_request) Successful in 7m53s
Everything clients, users and operators meet now carries the fork's name,
with no aliases (SPEC.md §2.4, changed here from "protocol identifiers
stay"):

- JMAP: upstream's registry capability is urn:inbuxa:jmap:registry, beside
  the fork's own urn:inbuxa:jmap.
- WebDAV lock and sync tokens are urn:inbuxa:dav*; clients resync once.
- Sieve: vnd.inbuxa.while and vnd.inbuxa.expressions. sieve-rs spells these
  into its compiler, so it's vendored (vendor/sieve-rs, 0.7.3) and patched in;
  a unit test fails if Cargo.lock ever moves past the vendored copy. The
  trusted runtime now names itself too, rather than answering sieve-rs's
  default.
- The web interface's OAuth client is inbuxa-webui. On every start the old
  stalwart-webui client is removed and any application naming it is moved
  over.
- The spam filter's blobs are INBUXA_SPAM_*; every start moves any left
  under the old keys, so a trained model survives.
- SQL stores and log files default to inbuxa, in the code and in the
  schema served to the admin (checksum regenerated).
- Settings are INBUXA_* only. A STALWART_* variable that's set where its
  INBUXA_* one isn't stops the server at startup, naming it.
- The version-upgrade messages link docs.inbuxa.org's migration page, and
  the OpenAPI description, smtp crate metadata and web-push test fixtures
  lose the name.

Kept on purpose, allowlisted with reasons: the OAuth key-derivation
contexts (renaming them would end every session and invalidate every
sealed client id) and the hashed application prefix.

Also fixes a latent start-up failure: ensure_client updated an existing
first-party client with a revision of 0, which the registry's assertion
never matches, so adding a redirect URI or changing the webmail secret
failed start-up. And the principal session test now expects
legacyProtocols (C-1, added 2026-09-21), which it had missed.

Tested: the server builds without warnings; common's 106 unit tests,
including the vendoring check; a new integration test for the two
start-up migrations; and the webdav, jmap, imap and SMTP Sieve suites.
2026-09-22 19:33:02 -07:00
jcoffey-dev 79b6787397 Merge main, and put LP-8's event on top of the Hardening link's schema
The Hardening link merged to main changed the packaged schema, which this
branch also changes. The file is gzipped, so the two can't be merged line
by line: this takes main's schema and adds security.legacy-protocols-changed
to it again, with the hash recomputed.
2026-09-21 13:48:25 -07:00
jcoffey-dev 64cddc9246 The switch reports every change as an event (LP-8)
Turning legacy mail protocols off or back on raises
security.legacy-protocols-changed (id 643, info level, also in the packaged
schema), with the scope (policy = server), the new value, who made the
change (accountId), whether listeners closed or reopened (details), which
ones (listenerId), and -- only when a listener could not be put back --
which and why (reason).

It is raised in Server::set_protocol_policy rather than by the JMAP
method, so whatever turns the switch is reported. A /set that changes
nothing -- the switch already where it was asked to be, nothing to close
or reopen -- is not a change and raises nothing.

The event is never an error, but jmap's exhaustive map from security
events to HTTP errors has to name it; it joins the other two that can't
occur there. rustfmt now also wraps LP-6's two over-long lines in
enums_impl.rs, which it flagged along with this change's.

tests/e2e/legacy_protocols.py now gives the server a stdout tracer and
reads events from the container's log: turning the switch off is exactly
one event naming the scope, value, author and listeners closed; setting it
off again raises none; turning it on is one event naming the listeners
reopened. It also proves LP-6's side: seven refused submission sign-ins
are seven auth.legacy-protocol-refused events, and there is no auth.failed
or auth.too-many-attempts among them. All checks pass.
2026-09-21 10:53:01 -07:00
jcoffey-dev 30be928e14 Merge main, and put the Hardening link on top of LP-6's schema
LP-6 added auth.legacy-protocol-refused to the packaged schema, which this
branch also changes. The file is gzipped, so the two can't be merged line
by line: this takes main's schema and adds the Settings › Security ›
Hardening link to it again, with the hash recomputed.
2026-09-21 10:21:01 -07:00
jcoffey-dev 04252000da Legacy sign-in is refused while the switch is off (LP-6)
The second lock. While legacy mail protocols are off, a sign-in over IMAP,
POP3, ManageSieve or SMTP AUTH is refused for every account, so a listener
that exists by mistake -- or submission, which the SMTP lock keeps open --
still lets nobody in.

The check sits at the top of each protocol's sign-in, before the
credentials are looked at. So the answer is the same for a right password,
a wrong one and an account that doesn't exist; it isn't auth.failed, so it
counts nothing against the account and never feeds the auto-ban; and the
session stays open, since the mail app is being told, not thrown off.

Mail apps read the spec's words (LP-12, at server scope):

  IMAP         NO [ALERT] This server allows only INBUXA webmail and JMAP
               apps. This mail app can't sign in.
  POP3         -ERR [AUTH] ...the same...
  ManageSieve  NO "This server allows only INBUXA webmail and JMAP apps."
  SMTP         535 5.7.0 This server allows only INBUXA webmail and JMAP
               apps. This mail app can't send.

SMTP AUTH is refused on every SMTP listener, port 25 included: only mail
apps authenticate, so inbound delivery is untouched. LMTP is left alone.

The policy is read from the store on each sign-in rather than cached, so
every node of a cluster answers the same the moment the switch turns.

Each refusal raises a new event, auth.legacy-protocol-refused (id 642, info
level, also in the packaged schema), with the protocol as source, the
policy's scope and the domain -- never the account. The session adds the
listener and remote IP.

tests/e2e/legacy_protocols.py now also proves, on a running server: a
normal IMAP and submission sign-in works with the switch on, before and
after; while off, submission refuses the right password and six wrong ones
with the same words and without hanging up; and an IMAP listener created by
mistake while off refuses the right password, a wrong one and an account
that doesn't exist. All 33 checks pass. SMTP sign-ins in the script wait
out a second first: every connection arrives from Docker's gateway, and the
stock inbound throttle takes five a second from one IP.
2026-09-21 09:49:40 -07:00
jcoffey-dev 1a48474957 Settings › Security gains Hardening, the legacy protocols screen
Adds a link to CustomComponent/LegacyProtocols in the packaged schema's
Settings › Security, between Settings and Blocked IPs, and updates the
schema hash so admins fetch the new layout rather than a cached one.

INBUXA Admin draws the screen; this is what makes it reachable. An admin
from before that screen would show "Unknown component" here, so this
lands after the admin release that carries it.
2026-09-21 09:19:49 -07:00
jcoffey-dev 0ca26070d7 SCIM: users, groups, queries, PATCH, Bulk and cursors at /scim/v2, over x:Account (SCIM-1 to SCIM-57)
Every SCIM operation becomes the x:Account get, query or set JMAP makes,
as the service principal, so permissions, tenant scope and limits,
address uniqueness and account destruction are enforced in one place.
Discovery is anonymous; everything else takes an API key as a bearer
token and nothing else. Domains open to SCIM carry a flag in the domain
cache. Filters take eq and and, answered from the account indexes, with
unindexed attributes checked on at most 200 candidates. Cursors are
stateless, HMAC-sealed under the server key. PATCH applies to the
resource in memory and saves it as a PUT, so it is all or nothing.
Groups get an address from their display name on the principal's
domain; membership is written on each user.

Every write emits one of five new scim.* events (ids 637 to 641), also
added to the packaged schema. The helpers the surviving SCIM suites
import are rebuilt from the spec; scim_tests runs the new acceptance
suite and the surviving tenant isolation suite, and both pass.
2026-09-19 09:35:23 -07:00
jcoffey-dev 0bc6b03dcd Branding and templates: per-domain, tenant and server logos, /logo, operator calendar email templates and RSVP page (BT-1 to BT-26)
Logos resolve domain, then tenant, then server-wide, then the built-in, with
subdomains finding their domain. GET /logo serves a data-URL image, redirects
to a URL logo without fetching it, sandboxes SVG, and answers 404 when no
custom logo applies. Emails embed the first PNG, JPEG or GIF logo. Logo and
template writes are checked; stored templates are read at send time, always
escaped, and fall back to the built-in with a build warning when they don't
parse. The RSVP page is served byte for byte with a CSP and no-referrer. The
sign-in and RSVP pages load the logo through an image element. MT-22's
session logo follows the chain to the server-wide logo.
Acceptance tests 1 to 17; test 18 written as the ignored branding_compat.
2026-09-18 22:27:19 -07:00
jcoffey-dev 31e64f5a4d Rebrand the packaged schema: defaults, labels and descriptions; the Enterprise page becomes Branding; log prefix inbuxa.log 2026-09-18 13:15:17 -07:00
jcoffey-dev d6fc4600cd OAuth: registration required by default; first-party clients registered on every start (contract C-5, C-6) 2026-09-18 13:04:20 -07:00
jcoffey-dev d3f0b36dd2 Packaging: the binary and package are inbuxa, INBUXA_* settings with STALWART_* fallback
- crates/main: package and [[bin]] renamed to inbuxa; homepage inbuxa.org;
  license AGPL-3.0-only (upstream is dual; the fork takes the AGPL).
- types::branding::env_var reads INBUXA_<name>, falling back to
  STALWART_<name> with a warning, for all nine server settings.
  STALWART_APP_ and STALWART_SPAM_* storage keys are unchanged.
- New-install default paths /var/lib/inbuxa and /var/log/inbuxa.
- Dockerfiles, systemd unit, launchd plist and AppArmor profile renamed.
- Upstream's .github moved to .github-upstream so none of it runs.
- install.sh stubbed: upstream's would install Stalwart.
- Two missed brand strings: the SMTP Received header and the utils user agent.
2026-09-18 11:09:22 -07:00
jcoffey-dev 189f688768 Sign-in page: show the logo when the page is opened without redirect_uri
The early return for a missing redirect_uri came before the logo loader,
so the logo stayed hidden behind data-loading. An upstream bug, surfaced by
the rebrand check.
2026-09-18 10:58:39 -07:00