Commit Graph
19 Commits
Author SHA1 Message Date
jcoffey-dev 17426f6d60 Bundle the spam filter rules with the server
The server fetched upstream's latest published rules from GitHub at run
time: a version nobody here tested, code-like expressions from an account
we don't control, and the upstream name as a default in the admin form.

The published rules of spam-filter v3.0.2 are now embedded
(resources/spam-filter/, MIT, in THIRD-PARTY.md) and used whenever no other
source is configured. An empty setting and upstream's old default both mean
the bundled rules, so existing installs switch without a settings change;
the URL stays an operator override (https:// or file://). The schema default
is dropped and its description says what empty means, and the strip's
rename pass does the same to each import.

Rules load on first boot as before, and again whenever the bundled version
differs from the last one loaded, which only adds missing rules and tags.
That brings the AI classifier's LLM_* scores to installs that predate them:
production has none today.

upstream-watch now also opens an issue when spam-filter publishes a newer
release; resources/spam-filter/README.md says how to take it.

The antispam test now runs on the bundled rules, the path production
takes; SPAM_RULES_URL tests another set. Unit tests cover the URL handling
and that the bundled rules parse and score the AI tags as the AI spec says.
2026-09-22 22:01:30 -07:00
jcoffey-dev cc6f1eb298 Rename the identifiers that carried the upstream name
ci / fork-checks (pull_request) Successful in 16s
ci / build (pull_request) Successful in 7m53s
Everything clients, users and operators meet now carries the fork's name,
with no aliases (SPEC.md §2.4, changed here from "protocol identifiers
stay"):

- JMAP: upstream's registry capability is urn:inbuxa:jmap:registry, beside
  the fork's own urn:inbuxa:jmap.
- WebDAV lock and sync tokens are urn:inbuxa:dav*; clients resync once.
- Sieve: vnd.inbuxa.while and vnd.inbuxa.expressions. sieve-rs spells these
  into its compiler, so it's vendored (vendor/sieve-rs, 0.7.3) and patched in;
  a unit test fails if Cargo.lock ever moves past the vendored copy. The
  trusted runtime now names itself too, rather than answering sieve-rs's
  default.
- The web interface's OAuth client is inbuxa-webui. On every start the old
  stalwart-webui client is removed and any application naming it is moved
  over.
- The spam filter's blobs are INBUXA_SPAM_*; every start moves any left
  under the old keys, so a trained model survives.
- SQL stores and log files default to inbuxa, in the code and in the
  schema served to the admin (checksum regenerated).
- Settings are INBUXA_* only. A STALWART_* variable that's set where its
  INBUXA_* one isn't stops the server at startup, naming it.
- The version-upgrade messages link docs.inbuxa.org's migration page, and
  the OpenAPI description, smtp crate metadata and web-push test fixtures
  lose the name.

Kept on purpose, allowlisted with reasons: the OAuth key-derivation
contexts (renaming them would end every session and invalidate every
sealed client id) and the hashed application prefix.

Also fixes a latent start-up failure: ensure_client updated an existing
first-party client with a revision of 0, which the registry's assertion
never matches, so adding a redirect URI or changing the webmail secret
failed start-up. And the principal session test now expects
legacyProtocols (C-1, added 2026-09-21), which it had missed.

Tested: the server builds without warnings; common's 106 unit tests,
including the vendoring check; a new integration test for the two
start-up migrations; and the webdav, jmap, imap and SMTP Sieve suites.
2026-09-22 19:33:02 -07:00
jcoffey-dev ee4988e00d Mark eight more changed files (AGPL section 5(a))
These upstream files were changed after the fork marked the files it had
modified, and never got the notice: six by the listener and schema-cache
work on 2026-09-20, two by the name check. Found by diffing against the
upstream snapshot branch, as before.
2026-09-22 16:57:15 -07:00
jcoffey-dev b2ded0a776 Merge upstream v0.16.23
Five conflicts, resolved:

- crates/common/src/auth/authentication.rs: upstream's get_directory_for_token
  and JwtClaims replace extract_jwt_domain; the per-domain directory code
  (DIR-1, DIR-5 to DIR-7) is kept, and the token lookup routes through it.
  The release's one new Enterprise snippet was the body of
  get_directory_for_issuer, which stays returning None: a token naming no
  address gets the server default, as DIR-2 specifies and as v0.16.22 did.
- crates/common/src/manager/application.rs: upstream's rewrite of the tests,
  with the temp directory names renamed again, and the 5(a) notice the
  name-purge change should have added.
- crates/common/src/network/mta.rs: both sides' imports.
- crates/main/Cargo.toml: the AGPL-only license kept, version 0.16.23.
- Cargo.lock: upstream's, with the fork's crates added by Cargo.
2026-09-22 16:57:06 -07:00
jcoffey-dev 3a272096c0 Import upstream v0.16.23, stripped
trivy / Check (pull_request) Waiting to run
Upstream commit: 9d1c75ab68435e4417337f768291e5f947686203
Enterprise-only files removed or emptied: 63
Enterprise-only snippets removed: 118 in 50 files
Dangling module declarations removed: 5
Edits turning enterprise off: 25
Third-party code: 14 files, 0 not in THIRD-PARTY.md
Verification: clean

One snippet more than v0.16.22, in crates/common/src/auth/authentication.rs
(3, was 2).
2026-09-22 16:31:25 -07:00
jcoffey-dev 7f14992e81 Keep the upstream project's name out of user-visible strings
ci / build (pull_request) Successful in 3m46s
The first-party application descriptions and the telemetry service name and
instrumentation scope are shown to operators, and the unpacked-application
temp directory carried the name too.

Left alone deliberately: the OAuth key-derivation contexts (renaming them
would invalidate every sealed token and client id), the migration defaults
that read an upstream installation, links to upstream's upgrade guide, the
wire-protocol identifiers, and upstream's own license and templates.
2026-09-22 13:21:15 -07:00
jcoffey-dev 4b585905d7 Nothing advertises the legacy protocols while they are off (LP-7)
While the switch is off, the answers that tell a mail app where to connect
stop offering what the switch closed, so a new phone or desktop app is not
sent to a port that is shut or a sign-in that will be refused:

- Thunderbird-style autoconfig (/mail/config-v1.1.xml and its other
  paths) and Outlook autodiscover leave out IMAP, POP3 and SMTP
  submission.
- PACC (/.well-known/user-agent-configuration.json) offers JMAP, CalDAV,
  CardDAV and WebDAV, and no IMAP, POP3, SMTP or ManageSieve. The document
  is rendered once per configuration load, so the JMAP-only version is
  rendered beside it and chosen per request; the _ua-auto-config digest in
  the suggested zone follows, since it hashes the same document.
- The suggested zone publishes _imap, _imaps, _pop3, _pop3s, _submission
  and _submissions with target "." -- "not offered", RFC 6186 section 3.4 --
  the spec's decision, rather than dropping them: a client that looks is
  told, and an automatically managed zone replaces the old records instead
  of leaving them behind.
- It also drops the TLSA records for ports 993 and 995. A TLS pin for a
  port the switch has closed advertises a service that is not there.
  Submission's 465 keeps its record: the SMTP lock keeps that port open.

The switch is read per answer, as sign-in reads it, so every node agrees
the moment it turns. Inbound mail, MX records and the JMAP, CalDAV and
CardDAV answers are untouched.

tests/e2e/legacy_protocols.py checks all four on a running server: with the
switch on they offer IMAP, POP3 and SMTP (the control); while it is off
they offer none of them and every legacy SRV name has target "."; and once
it is back on, autoconfig and the zone read as they did before. All checks
pass.
2026-09-21 10:41:20 -07:00
jcoffey-dev fee6b74e79 Listeners can be stopped one at a time, which LP-2 needs
The legacy-protocols switch has to close the IMAP, POP3 and ManageSieve
ports and leave everything else accepting. The server could not do that.

Two findings from the source, both now recorded in the spec. A settings
reload never closes a port: cache/reload.rs parses the listeners only to
collect configuration errors and drops the result, and sockets are bound
once at startup through init.servers.spawn in main.rs. And there is only
one shutdown signal -- Listeners::spawn makes a single watch channel and
hands every listener a clone -- so the one thing the server could do was
stop all of them at once, port 25 included. That answers the spec's open
question 1, and the answer was neither of the two it offered.

So each listener gets its own channel. ListenerControl holds the sending
ends keyed by listener id; firing one breaks that accept loop, which drops
its TcpListener and closes the socket. The accept loop itself is unchanged
-- it already did the right thing, it just had no way to be told about one
listener. stop_matching takes a predicate and a keep list, because the
inbound listener shares its protocol with submission and telling them
apart is the caller's job (LP-3), not this registry's.

spawn_with_control is a second method rather than a change to spawn. The
registry owns the senders, so a dropped registry would stop every listener
at once; the four test callers pass no registry and keep the old shared
channel exactly as it was.

Whole-server shutdown now fires the per-listener channels too, since the
returned sender no longer reaches them.

No policy, no JMAP and no screen yet: this is only the mechanism, with
seven tests over stopping one, stopping many, sparing port 25 and sparing
submission. It closes no port on its own, and it does not touch the host's
firewall or any port-forward -- that is LP-20, and stays the operator's.
2026-09-20 15:10:41 -07:00
jcoffey-dev 6a53d47106 Mark the files this fork changed (AGPL section 5(a))
The AGPL asks a modified version to carry prominent notices saying it was
modified, and giving a date. Publishing the source is the conveyance that
asks for it, so it wants doing before the repository is public rather than
at the release.

Every upstream file the fork changed now says so in its header, beneath the
notice it came with: 164 files, found by diffing against the upstream
snapshot branch rather than by guessing, so the list is what actually
differs. Files the fork wrote itself already carry their own copyright and
need nothing. Upstream's notices are untouched, which its licence requires
and which was already true.

The README says the same thing in prose, since the obligation is on the
work as a whole and not only its Rust files.

Builds unchanged: the server and the test binary both compile.
2026-09-19 23:48:35 -07:00
jcoffey-dev 1518c69033 Scale-out storage: PostgreSQL and MySQL read replicas (ST-5 to ST-15)
A data store with readReplicas becomes a replicated store. Writes,
operator-written SQL and everything outside a read scope go to the
primary. JMAP reads before a request's first write, IMAP LIST, STATUS,
SEARCH, SORT and FETCH, POP3 RETR and TOP, DAV GET, PROPFIND and REPORT,
and blob downloads run in a read scope. Only account data (properties,
indexes, change logs, counters, ACLs, blobs, the search index) is read
from a replica; the registry, in-memory values, the task queue and the
rest stay on the primary.

In a scope, the first read picks a replica round-robin among those up
and under the lag limit, and only if it has every change this node has
written or heard of for the scope's accounts: marks come from write
results, the cluster's state-change broadcasts, a sinceState the client
presents, and, with more than one node, Redis. A write inside the scope
sends the rest of it to the primary. A miss on a replica is looked up on
the primary, and a replica error retries the read there and marks the
replica down.

Each node samples lag every second (WAL positions on PostgreSQL; GTID
sets or Seconds_Behind_Source on MySQL), stops reading from a replica
over 5 s and starts again under 2.5 s, and probes a down replica every
10 s. At startup a replica is left out if it's the primary, isn't
read-only, applies out of commit order, or doesn't show a marker written
to the primary within six tries.

replica_tests (postgres, STORE=PostgreSqlReplicated) runs a primary and a
streaming hot standby in containers: tests 9, 10, 12, 13, 14 and 15 pass.
2026-09-19 14:05:59 -07:00
jcoffey-dev fc2d4f237f Monitoring: trace history, trace search, and x:Trace over the stored traces (MON-10 to MON-17, MON-32)
A lossy collector subscriber keeps each inbound SMTP session that reached
MAIL FROM and each delivery attempt, info level and above and never raw
I/O, at most 1,000 events with strings cut at 4 KiB, and writes it when
the span closes as an x:Trace under the telemetry key class, scheduling
its indexing. The index task builds a document of event types, queue ids
and keywords when indexTelemetry is on. x:Trace/get derives timestamp,
from, to and size; /query filters by opening event, text, queueId and
time; /set destroys only. The data purge honours holdTracesFor. The shared
tracing and webhook suites run.
2026-09-19 08:31:04 -07:00
jcoffey-dev 9490fc4677 AI spam classification: the model's opinion as one bounded spam signal, and the llm_prompt Sieve function (AI-1 to AI-28)
The classifier sends only the subject and text, between unforgeable markers
after the operator's prompt, to an OpenAI-compatible endpoint the operator
configured; nothing is preset. Its answer maps to an LLM_ tag whose score is
clamped (+5.0, -1.0 by default) and can never discard or reject on its own;
X-Spam-LLM is sanitized, encoded and folded, and a planted one is removed.
Failures, timeouts past the ceiling, a full slot or a paused model leave
mail flowing untagged. llm_prompt answers trusted scripts, and accounts
holding interactAi within an hourly limit. Redirects aren't followed and no
content or secret is logged. The limits live in inbuxa:AiLimits.
Acceptance tests 1 and 3 to 21; test 2 as the re-enabled shared llm case,
whose setup no longer waits on a rules file from a developer's own path;
test 22 written as the ignored ai_compat.
2026-09-19 00:41:00 -07:00
jcoffey-dev 0bc6b03dcd Branding and templates: per-domain, tenant and server logos, /logo, operator calendar email templates and RSVP page (BT-1 to BT-26)
Logos resolve domain, then tenant, then server-wide, then the built-in, with
subdomains finding their domain. GET /logo serves a data-URL image, redirects
to a URL logo without fetching it, sandboxes SVG, and answers 404 when no
custom logo applies. Emails embed the first PNG, JPEG or GIF logo. Logo and
template writes are checked; stored templates are read at send time, always
escaped, and fall back to the built-in with a build warning when they don't
parse. The RSVP page is served byte for byte with a CSP and no-referrer. The
sign-in and RSVP pages load the logo through an image element. MT-22's
session logo follows the chain to the server-wide logo.
Acceptance tests 1 to 17; test 18 written as the ignored branding_compat.
2026-09-18 22:27:19 -07:00
jcoffey-dev 8c1879e853 Cross-origin requests only from the front ends' origins (contract C-14) 2026-09-18 13:44:55 -07:00
jcoffey-dev faedf7a1da Versioning: INBUXA's own dated version, with the Stalwart base shown
inbuxa --version, the banner, startup events, OpenTelemetry and the JMAP
implementation string read "2026.9.18 (Stalwart 0.16.22)". The base comes
from Cargo, which keeps following upstream so version bumps merge cleanly.
Received headers and IMAP ID carry the INBUXA version.
2026-09-18 12:08:41 -07:00
jcoffey-dev c9c761fab5 Quick fixes from the first boot: recovery admin, upsell, warnings
- The recovery administrator (INBUXA_RECOVERY_ADMIN, or STALWART_RECOVERY_ADMIN)
  is honored only in bootstrap and recovery mode. On a configured server it's
  ignored with a startup warning. Before, it was a standing full-admin login
  for as long as the variable stayed set.
- The Enterprise upsell error is replaced by "This feature isn't available in
  INBUXA yet" for the features still to be rebuilt.
- Workspace warnings: 25 to 0. cargo fix removed the unused imports. The
  seven places where Enterprise code used to plug in keep their parameters,
  each with an inbuxa: comment naming the rebuild that uses it again. The
  antispam test's mock-server imports are back behind pending-rebuild.
2026-09-18 11:36:44 -07:00
jcoffey-dev d3f0b36dd2 Packaging: the binary and package are inbuxa, INBUXA_* settings with STALWART_* fallback
- crates/main: package and [[bin]] renamed to inbuxa; homepage inbuxa.org;
  license AGPL-3.0-only (upstream is dual; the fork takes the AGPL).
- types::branding::env_var reads INBUXA_<name>, falling back to
  STALWART_<name> with a warning, for all nine server settings.
  STALWART_APP_ and STALWART_SPAM_* storage keys are unchanged.
- New-install default paths /var/lib/inbuxa and /var/log/inbuxa.
- Dockerfiles, systemd unit, launchd plist and AppArmor profile renamed.
- Upstream's .github moved to .github-upstream so none of it runs.
- install.sh stubbed: upstream's would install Stalwart.
- Two missed brand strings: the SMTP Received header and the utils user agent.
2026-09-18 11:09:22 -07:00
jcoffey-dev 89665decfa Rebrand to INBUXA: product name, protocol greetings, sign-in page, calendar templates, logos, README
One branding module (types::brand!) supplies the name to every protocol
greeting, the HTTP realm, the startup banner, Received headers, the
user agent, the IMAP ID response and calendar PRODIDs. The sign-in and
calendar pages take ihasmail's palette and font and the INBUXA logo, and
calendar emails embed the INBUXA lockup. Default calendar and address book
names follow. Protocol identifiers (urn:stalwart:jmap, vnd.stalwart Sieve
extensions) and upstream copyright notices are unchanged.
2026-09-18 10:52:34 -07:00
jcoffey-dev 7dae9b29fd Import upstream v0.16.22, stripped
Upstream commit: 474dd0229cb20cf513036619781ed97bd8073c3f
Enterprise-only files removed or emptied: 63
Enterprise-only snippets removed: 117 in 50 files
Dangling module declarations removed: 5
Cargo edits turning enterprise off: 14
Verification: clean
Enterprise feature gates left for rebuilt features: 19 in 18 files

Produced by tools/fork/strip.py. The full report is in docs/fork/strip-reports/ on main.
2026-09-18 10:21:56 -07:00