10 Commits
Author SHA1 Message Date
jcoffey-dev 7735780807 Merge pull request 'Image build: put the vendored crate where cargo chef cooks; release 2026.9.24.3' (#30) from fix/image-vendor-before-cook into main
publish / release (push) Blocked by required conditions
publish / binaries (push) Blocked by required conditions
ci / fork-checks (push) Successful in 41s
publish / version (push) Successful in 39s
publish / publish (push) In progress
ci / build (push) Successful in 36m33s
Reviewed-on: #30
2026-09-23 06:10:46 +00:00
jcoffey-dev e223f7d327 Release 2026.9.24.3
ci / fork-checks (pull_request) Successful in 45s
ci / build (pull_request) Successful in 4m26s
2026.9.24.3 is 2026.9.24.2 plus the image build fix; 2026.9.24.2's tag never
published an image. Everything in 2026.9.24.2's notes applies.
2026-09-22 23:04:59 -07:00
jcoffey-dev 30df055e39 Image build: put the vendored crate where cargo chef cooks
#27 let the build context see vendor/, but the Dockerfile cooks the
dependencies before it copies the tree, from a recipe that carries only the
workspace's manifests. [patch.crates-io] points sieve-rs at vendor/, so the
cook failed the same way: failed to read /build/vendor/sieve-rs/Cargo.toml.
That's why 2026.9.24.2's publish failed. The builder stage now copies
vendor/ before cooking; a local build got past it into compiling the
dependencies.

context-check.py now also checks that each patched path is copied into the
cooking stage before the cook, and fails on the Dockerfile as it was.
2026-09-22 23:04:50 -07:00
jcoffey-dev 5393c4405a Merge pull request 'Release 2026.9.24.2' (#29) from release/2026.9.24.2 into main
ci / fork-checks (push) Successful in 50s
publish / version (push) Successful in 24s
publish / publish (push) Failing after 2m38s
publish / release (push) Skipped
publish / binaries (push) Skipped
ci / build (push) Successful in 22m35s
Reviewed-on: #29
2026-09-23 05:47:10 +00:00
jcoffey-dev cc532b914c Release 2026.9.24.2
ci / fork-checks (pull_request) Successful in 1m49s
ci / build (pull_request) Successful in 4m8s
Replaces 2026.9.24, whose tag predates the image build fix (#27) and never
published. Carries everything 2026.9.24 did -- upstream 0.16.23 and its
fixes, the scim release-profile fix -- and since then:

- identifiers renamed from the upstream name, with no aliases: the JMAP
  registry capability is urn:inbuxa:jmap:registry, WebDAV tokens
  urn:inbuxa:dav*, Sieve extensions vnd.inbuxa.*, the web interface client
  inbuxa-webui; INBUXA_* settings only. Deploy with admin and webmail
  releases that use the new names.
- the brand in lowercase where people see it.
- the spam filter rules bundled with the server; on first start they add
  the AI classifier's LLM_* scores.
- a Local AI page link in Settings › Spam Filter, for the admin release
  that draws it.
- two start-up migrations: the spam model moves to its renamed keys, and
  the web interface's old OAuth client is retired.
2026-09-22 22:40:17 -07:00
jcoffey-dev c09eff2214 Merge pull request 'Bundle the spam filter rules with the server, and link the Local AI page' (#28) from fork/bundled-spam-rules into main
ci / fork-checks (push) Successful in 20s
ci / build (push) Canceled after 7m16s
Reviewed-on: #28
2026-09-23 05:39:50 +00:00
jcoffey-dev eba4c7a32e Settings › Spam Filter gains Local AI, the AI spam filtering setup page
ci / fork-checks (pull_request) Successful in 14s
ci / build (pull_request) Successful in 4m23s
Adds a link to CustomComponent/LocalAi in the packaged schema's Settings ›
Spam Filter, above LLM Classifier, and updates the schema hash so admins
fetch the new layout rather than a cached one.

INBUXA Admin draws the page (feature/local-ai-setup); this makes it
reachable. An admin from before that page would show "Unknown component"
here, so this lands after the admin release that carries it.
2026-09-22 22:08:50 -07:00
jcoffey-dev 17426f6d60 Bundle the spam filter rules with the server
The server fetched upstream's latest published rules from GitHub at run
time: a version nobody here tested, code-like expressions from an account
we don't control, and the upstream name as a default in the admin form.

The published rules of spam-filter v3.0.2 are now embedded
(resources/spam-filter/, MIT, in THIRD-PARTY.md) and used whenever no other
source is configured. An empty setting and upstream's old default both mean
the bundled rules, so existing installs switch without a settings change;
the URL stays an operator override (https:// or file://). The schema default
is dropped and its description says what empty means, and the strip's
rename pass does the same to each import.

Rules load on first boot as before, and again whenever the bundled version
differs from the last one loaded, which only adds missing rules and tags.
That brings the AI classifier's LLM_* scores to installs that predate them:
production has none today.

upstream-watch now also opens an issue when spam-filter publishes a newer
release; resources/spam-filter/README.md says how to take it.

The antispam test now runs on the bundled rules, the path production
takes; SPAM_RULES_URL tests another set. Unit tests cover the URL handling
and that the bundled rules parse and score the AI tags as the AI spec says.
2026-09-22 22:01:30 -07:00
jcoffey-dev d7c9416713 Merge pull request 'Let the image build see the dependency Cargo patches' (#27) from fix/vendor-in-build-context into main
ci / fork-checks (push) Successful in 14s
ci / build (push) Successful in 31m11s
2026-09-23 04:48:35 +00:00
jcoffey-dev 238079da66 Let the image build see the dependency Cargo patches
ci / fork-checks (pull_request) Successful in 49s
ci / build (pull_request) Successful in 4m22s
The rename pass vendored a patched sieve-rs and pointed Cargo.toml's
[patch.crates-io] at vendor/sieve-rs. .dockerignore ignores everything and
re-includes a short list that did not have vendor on it, so the image build
had no such directory and stopped at

    failed to load source for dependency `sieve-rs`
    failed to read /build/vendor/sieve-rs/Cargo.toml

CI could not have caught that: it builds from a checkout, where the
directory is simply there, and only the image build has a context to prune.
The first that was known about it was a tag that had already been pushed.

So: vendor is re-included, and tools/fork/context-check.py now asserts the
thing that was quietly assumed -- every path a [patch] section names exists
and survives .dockerignore. It runs beside the other fork checks and takes
no toolchain.

Also, the comments in .dockerignore started with // , which Docker does not
read as a comment: they were patterns that happened to match nothing. They
are # now.
2026-09-22 21:43:37 -07:00
21 changed files with 372 additions and 45 deletions
+8 -2
View File
@@ -1,10 +1,16 @@
// Ignore everything
# Ignore everything
*
// Allow what is needed
# Allow what is needed
!crates
!tests
!resources
# The patched dependency Cargo.toml's [patch.crates-io] points at. Without
# it the build context has no vendor/, and `cargo chef cook` fails on
# "failed to load source for dependency sieve-rs" -- which CI cannot see,
# because CI builds from a checkout and only the image build has a context.
!vendor
!Cargo.lock
!Cargo.toml
+5
View File
@@ -35,6 +35,11 @@ jobs:
- run: python3 tools/fork/name-check.py
- if: always()
run: python3 tools/fork/notice-check.py
# Cargo can patch a dependency to a directory in this repository, and
# the image builds from a context .dockerignore prunes to almost
# nothing. CI never sees the difference; a release does.
- if: always()
run: python3 tools/fork/context-check.py
build:
# Either runner (host1 or host2): the build needs no docker socket.
+35 -1
View File
@@ -12,6 +12,9 @@
# An issue is opened once per release: an existing one with the same title,
# open or closed, stops a second.
#
# It also watches spam-filter, whose rules the server bundles
# (resources/spam-filter/), and opens an issue for a newer release.
#
# Daily 06:17 UTC; run it by hand with workflow_dispatch.
name: upstream-watch
@@ -64,7 +67,7 @@ jobs:
and key(r["tag_name"]) > key(base)),
key=lambda r: key(r["tag_name"]))
if not newer:
print(f"Up to date: {base} is the newest upstream release."); sys.exit(0)
print(f"Up to date: {base} is the newest upstream release.")
# Titles and bodies stay free of the upstream project's name, as the
# rest of the fork's user-visible text does.
@@ -85,4 +88,35 @@ jobs:
"add any new third-party notices to `THIRD-PARTY.md`, then merge `upstream` into `main`.")
issue = call("POST", f"{api}/issues", {"title": title, "body": body})
print(f"{tag}: opened #{issue['number']}.")
# The spam filter rules bundled with the server (resources/spam-filter/):
# an issue when spam-filter publishes a newer release than the one
# BUNDLED_SPAM_RULES_VERSION names on main.
src = call("GET", f"{api}/contents/crates/common/src/manager/spam_rules.rs?ref=main")
import base64
text = base64.b64decode(src["content"]).decode()
m = re.search(r'BUNDLED_SPAM_RULES_VERSION: &str = "(\d+\.\d+\.\d+)"', text)
if not m:
print("Can't read BUNDLED_SPAM_RULES_VERSION from spam_rules.rs", file=sys.stderr); sys.exit(1)
bundled = "v" + m.group(1)
rels = call("GET", "https://api.github.com/repos/stalwartlabs/spam-filter/releases?per_page=30", token=None)
newer = sorted((r for r in rels
if not r["draft"] and not r["prerelease"] and SEMVER.match(r["tag_name"])
and key(r["tag_name"]) > key(bundled)),
key=lambda r: key(r["tag_name"]))
if not newer:
print(f"Up to date: the bundled spam rules are {bundled}, the newest release."); sys.exit(0)
latest = newer[-1]
tag = latest["tag_name"]
title = f"Update the bundled spam rules to {tag}"
existing = {i["title"] for i in call("GET", f"{api}/issues?state=all&type=issues&q=bundled+spam+rules&limit=50")}
if title in existing:
print(f"spam rules {tag}: issue already exists."); sys.exit(0)
body = (f"spam-filter published {tag} on {latest['published_at'][:10]}. "
f"The server bundles {bundled}.\n\n"
"Update it as resources/spam-filter/README.md describes: take the rules file "
f"from the {tag} release (by tag, not `latest`), set BUNDLED_SPAM_RULES_VERSION, "
"and run the antispam test.")
issue = call("POST", f"{api}/issues", {"title": title, "body": body})
print(f"spam rules {tag}: opened #{issue['number']}.")
PY
+4
View File
@@ -19,6 +19,10 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
g++-x86-64-linux-gnu binutils-x86-64-linux-gnu
RUN rustup target add "$(cat /target.txt)"
COPY --from=planner /recipe.json /recipe.json
# inbuxa: [patch.crates-io] points sieve-rs at vendor/, and the recipe only
# carries the workspace's own manifests, so cooking the dependencies needs the
# vendored crate itself (the context allows it since #27; this puts it here).
COPY vendor/ vendor/
RUN RUSTFLAGS="$(cat /flags.txt)" cargo chef cook --target "$(cat /target.txt)" --release --no-default-features --features "sqlite postgres mysql rocks s3 redis azure nats" --recipe-path /recipe.json
COPY . .
RUN RUSTFLAGS="$(cat /flags.txt)" cargo build --target "$(cat /target.txt)" --release -p inbuxa --no-default-features --features "sqlite postgres mysql rocks s3 redis azure nats"
+1 -1
View File
@@ -24,7 +24,7 @@ carry their own license files.
| `crates/common/src/network/acme/directory.rs`, `crates/common/src/network/acme/jose.rs`, `crates/common/src/network/acme/order.rs` | [rustls-acme](https://github.com/FlorianUekermann/rustls-acme) (MIT or Apache-2.0) | Copyright (c) Florian Uekermann |
| `crates/types/src/id.rs` | [crockford](https://github.com/archer884/crockford) (MIT or Apache-2.0) | Copyright (c) 2017 J/A <archer884@gmail.com> |
| `crates/nlp/src/tokenizers/types.rs` | test cases from [linkify](https://github.com/robinst/linkify) (MIT or Apache-2.0) | Copyright (c) 2017 Robin Stocker |
| `tests/resources/smtp/antispam/spam-filter-rules.json.gz` | the published rules of [spam-filter](https://github.com/stalwartlabs/spam-filter) v3.0.2, unmodified, for the spam filter's tests (MIT or Apache-2.0) | Copyright (C) 2024, Stalwart Labs LLC |
| `resources/spam-filter/spam-filter-rules.json.gz` | the published rules of [spam-filter](https://github.com/stalwartlabs/spam-filter) v3.0.2, unmodified, built into the server as its default spam rules (MIT or Apache-2.0) | Copyright (C) 2024, Stalwart Labs LLC |
Each notice above applies with this permission notice:
@@ -243,7 +243,8 @@ impl SpamFilterConfig {
spam_threshold: spam.score_spam.into_inner() as f32,
},
grey_list_expiry: spam.greylist_for.map(|d| d.into_inner().as_secs()),
spam_rules_url: spam.spam_filter_rules_url,
// inbuxa: unset, empty or upstream's old default means the bundled rules
spam_rules_url: crate::manager::spam_rules::rules_url(spam.spam_filter_rules_url),
url_client: utils::http::http_client_builder(true)
.pool_max_idle_per_host(0)
.redirect(reqwest::redirect::Policy::none())
+14 -5
View File
@@ -530,13 +530,22 @@ async fn insert_safe_defaults(bp: &mut Bootstrap) -> trc::Result<()> {
use store::write::BatchBuilder;
use types::id::Id;
if bp.registry.count_object(ObjectType::SpamRule).await? == 0
&& bp
.registry
// inbuxa: rules are always to hand, since a copy ships with the server
// (spam_rules). They load on first boot, and again when the bundled
// version differs from the one last loaded, which only adds what's
// missing: new tags and rules, never a changed score.
let rules_url = super::spam_rules::rules_url(
bp.registry
.object::<SpamSettings>(Id::singleton())
.await?
.is_none_or(|spam| spam.spam_filter_rules_url.is_some())
{
.and_then(|spam| spam.spam_filter_rules_url),
);
let bundled_is_new = rules_url.is_none()
&& super::spam_rules::applied_version(&bp.data_store)
.await?
.as_deref()
!= Some(super::spam_rules::BUNDLED_SPAM_RULES_VERSION);
if bp.registry.count_object(ObjectType::SpamRule).await? == 0 || bundled_is_new {
let mut batch = BatchBuilder::new();
batch.schedule_task(Task::SpamFilterMaintenance(TaskSpamFilterMaintenance {
maintenance_type: TaskSpamFilterMaintenanceType::UpdateRules,
+1
View File
@@ -22,6 +22,7 @@ pub mod console;
pub mod defaults;
pub mod first_party;
pub mod restore;
pub mod spam_rules; // inbuxa: rules bundled with the server
pub const SPAM_TRAINER_KEY: &[u8] = "INBUXA_SPAM_TRAIN_DATA.lz4".as_bytes();
pub const SPAM_CLASSIFIER_KEY: &[u8] = "INBUXA_SPAM_CLASSIFIER_MODEL.lz4".as_bytes();
+103
View File
@@ -0,0 +1,103 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! inbuxa: the spam filter rules that ship with the server.
//!
//! Upstream fetches its latest published rules from GitHub at run time, so
//! scoring changes with a release nobody here tested and depends on reaching
//! it. The fork embeds a pinned copy (resources/spam-filter/, with its version
//! and license) and uses it whenever no other source is configured. The rules
//! URL remains an operator override (`https://` or `file://`).
//!
//! Loading rules only ever adds what's missing, never changes an existing rule
//! or score. They load on first boot, and again whenever the bundled version
//! differs from the one last applied, so an upgrade brings new tags (the AI
//! classifier's `LLM_*` scores, say) to an install that already had rules.
use std::io::Read;
use store::{
SUBSPACE_INBUXA, Store, ValueKey,
write::{AnyClass, BatchBuilder, ValueClass},
};
use trc::AddContext;
/// The version of spam-filter the embedded rules come from.
pub const BUNDLED_SPAM_RULES_VERSION: &str = "3.0.2";
static BUNDLED_SPAM_RULES: &[u8] =
include_bytes!("../../../../resources/spam-filter/spam-filter-rules.json.gz");
/// Upstream's default rules source, the value every install created before
/// the rules were bundled has saved. Read only to treat it as unset.
const LEGACY_DEFAULT_URL: &str =
"https://github.com/stalwartlabs/spam-filter/releases/latest/download/spam-filter-rules.json.gz";
/// The URL to fetch rules from, or `None` for the bundled rules. An empty
/// setting and upstream's old default both mean the bundled rules.
pub fn rules_url(configured: Option<String>) -> Option<String> {
configured.filter(|url| !url.trim().is_empty() && url != LEGACY_DEFAULT_URL)
}
/// The bundled rules, uncompressed: the same JSON the rules URL serves.
pub fn bundled_rules() -> Result<Vec<u8>, String> {
let mut json = Vec::new();
mail_auth::flate2::read::GzDecoder::new(BUNDLED_SPAM_RULES)
.read_to_end(&mut json)
.map_err(|err| format!("Failed to decompress the bundled spam rules: {err}"))?;
Ok(json)
}
fn applied_key() -> ValueClass {
ValueClass::Any(AnyClass {
subspace: SUBSPACE_INBUXA,
key: b"Sr".to_vec(),
})
}
/// The bundled version last loaded into the registry, if any.
pub async fn applied_version(data: &Store) -> trc::Result<Option<String>> {
data.get_value::<String>(ValueKey::from(applied_key()))
.await
.caused_by(trc::location!())
}
/// Records that the bundled rules of this version have been loaded.
pub async fn set_applied_version(data: &Store, version: &str) -> trc::Result<()> {
let mut batch = BatchBuilder::new();
batch.set(applied_key(), version.as_bytes().to_vec());
data.write(batch.build_all())
.await
.caused_by(trc::location!())
.map(|_| ())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn upstream_default_and_empty_mean_bundled() {
assert_eq!(rules_url(None), None);
assert_eq!(rules_url(Some(String::new())), None);
assert_eq!(rules_url(Some(" ".into())), None);
assert_eq!(rules_url(Some(LEGACY_DEFAULT_URL.into())), None);
assert_eq!(
rules_url(Some("file:///srv/rules.json.gz".into())).as_deref(),
Some("file:///srv/rules.json.gz")
);
}
#[test]
fn bundled_rules_parse_and_score_the_ai_tags() {
let rules: serde_json::Value = serde_json::from_slice(&bundled_rules().unwrap()).unwrap();
let tags = rules["SpamTag"].as_array().unwrap();
for (tag, score) in [("LLM_UNSOLICITED_HIGH", 3.0), ("LLM_LEGITIMATE_HIGH", -3.0)] {
let found = tags.iter().find(|t| t["tag"] == tag).unwrap();
assert_eq!(found["score"].as_f64(), Some(score), "{tag}");
}
assert!(!rules["SpamRule"].as_array().unwrap().is_empty());
}
}
+1 -1
View File
@@ -40215,7 +40215,7 @@ impl Default for SpamSettings {
score_reject: Float::new(0.0f64),
score_spam: Float::new(5.0f64),
trust_replies: true,
spam_filter_rules_url: Some("https://github.com/stalwartlabs/spam-filter/releases/latest/download/spam-filter-rules.json.gz".to_string()),
spam_filter_rules_url: None,
}
}
}
@@ -2,13 +2,15 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/
use crate::task_manager::{TaskFailureType, TaskResult};
use common::{
Server,
ipc::{BroadcastEvent, RegistryChange},
manager::{SPAM_CLASSIFIER_KEY, SPAM_TRAINER_KEY, fetch_resource},
manager::{SPAM_CLASSIFIER_KEY, SPAM_TRAINER_KEY, fetch_resource, spam_rules},
};
use registry::{
schema::{
@@ -106,6 +108,7 @@ struct RuleUpdateResult {
async fn update_spam_rules(server: &Server) -> trc::Result<TaskResult> {
let started = Instant::now();
let bundled = server.core.spam.spam_rules_url.is_none();
let rules = match fetch_spam_rules(server).await {
Ok(rules) => rules,
Err(err) => {
@@ -289,29 +292,36 @@ async fn update_spam_rules(server: &Server) -> trc::Result<TaskResult> {
Elapsed = started.elapsed(),
);
// inbuxa: so the next start knows these bundled rules are in
if bundled {
spam_rules::set_applied_version(server.store(), spam_rules::BUNDLED_SPAM_RULES_VERSION)
.await?;
}
Ok(TaskResult::Success(vec![]))
}
async fn fetch_spam_rules(server: &Server) -> Result<Rules, RuleUpdateError> {
let Some(rules_url) = server.core.spam.spam_rules_url.as_ref() else {
return Err(RuleUpdateError {
typ: TaskFailureType::Permanent,
reason: "Spam rules resource URL not configured".to_string(),
});
};
let rules_json: AHashMap<String, Vec<serde_json::Value>> =
fetch_resource(rules_url, None, Duration::from_secs(60), 1024 * 500)
// inbuxa: no URL means the rules bundled with the server
let bytes = match server.core.spam.spam_rules_url.as_ref() {
Some(rules_url) => fetch_resource(rules_url, None, Duration::from_secs(60), 1024 * 500)
.await
.map_err(|reason| RuleUpdateError {
typ: TaskFailureType::Temporary,
reason,
}),
None => spam_rules::bundled_rules().map_err(|reason| RuleUpdateError {
typ: TaskFailureType::Permanent,
reason,
}),
};
let rules_json: AHashMap<String, Vec<serde_json::Value>> =
bytes.and_then(|bytes| {
serde_json::from_slice(&bytes).map_err(|err| RuleUpdateError {
typ: TaskFailureType::Permanent,
reason: format!("Failed to parse spam rules JSON: {err}"),
})
.and_then(|bytes| {
serde_json::from_slice(&bytes).map_err(|err| RuleUpdateError {
typ: TaskFailureType::Permanent,
reason: format!("Failed to parse spam rules JSON: {err}"),
})
})?;
})?;
let mut rules = Rules::default();
for (object_type, values) in rules_json {
+1 -1
View File
@@ -81,7 +81,7 @@ fn legacy_setting(name: &str, is_set: impl Fn(&str) -> bool) -> Option<String> {
#[macro_export]
macro_rules! brand_version {
() => {
"2026.9.24"
"2026.9.24.3"
};
}
+4 -1
View File
@@ -102,7 +102,10 @@ Permissions: `sysSpamLlmGet`, `sysSpamLlmUpdate`.
- **Tags and scores.** The classifier's tags are ordinary spam tags, scored
by `x:SpamTag` entries like every other tag: `Score` (a number), `Discard`
or `Reject`. The documented defaults are `LLM_UNSOLICITED_HIGH` 3.0 and
`LLM_LEGITIMATE_HIGH` 3.0. A tag with no entry scores 0.
`LLM_LEGITIMATE_HIGH` 3.0. A tag with no entry scores 0. The server ships
those entries in its bundled spam rules (`resources/spam-filter/`), loaded
on first boot and again when the bundled version changes, so an install
that predates them gains them on upgrade (added 2026-09-23).
- **`interactAi`** permission ("Interact with AI models"): lets an account's
own Sieve scripts call `llm_prompt`. This repository's default roles give it
to users, tenant administrators and superusers
Binary file not shown.
+1 -1
View File
@@ -1 +1 @@
rWqJwNJkqgKsbC1eqcEmmIAMtJPmnlDlThR2ZtW_N1c
VbnFuwCOTBh0s2T-NuRhb2JaJr8Jl5s3LgXv4Pv2sTg
+32
View File
@@ -0,0 +1,32 @@
# Bundled spam filter rules
`spam-filter-rules.json.gz` is the published rules file of
[spam-filter](https://github.com/stalwartlabs/spam-filter) **v3.0.2**,
unmodified. The server embeds it (`crates/common/src/manager/spam_rules.rs`)
and loads it whenever no other rules source is configured, so a release
scores mail with the rules it was tested with, offline and with nothing to
fetch. The rules URL setting stays an operator override.
The rules are dual-licensed MIT or Apache-2.0, Copyright (C) 2024, Stalwart
Labs LLC; the fork takes them under MIT, with the notice in `THIRD-PARTY.md`.
They include the scores for the AI classifier's tags (`LLM_*`, 3.0 for the
high-confidence spam categories, 3.0 for legitimate), which match
`docs/spec/features/ai-spam-classification.md`.
## Updating
The `upstream-watch` workflow opens an issue when spam-filter publishes a
newer release. To take it:
1. Download `spam-filter-rules.json.gz` from that release, pinned by tag
(`releases/download/vX.Y.Z/…`, not `latest`), over this file.
2. Set `BUNDLED_SPAM_RULES_VERSION` in `spam_rules.rs` and the version in
this README and in `THIRD-PARTY.md`.
3. Run the antispam test (`STORE=RocksDb RUST_MIN_STACK=16777216 cargo test
-p tests --lib -- smtp::inbound::antispam::antispam --exact`) and fix
expectations the new rules change, knowingly.
On the next start each server loads the new version once. Loading only adds
rules and tags that are missing; it never changes an existing one, so an
operator's own adjustments survive.
+4 -13
View File
@@ -86,19 +86,10 @@ async fn antispam() {
.registry_create_object(SpamSettings {
score_spam: Float::new(5.0),
// inbuxa: the rules carry the scores the expectations are written
// against, so they're pinned (spam-filter v3.0.2, beside the test
// cases) rather than read from a developer's own checkout, which
// left every score at zero. SPAM_RULES_URL still overrides.
spam_filter_rules_url: std::env::var("SPAM_RULES_URL")
.unwrap_or_else(|_| {
concat!(
"file://",
env!("CARGO_MANIFEST_DIR"),
"/resources/smtp/antispam/spam-filter-rules.json.gz"
)
.to_string()
})
.into(),
// against. Unset, the server uses the rules bundled with it
// (resources/spam-filter/), the path production takes;
// SPAM_RULES_URL tests another set.
spam_filter_rules_url: std::env::var("SPAM_RULES_URL").ok(),
..Default::default()
})
.await;
+120
View File
@@ -0,0 +1,120 @@
#!/usr/bin/env python3
# SPDX-FileCopyrightText: 2026 Coffey Labs
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Every path Cargo patches has to be in the image's build context.
Cargo.toml's [patch.crates-io] can point at a directory in this repository,
and the Dockerfile builds from a context that .dockerignore prunes to almost
nothing. Those two facts met on 2026-09-23: a vendored, patched sieve-rs
landed, CI stayed green -- it builds from a checkout, where the directory is
simply there -- and the release build failed on
failed to load source for dependency `sieve-rs`
failed to read /build/vendor/sieve-rs/Cargo.toml
after a tag had already been pushed. This is seconds, and it runs beside the
other fork checks rather than waiting for a release to find out.
Being in the context isn't enough on its own: the Dockerfile cooks the
dependencies (`cargo chef cook`) before it copies the tree in, from a recipe
that carries only the workspace's manifests. So each patched path must also be
copied into that stage before the cook step, or the same error comes back
there -- as it did for 2026.9.24.2, the first tag after the context fix.
"""
import re
import sys
from pathlib import Path
root = Path(__file__).resolve().parents[2]
def patched_paths(manifest: Path) -> list[str]:
"""Directories named by a [patch...] section's `path = "..."` entries."""
out, in_patch = [], False
for line in manifest.read_text().splitlines():
stripped = line.strip()
if stripped.startswith("["):
in_patch = stripped.startswith("[patch")
continue
if not in_patch:
continue
m = re.search(r'path\s*=\s*"([^"]+)"', stripped)
if m:
out.append(m.group(1))
return out
def allowed(dockerignore: Path) -> set[str]:
"""The first path segment of every re-inclusion rule."""
keep = set()
for line in dockerignore.read_text().splitlines():
stripped = line.strip()
if stripped.startswith("!"):
keep.add(stripped[1:].strip("/").split("/")[0])
return keep
def copied_before_cook(dockerfile: Path) -> list[str] | None:
"""Sources COPY'd into the stage that runs `cargo chef cook`, before it.
None when no stage cooks. A `COPY . .` covers everything.
"""
stage: list[str] = []
for line in dockerfile.read_text().splitlines():
stripped = line.strip()
if re.match(r"(?i)^FROM\s", stripped):
stage = []
continue
if "cargo chef cook" in stripped:
return stage
m = re.match(r"(?i)^COPY\s+(?!--from)(.+)$", stripped)
if m:
parts = m.group(1).split()
stage.extend(p.strip("./").split("/")[0] or "." for p in parts[:-1])
return None
def main() -> int:
paths = patched_paths(root / "Cargo.toml")
if not paths:
print("no patched paths to check")
return 0
keep = allowed(root / ".dockerignore")
bad = []
for p in paths:
top = p.strip("/").split("/")[0]
if top not in keep:
bad.append((p, top))
elif not (root / p).is_dir():
bad.append((p, None))
for path, top in bad:
if top is None:
print(f"Cargo.toml patches {path}, which does not exist", file=sys.stderr)
else:
print(
f"Cargo.toml patches {path}, but .dockerignore does not re-include {top!r}:\n"
f" the image build would not see it, and cargo would fail on it.\n"
f" Add `!{top}` to .dockerignore.",
file=sys.stderr,
)
copied = copied_before_cook(root / "Dockerfile")
if copied is not None and "." not in copied:
for p in paths:
top = p.strip("/").split("/")[0]
if top not in copied:
print(
f"Cargo.toml patches {p}, but the Dockerfile doesn't copy {top!r} into the\n"
f" stage that runs `cargo chef cook` before that step, so cooking the\n"
f" dependencies fails on it. Add `COPY {top}/ {top}/` before the cook.",
file=sys.stderr,
)
bad.append((p, top))
if bad:
return 1
print(f"build context and cook stage include every patched path: {', '.join(paths)}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+3 -3
View File
@@ -23,6 +23,6 @@ crates/migration/src/lib.rs "STALWART_SPAM_CLASSIFIER_MODEL.lz4"
crates/migration/src/lib.rs "STALWART_SPAM_TRAIN_DATA.lz4"
crates/types/src/branding.rs "STALWART"
# OPEN, not yet decided (2026-09-22): upstream's published spam-filter rules,
# which the server downloads at runtime from this address.
crates/registry/src/schema/structs_impl.rs "https://github.com/stalwartlabs/spam-filter/releases/latest/download/spam-filter-rules.json.gz"
# Upstream's old default rules source, read only to treat it as unset: the
# server uses the rules bundled with it (resources/spam-filter/).
crates/common/src/manager/spam_rules.rs "https://github.com/stalwartlabs/spam-filter/releases/latest/download/spam-filter-rules.json.gz"
+8
View File
@@ -46,6 +46,10 @@ TEXT_RENAMES = [
# that must match their containers and identity provider (database users,
# passwords, an OIDC audience), and name their databases explicitly.
('"stalwart".to_string()', '"inbuxa".to_string()', ('crates',)),
# The spam filter rules ship with the server (common::manager::spam_rules);
# upstream's default of fetching its latest from GitHub becomes unset.
('spam_filter_rules_url: Some("https://github.com/stalwartlabs/spam-filter/releases/latest/download/spam-filter-rules.json.gz".to_string()),',
'spam_filter_rules_url: None,', ('crates',)),
]
ROOTS = ('crates', 'tests', 'resources')
SKIP_SUFFIXES = {'.md', '.txt'}
@@ -58,6 +62,10 @@ SCHEMA_HASH = Path('resources/schema/schema.json.sha256')
SCHEMA_RENAMES = [
('"stalwart"', '"inbuxa"'),
('vnd.stalwart', 'vnd.inbuxa'),
# The bundled spam rules: no default URL, and say what empty means.
('"spamFilterRulesUrl":"https://github.com/stalwartlabs/spam-filter/releases/latest/download/spam-filter-rules.json.gz",', ''),
('"URL to download spam filter rules from"',
'"URL to download spam filter rules from. Empty uses the rules bundled with the server."'),
]