The server fetched upstream's latest published rules from GitHub at run time: a version nobody here tested, code-like expressions from an account we don't control, and the upstream name as a default in the admin form. The published rules of spam-filter v3.0.2 are now embedded (resources/spam-filter/, MIT, in THIRD-PARTY.md) and used whenever no other source is configured. An empty setting and upstream's old default both mean the bundled rules, so existing installs switch without a settings change; the URL stays an operator override (https:// or file://). The schema default is dropped and its description says what empty means, and the strip's rename pass does the same to each import. Rules load on first boot as before, and again whenever the bundled version differs from the last one loaded, which only adds missing rules and tags. That brings the AI classifier's LLM_* scores to installs that predate them: production has none today. upstream-watch now also opens an issue when spam-filter publishes a newer release; resources/spam-filter/README.md says how to take it. The antispam test now runs on the bundled rules, the path production takes; SPAM_RULES_URL tests another set. Unit tests cover the URL handling and that the bundled rules parse and score the AI tags as the AI spec says.
104 lines
3.9 KiB
Rust
104 lines
3.9 KiB
Rust
/*
|
|
* SPDX-FileCopyrightText: 2026 Coffey Labs
|
|
*
|
|
* SPDX-License-Identifier: AGPL-3.0-only
|
|
*/
|
|
|
|
//! inbuxa: the spam filter rules that ship with the server.
|
|
//!
|
|
//! Upstream fetches its latest published rules from GitHub at run time, so
|
|
//! scoring changes with a release nobody here tested and depends on reaching
|
|
//! it. The fork embeds a pinned copy (resources/spam-filter/, with its version
|
|
//! and license) and uses it whenever no other source is configured. The rules
|
|
//! URL remains an operator override (`https://` or `file://`).
|
|
//!
|
|
//! Loading rules only ever adds what's missing, never changes an existing rule
|
|
//! or score. They load on first boot, and again whenever the bundled version
|
|
//! differs from the one last applied, so an upgrade brings new tags (the AI
|
|
//! classifier's `LLM_*` scores, say) to an install that already had rules.
|
|
|
|
use std::io::Read;
|
|
use store::{
|
|
SUBSPACE_INBUXA, Store, ValueKey,
|
|
write::{AnyClass, BatchBuilder, ValueClass},
|
|
};
|
|
use trc::AddContext;
|
|
|
|
/// The version of spam-filter the embedded rules come from.
|
|
pub const BUNDLED_SPAM_RULES_VERSION: &str = "3.0.2";
|
|
|
|
static BUNDLED_SPAM_RULES: &[u8] =
|
|
include_bytes!("../../../../resources/spam-filter/spam-filter-rules.json.gz");
|
|
|
|
/// Upstream's default rules source, the value every install created before
|
|
/// the rules were bundled has saved. Read only to treat it as unset.
|
|
const LEGACY_DEFAULT_URL: &str =
|
|
"https://github.com/stalwartlabs/spam-filter/releases/latest/download/spam-filter-rules.json.gz";
|
|
|
|
/// The URL to fetch rules from, or `None` for the bundled rules. An empty
|
|
/// setting and upstream's old default both mean the bundled rules.
|
|
pub fn rules_url(configured: Option<String>) -> Option<String> {
|
|
configured.filter(|url| !url.trim().is_empty() && url != LEGACY_DEFAULT_URL)
|
|
}
|
|
|
|
/// The bundled rules, uncompressed: the same JSON the rules URL serves.
|
|
pub fn bundled_rules() -> Result<Vec<u8>, String> {
|
|
let mut json = Vec::new();
|
|
mail_auth::flate2::read::GzDecoder::new(BUNDLED_SPAM_RULES)
|
|
.read_to_end(&mut json)
|
|
.map_err(|err| format!("Failed to decompress the bundled spam rules: {err}"))?;
|
|
Ok(json)
|
|
}
|
|
|
|
fn applied_key() -> ValueClass {
|
|
ValueClass::Any(AnyClass {
|
|
subspace: SUBSPACE_INBUXA,
|
|
key: b"Sr".to_vec(),
|
|
})
|
|
}
|
|
|
|
/// The bundled version last loaded into the registry, if any.
|
|
pub async fn applied_version(data: &Store) -> trc::Result<Option<String>> {
|
|
data.get_value::<String>(ValueKey::from(applied_key()))
|
|
.await
|
|
.caused_by(trc::location!())
|
|
}
|
|
|
|
/// Records that the bundled rules of this version have been loaded.
|
|
pub async fn set_applied_version(data: &Store, version: &str) -> trc::Result<()> {
|
|
let mut batch = BatchBuilder::new();
|
|
batch.set(applied_key(), version.as_bytes().to_vec());
|
|
data.write(batch.build_all())
|
|
.await
|
|
.caused_by(trc::location!())
|
|
.map(|_| ())
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn upstream_default_and_empty_mean_bundled() {
|
|
assert_eq!(rules_url(None), None);
|
|
assert_eq!(rules_url(Some(String::new())), None);
|
|
assert_eq!(rules_url(Some(" ".into())), None);
|
|
assert_eq!(rules_url(Some(LEGACY_DEFAULT_URL.into())), None);
|
|
assert_eq!(
|
|
rules_url(Some("file:///srv/rules.json.gz".into())).as_deref(),
|
|
Some("file:///srv/rules.json.gz")
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn bundled_rules_parse_and_score_the_ai_tags() {
|
|
let rules: serde_json::Value = serde_json::from_slice(&bundled_rules().unwrap()).unwrap();
|
|
let tags = rules["SpamTag"].as_array().unwrap();
|
|
for (tag, score) in [("LLM_UNSOLICITED_HIGH", 3.0), ("LLM_LEGITIMATE_HIGH", -3.0)] {
|
|
let found = tags.iter().find(|t| t["tag"] == tag).unwrap();
|
|
assert_eq!(found["score"].as_f64(), Some(score), "{tag}");
|
|
}
|
|
assert!(!rules["SpamRule"].as_array().unwrap().is_empty());
|
|
}
|
|
}
|