Import upstream v0.16.24, stripped

Upstream commit: af37a234981722493b74623a983581691d2b70b6
Enterprise-only files removed or emptied: 63
Enterprise-only snippets removed: 118 in 50 files
Dangling module declarations removed: 5
Edits turning enterprise off: 25
Third-party code: 14 files, 0 not in THIRD-PARTY.md
Renamed identifiers: 62 in 18 files
Verification: clean

The same Enterprise footprint as v0.16.23. The build check fails only on
tests/src/directory/issuer.rs, unchanged since v0.16.23: it calls a helper
from upstream's Enterprise-only OIDC test, and tests issuer-based directory
routing, an Enterprise feature. main has never carried it.

CHANGELOG.md

@@ -2,6 +2,39 @@
 
 All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org/).
 
+## [0.16.24] - 2026-09-27
+
+If you are upgrading from v0.16.x, replace the binary (or run `docker pull`). If you are upgrading from v0.15.x and below, please read the [upgrading documentation](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING/v0_16.md) for more information on how to upgrade from previous versions.
+
+## Added
+- DNS: PowerDNS Authoritative provider for automatic DNS record management.
+
+## Changed
+
+## Fixed
+- Troubleshoot tool: `TLSA` records are looked up for every MX host, including hosts whose zone is not DNSSEC signed.
+- Spam filter:
+  - OpenPhish and PhishTank entries containing uppercase characters never match, since message URLs are lowercased while HTTP lookup entries keep their original case. HTTP lookups now match keys case-insensitively.
+  - URL shortener links are followed using the lowercased URL, so case-sensitive short links resolve to the wrong destination or not at all.
+  - Incremental training never advances its position past the first run, so every retained sample added since then is trained again, and counted again in the reservoir, on each run until it expires.
+  - Updating the rules only adds new objects, so upstream changes to existing rules, DNSBL servers, HTTP lookups, lookup keys and file extensions never reach an existing installation.
+  - Updating the rules reports success when objects fail to import, or when a configuration error stops the updated settings from being activated.
+- JMAP:
+  - A `PushSubscription` created within the verification rate limit window of another one on the same account never receives its `PushVerification`, since the blocked verification is dropped instead of being sent once the window expires.
+  - A push notification retried after a failed delivery can report an older state than a change queued during the failed attempt, since the older state changes are merged last and overwrite the newer ones.
+  - Changes made while a push request is in flight are not delivered until the next change reaches the same subscription, since a successful delivery cancels the pending retry.
+  - The VAPID `aud` claim is derived from a hand-written parse of the push URL, so a crafted push URL can make the server sign a token for a push service other than the one the request is sent to.
+  - `Email/import` rejects a `blobId` that refers to a `Blob/upload` creation id in the same request (`"#u0"`) with `Invalid blob id.`.
+  - `Email/set` with a full `mailboxIds` object identical to the current mailboxes, together with a keyword change, stores the message with IMAP UID 0, so IMAP clients stop seeing it.
+- MTA:
+  - A node without the `outboundMta` role stops replying to `DATA` and to JMAP submissions once about 1024 messages have been queued on it.
+  - MX records are resolved through the DNSSEC-validating resolver even when DANE is disabled.
+  - A `DATA` stage Sieve script does not see headers added by milters or MTA hooks, and discards every milter and MTA hook change when it edits the message.
+- MySQL: Range deletions and search index removals start with a single unbounded `DELETE` and switch to chunks only after a timeout.
+- IMAP: `COPY` and `MOVE` fail with `NO [CONTACTADMIN]` when another session changes the same message at the same time.
+- Autodiscover: Implicit TLS ports (993, 995, 465) are advertised with `<Encryption>TLS</Encryption>`, which Outlook reads as STARTTLS.
+- HTTP: Idle keep-alive connections are never closed.
+
 ## [0.16.23] - 2026-09-21
 
 If you are upgrading from v0.16.x, replace the binary (or run `docker pull`). If you are upgrading from v0.15.x and below, please read the [upgrading documentation](https://github.com/stalwartlabs/stalwart/blob/main/UPGRADING/v0_16.md) for more information on how to upgrade from previous versions.

crates/common/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "common"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 build = "build.rs"
 

crates/common/src/manager/application.rs

@@ -512,12 +512,12 @@ mod tests {
 
     #[test]
     fn index_is_rewritten_with_the_prefix_and_client_id() {
-        let meta = oauth_client_id_meta("stalwart-webui");
+        let meta = oauth_client_id_meta("inbuxa-webui");
         let html = String::from_utf8(rewrite_index(INDEX, "admin", Some(&meta))).unwrap();
 
         assert!(html.contains("<base href=\"/admin/\" />"), "{html}");
         assert!(
-            html.contains("<meta name=\"oauth-client-id\" content=\"stalwart-webui\" />"),
+            html.contains("<meta name=\"oauth-client-id\" content=\"inbuxa-webui\" />"),
             "{html}"
         );
         assert!(html.contains("<title>Portal</title>"), "{html}");
@@ -539,7 +539,7 @@ mod tests {
     #[test]
     fn index_without_a_placeholder_is_left_alone() {
         let bundle = "<head>\n  <base href=\"/\" />\n</head>";
-        let meta = oauth_client_id_meta("stalwart-webui");
+        let meta = oauth_client_id_meta("inbuxa-webui");
         let html = String::from_utf8(rewrite_index(bundle, "admin", Some(&meta))).unwrap();
 
         assert_eq!(html, "<head>\n  <base href=\"/admin/\" />\n</head>");

crates/common/src/manager/mod.rs

@@ -20,8 +20,8 @@ pub mod console;
 pub mod defaults;
 pub mod restore;
 
-pub const SPAM_TRAINER_KEY: &[u8] = "STALWART_SPAM_TRAIN_DATA.lz4".as_bytes();
-pub const SPAM_CLASSIFIER_KEY: &[u8] = "STALWART_SPAM_CLASSIFIER_MODEL.lz4".as_bytes();
+pub const SPAM_TRAINER_KEY: &[u8] = "INBUXA_SPAM_TRAIN_DATA.lz4".as_bytes();
+pub const SPAM_CLASSIFIER_KEY: &[u8] = "INBUXA_SPAM_CLASSIFIER_MODEL.lz4".as_bytes();
 
 pub async fn fetch_resource(
     url: &str,

crates/common/src/network/autoconfig/autodiscover.rs

@@ -8,8 +8,9 @@ use crate::{Server, manager::application::Resource};
 use quick_xml::Reader;
 use quick_xml::XmlVersion;
 use quick_xml::events::Event;
-use registry::schema::enums::ServiceProtocol;
+use registry::schema::{enums::ServiceProtocol, structs::Service};
 use std::fmt::Write;
+use utils::map::vec_map::VecMap;
 
 impl Server {
     pub async fn handle_autodiscover_request(
@@ -24,8 +25,24 @@ impl Server {
                     .details("Failed to parse autodiscover request")
                     .ctx(trc::Key::Reason, err)
             })?;
-        let default_host = &self.core.network.server_name;
 
+        Ok(Resource::new(
+            "application/xml; charset=utf-8",
+            build_autodiscover_response(
+                &emailaddress,
+                &self.core.network.server_name,
+                &self.core.network.info.services,
+            )
+            .into_bytes(),
+        ))
+    }
+}
+
+fn build_autodiscover_response(
+    emailaddress: &str,
+    default_host: &str,
+    services: &VecMap<ServiceProtocol, Service>,
+) -> String {
     // Build XML response
     let mut config = String::with_capacity(1024);
     let _ = writeln!(&mut config, "<?xml version=\"1.0\" encoding=\"UTF-8\"?>");
@@ -55,16 +72,17 @@ impl Server {
     let _ = writeln!(&mut config, "\t\t<Account>");
     let _ = writeln!(&mut config, "\t\t\t<AccountType>email</AccountType>");
     let _ = writeln!(&mut config, "\t\t\t<Action>settings</Action>");
-        for (protocol, service) in &self.core.network.info.services {
+    for (protocol, service) in services {
         let (protocol, ports) = match protocol {
-                ServiceProtocol::Imap => ("IMAP", [143, 993]),
-                ServiceProtocol::Pop3 => ("POP3", [110, 995]),
-                ServiceProtocol::Smtp => ("SMTP", [587, 465]),
+            ServiceProtocol::Imap => ("IMAP", [(993, true), (143, false)]),
+            ServiceProtocol::Pop3 => ("POP3", [(995, true), (110, false)]),
+            ServiceProtocol::Smtp => ("SMTP", [(465, true), (587, false)]),
             _ => continue,
         };
 
-            for (is_tls, port) in ports.into_iter().enumerate() {
-                if is_tls == 1 || service.cleartext {
+        // Implicit TLS is listed first so that it is preferred (RFC 8314)
+        for (port, is_tls) in ports {
+            if is_tls || service.cleartext {
                 let server_name = service.hostname.as_deref().unwrap_or(default_host);
                 let _ = writeln!(&mut config, "\t\t\t<Protocol>");
                 let _ = writeln!(&mut config, "\t\t\t\t<Type>{protocol}</Type>",);
@@ -74,14 +92,13 @@ impl Server {
                 let _ = writeln!(&mut config, "\t\t\t\t<AuthRequired>on</AuthRequired>");
                 let _ = writeln!(&mut config, "\t\t\t\t<DirectoryPort>0</DirectoryPort>");
                 let _ = writeln!(&mut config, "\t\t\t\t<ReferralPort>0</ReferralPort>");
-                    let _ = writeln!(
-                        &mut config,
-                        "\t\t\t\t<SSL>{}</SSL>",
-                        if is_tls == 1 { "on" } else { "off" }
-                    );
-                    if is_tls == 1 {
-                        let _ = writeln!(&mut config, "\t\t\t\t<Encryption>TLS</Encryption>");
-                    }
+                let (ssl, encryption) = if is_tls {
+                    ("on", "SSL")
+                } else {
+                    ("off", "TLS")
+                };
+                let _ = writeln!(&mut config, "\t\t\t\t<SSL>{ssl}</SSL>");
+                let _ = writeln!(&mut config, "\t\t\t\t<Encryption>{encryption}</Encryption>");
                 let _ = writeln!(&mut config, "\t\t\t\t<SPA>off</SPA>");
                 let _ = writeln!(&mut config, "\t\t\t</Protocol>");
             }
@@ -92,11 +109,7 @@ impl Server {
     let _ = writeln!(&mut config, "\t</Response>");
     let _ = writeln!(&mut config, "</Autodiscover>");
 
-        Ok(Resource::new(
-            "application/xml; charset=utf-8",
-            config.into_bytes(),
-        ))
-    }
+    config
 }
 
 fn parse_autodiscover_request(bytes: &[u8]) -> Result<String, String> {
@@ -201,4 +214,78 @@ mod tests {
             "[email protected]"
         );
     }
+
+    #[test]
+    fn autodiscover_encryption() {
+        use registry::schema::{enums::ServiceProtocol, structs::Service};
+        use utils::map::vec_map::VecMap;
+
+        fn tag<'x>(block: &'x str, name: &str) -> &'x str {
+            block
+                .split_once(&format!("<{name}>"))
+                .and_then(|(_, rest)| rest.split_once(&format!("</{name}>")))
+                .map(|(value, _)| value)
+                .unwrap()
+        }
+
+        for (cleartext, expected) in [
+            (
+                false,
+                vec![
+                    ("IMAP", "993", "on", "SSL"),
+                    ("POP3", "995", "on", "SSL"),
+                    ("SMTP", "465", "on", "SSL"),
+                ],
+            ),
+            (
+                true,
+                vec![
+                    ("IMAP", "993", "on", "SSL"),
+                    ("IMAP", "143", "off", "TLS"),
+                    ("POP3", "995", "on", "SSL"),
+                    ("POP3", "110", "off", "TLS"),
+                    ("SMTP", "465", "on", "SSL"),
+                    ("SMTP", "587", "off", "TLS"),
+                ],
+            ),
+        ] {
+            let services: VecMap<ServiceProtocol, Service> = [
+                ServiceProtocol::Imap,
+                ServiceProtocol::Pop3,
+                ServiceProtocol::Smtp,
+                ServiceProtocol::Jmap,
+            ]
+            .into_iter()
+            .map(|protocol| {
+                (
+                    protocol,
+                    Service {
+                        hostname: None,
+                        cleartext,
+                    },
+                )
+            })
+            .collect();
+            let response = super::build_autodiscover_response(
+                "[email protected]",
+                "mail.example.com",
+                &services,
+            );
+
+            assert_eq!(
+                response
+                    .split("<Protocol>")
+                    .skip(1)
+                    .map(|block| (
+                        tag(block, "Type"),
+                        tag(block, "Port"),
+                        tag(block, "SSL"),
+                        tag(block, "Encryption"),
+                    ))
+                    .collect::<Vec<_>>(),
+                expected,
+                "cleartext: {cleartext}"
+            );
+        }
+    }
 }

crates/common/src/network/dns/update.rs

@@ -961,6 +961,20 @@ impl DnsUpdater {
                 )
                 .map_err(|err| format!("Failed to build DNS updater: {}", err))?,
             }),
+            DnsServer::PowerDns(server) => Ok(DnsUpdater {
+                polling_interval: server.polling_interval.into_inner(),
+                propagation_timeout: server.propagation_timeout.into_inner(),
+                propagation_delay: server.propagation_delay.map(|d| d.into_inner()),
+                ttl: server.ttl.into_inner(),
+                core,
+                updater: dns_update::DnsUpdater::new_pdns(
+                    server.api_key.secret().await?,
+                    server.endpoint,
+                    server.server_id,
+                    server.timeout.into_inner().into(),
+                )
+                .map_err(|err| format!("Failed to build DNS updater: {}", err))?,
+            }),
             DnsServer::Safedns(server) => Ok(DnsUpdater {
                 polling_interval: server.polling_interval.into_inner(),
                 propagation_timeout: server.propagation_timeout.into_inner(),

crates/common/src/network/webpush.rs

@@ -4,33 +4,79 @@
  * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
  */
 
+use ahash::AHashMap;
 use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};
 use p256::{
     SecretKey,
     ecdsa::{Signature, SigningKey, signature::Signer},
     pkcs8::{DecodePrivateKey, PrivateKeyInfo, der::SecretDocument},
 };
+use parking_lot::Mutex;
+use reqwest::{Url, header::HeaderValue};
+use std::sync::Arc;
 
 const VAPID_TOKEN_TTL: u64 = 12 * 60 * 60;
+const VAPID_TOKEN_REFRESH: u64 = VAPID_TOKEN_TTL / 2;
 
 #[derive(Clone)]
 pub struct Vapid {
     key: VapidKey,
     contact: Option<String>,
+    tokens: Arc<Mutex<AHashMap<String, VapidToken>>>,
+}
+
+struct VapidToken {
+    authorization: HeaderValue,
+    issued_at: u64,
 }
 
 impl Vapid {
     pub fn new(key: VapidKey, contact: Option<String>) -> Self {
-        Self { key, contact }
+        Self {
+            key,
+            contact,
+            tokens: Arc::default(),
+        }
     }
 
     pub fn public_key(&self) -> &str {
         self.key.public_key()
     }
 
-    pub fn authorization(&self, endpoint: &str, now: u64) -> Option<String> {
-        self.key
-            .authorization(endpoint, self.contact.as_deref(), now)
+    pub fn authorization(&self, endpoint: &str, now: u64) -> Option<HeaderValue> {
+        let prefix = endpoint_prefix(endpoint)?;
+        if let Some(token) = self
+            .tokens
+            .lock()
+            .get(prefix)
+            .filter(|token| token.is_fresh(now))
+        {
+            return Some(token.authorization.clone());
+        }
+
+        let authorization = HeaderValue::try_from(self.key.authorization(
+            endpoint,
+            self.contact.as_deref(),
+            now,
+        )?)
+        .ok()?;
+        let mut tokens = self.tokens.lock();
+        tokens.retain(|_, token| token.is_fresh(now));
+        tokens.insert(
+            prefix.to_string(),
+            VapidToken {
+                authorization: authorization.clone(),
+                issued_at: now,
+            },
+        );
+        Some(authorization)
+    }
+}
+
+impl VapidToken {
+    fn is_fresh(&self, now: u64) -> bool {
+        now.checked_sub(self.issued_at)
+            .is_some_and(|age| age < VAPID_TOKEN_REFRESH)
     }
 }
 
@@ -103,41 +149,15 @@ impl VapidKey {
     }
 }
 
-fn endpoint_origin(url: &str) -> Option<String> {
+fn endpoint_prefix(url: &str) -> Option<&str> {
     let (scheme, rest) = url.split_once("://")?;
-    let scheme = scheme.to_ascii_lowercase();
     let authority = rest.split(['/', '?', '#']).next()?;
-    let authority = authority
-        .rsplit_once('@')
-        .map(|(_, host)| host)
-        .unwrap_or(authority);
-    if authority.is_empty() {
-        return None;
-    }
+    url.get(..scheme.len() + "://".len() + authority.len())
+}
 
-    let (host, port) = if let Some(rest) = authority.strip_prefix('[') {
-        let (addr, tail) = rest.split_once(']')?;
-        (
-            format!("[{}]", addr.to_ascii_lowercase()),
-            tail.strip_prefix(':').filter(|port| !port.is_empty()),
-        )
-    } else if let Some((host, port)) = authority.rsplit_once(':') {
-        (
-            host.to_ascii_lowercase(),
-            Some(port).filter(|p| !p.is_empty()),
-        )
-    } else {
-        (authority.to_ascii_lowercase(), None)
-    };
-
-    match port {
-        Some(port)
-            if !((scheme == "https" && port == "443") || (scheme == "http" && port == "80")) =>
-        {
-            Some(format!("{scheme}://{host}:{port}"))
-        }
-        _ => Some(format!("{scheme}://{host}")),
-    }
+fn endpoint_origin(url: &str) -> Option<String> {
+    let origin = Url::parse(url).ok()?.origin();
+    origin.is_tuple().then(|| origin.ascii_serialization())
 }
 
 pub fn normalize_contact(contact: &str) -> Option<String> {
@@ -204,7 +224,12 @@ mod tests {
             endpoint_origin("http://[2001:DB8::1]:80/p").unwrap(),
             "http://[2001:db8::1]"
         );
+        assert_eq!(
+            endpoint_origin("https://attacker.example\\@fcm.googleapis.com/fcm/send/x").unwrap(),
+            "https://attacker.example"
+        );
         assert!(endpoint_origin("not-a-url").is_none());
+        assert!(endpoint_origin("mailto:[email protected]").is_none());
     }
 
     #[test]
@@ -334,6 +359,47 @@ B4yDfR2rGOd2H6Kv3fQNHPj9Nu5Tks8QYMLzrX8ONCNoFnNUQl9S0r0QS6phVqD0
         }
     }
 
+    #[test]
+    fn authorization_is_reused_per_endpoint_prefix() {
+        let vapid = Vapid::new(test_key(), None);
+        let now = 1_700_000_000;
+        let token = vapid
+            .authorization("https://push.example.com/push/a", now)
+            .unwrap();
+
+        assert_eq!(
+            vapid
+                .authorization("https://push.example.com/push/b?x=1", now + 60)
+                .unwrap(),
+            token
+        );
+        assert_ne!(
+            vapid
+                .authorization("https://other.example.com/push/a", now)
+                .unwrap(),
+            token
+        );
+        assert_ne!(
+            vapid
+                .authorization("https://push.example.com/push/a", now - 1)
+                .unwrap(),
+            token
+        );
+        let refreshed = vapid
+            .authorization("https://push.example.com/push/a", now + VAPID_TOKEN_REFRESH)
+            .unwrap();
+        assert_ne!(refreshed, token);
+        assert_eq!(
+            vapid
+                .authorization(
+                    "https://push.example.com/push/c",
+                    now + VAPID_TOKEN_REFRESH + 1
+                )
+                .unwrap(),
+            refreshed
+        );
+    }
+
     #[test]
     fn authorization_omits_subject_when_no_contact() {
         let key = test_key();

crates/coordinator/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "coordinator"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/dav-proto/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "dav-proto"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/dav/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "dav"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/dav/src/common/lock.rs

@@ -490,7 +490,7 @@ impl LockRequestHandler for Server {
                 for cond in &if_.list {
                     match cond {
                         Condition::StateToken { token, .. } => {
-                            if token.starts_with("urn:stalwart:davsync:") {
+                            if token.starts_with("urn:inbuxa:davsync:") {
                                 needs_sync_token = true;
                             } else {
                                 needs_lock_token = true;

crates/dav/src/common/uri.rs

@@ -181,12 +181,12 @@ impl OwnedUri<'_> {
 impl Urn {
     pub fn try_extract_sync_id(token: &str) -> Option<&str> {
         token
-            .strip_prefix("urn:stalwart:davsync:")
+            .strip_prefix("urn:inbuxa:davsync:")
             .map(|x| x.split_once(':').map(|(x, _)| x).unwrap_or(x))
     }
 
     pub fn parse(input: &str) -> Option<Self> {
-        let inbox = input.strip_prefix("urn:stalwart:")?;
+        let inbox = input.strip_prefix("urn:inbuxa:")?;
         let (kind, id) = inbox.split_once(':')?;
         match kind {
             "davlock" => u64::from_str_radix(id, 16).ok().map(Urn::Lock),
@@ -223,12 +223,12 @@ impl Urn {
 impl Display for Urn {
     fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
         match self {
-            Urn::Lock(id) => write!(f, "urn:stalwart:davlock:{id:x}",),
+            Urn::Lock(id) => write!(f, "urn:inbuxa:davlock:{id:x}",),
             Urn::Sync { id, seq } => {
                 if *seq == 0 {
-                    write!(f, "urn:stalwart:davsync:{id:x}")
+                    write!(f, "urn:inbuxa:davsync:{id:x}")
                 } else {
-                    write!(f, "urn:stalwart:davsync:{id:x}:{seq:x}")
+                    write!(f, "urn:inbuxa:davsync:{id:x}:{seq:x}")
                 }
             }
         }

crates/directory/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "directory"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/email/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "email"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/groupware/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "groupware"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/http-proto/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "http_proto"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/http/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "http"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/http/src/api/diagnose.rs

@@ -12,7 +12,7 @@ use common::{
     },
 };
 use hyper::body::{Bytes, Frame};
-use mail_auth::{IpLookupStrategy, mta_sts::TlsRpt};
+use mail_auth::{DnssecStatus, IpLookupStrategy, mta_sts::TlsRpt};
 use serde::{Deserialize, Serialize};
 use smtp::outbound::{
     client::{SmtpClient, StartTlsResult},
@@ -382,11 +382,59 @@ async fn delivery_diagnose(
             }
         }
 
+        tx.send(DeliveryStage::IpLookupStart).await?;
+
+        let now = Instant::now();
+        let validate_addresses = server.core.smtp.resolvers.dnssec_available
+            && host.dnssec_status() == DnssecStatus::Secure;
+        let (remote_ips, addresses_dnssec_status) = match host.fqdn_hostname() {
+            HostOrIp::Host(hostname) => {
+                match server
+                    .ip_lookup(
+                        &hostname,
+                        IpLookupStrategy::Ipv4thenIpv6,
+                        usize::MAX,
+                        validate_addresses,
+                    )
+                    .await
+                {
+                    Ok((remote_ips, dnssec_status)) if !remote_ips.is_empty() => {
+                        (remote_ips, dnssec_status)
+                    }
+                    Ok(_) => {
+                        tx.send(DeliveryStage::IpLookupError {
+                            reason: "No IP addresses found for host".to_string(),
+                            elapsed: now.elapsed_ms(),
+                        })
+                        .await?;
+                        continue;
+                    }
+                    Err(err) => {
+                        tx.send(DeliveryStage::IpLookupError {
+                            reason: err.to_string(),
+                            elapsed: now.elapsed_ms(),
+                        })
+                        .await?;
+                        continue;
+                    }
+                }
+            }
+            HostOrIp::Ip(ip) => (vec![ip], DnssecStatus::Indeterminate),
+        };
+
+        tx.send(DeliveryStage::IpLookupSuccess {
+            remote_ips: remote_ips.clone(),
+            elapsed: now.elapsed_ms(),
+        })
+        .await?;
+
         // Fetch TLSA record
         tx.send(DeliveryStage::TlsaLookupStart).await?;
 
         let now = Instant::now();
-        let dane_policy = match server.tlsa_lookup(format!("_25._tcp.{hostname}.")).await {
+        let dane_policy = match host.dane_status(addresses_dnssec_status) {
+            (DnssecStatus::Secure, _) => {
+                match server.tlsa_lookup(format!("_25._tcp.{hostname}.")).await {
                     Ok(TlsaResult::Secure(tlsa)) if tlsa.has_end_entities => {
                         tx.send(DeliveryStage::TlsaLookupSuccess {
                             record: tlsa.as_ref().clone(),
@@ -445,45 +493,30 @@ async fn delivery_diagnose(
                             continue 'outer;
                         }
                     }
+                }
+            }
+            (DnssecStatus::Bogus, dnssec_entity) => {
+                tx.send(DeliveryStage::TlsaLookupError {
+                    elapsed: now.elapsed_ms(),
+                    reason: format!("Bogus {dnssec_entity} records were found"),
+                })
+                .await?;
+
+                continue 'outer;
+            }
+            (_, dnssec_entity) => {
+                tx.send(DeliveryStage::TlsaNotFound {
+                    elapsed: now.elapsed_ms(),
+                    reason: format!(
+                        "{dnssec_entity} records are not DNSSEC signed, DANE does not apply"
+                    ),
+                })
+                .await?;
+
+                None
+            }
         };
 
-        tx.send(DeliveryStage::IpLookupStart).await?;
-
-        let now = Instant::now();
-        let remote_ips = match host.fqdn_hostname() {
-            HostOrIp::Host(hostname) => {
-                match server
-                    .ip_lookup(&hostname, IpLookupStrategy::Ipv4thenIpv6, usize::MAX, false)
-                    .await
-                {
-                    Ok((remote_ips, _)) if !remote_ips.is_empty() => remote_ips,
-                    Ok(_) => {
-                        tx.send(DeliveryStage::IpLookupError {
-                            reason: "No IP addresses found for host".to_string(),
-                            elapsed: now.elapsed_ms(),
-                        })
-                        .await?;
-                        continue;
-                    }
-                    Err(err) => {
-                        tx.send(DeliveryStage::IpLookupError {
-                            reason: err.to_string(),
-                            elapsed: now.elapsed_ms(),
-                        })
-                        .await?;
-                        continue;
-                    }
-                }
-            }
-            HostOrIp::Ip(ip) => vec![ip],
-        };
-
-        tx.send(DeliveryStage::IpLookupSuccess {
-            remote_ips: remote_ips.clone(),
-            elapsed: now.elapsed_ms(),
-        })
-        .await?;
-
         for remote_ip in remote_ips {
             // Start connection
             tx.send(DeliveryStage::ConnectionStart { remote_ip })

crates/http/src/request.rs

@@ -34,7 +34,7 @@ use hyper::{
     server::conn::http1,
     service::service_fn,
 };
-use hyper_util::rt::TokioIo;
+use hyper_util::rt::{TokioIo, TokioTimer};
 use jmap::{
     api::{
         ToJmapHttpResponse, event_source::EventSourceHandler, request::RequestHandler,
@@ -651,6 +651,7 @@ async fn handle_session<T: SessionStream>(inner: Arc<Inner>, session: SessionDat
     let is_tls = session.stream.is_tls();
 
     if let Err(http_err) = http1::Builder::new()
+        .timer(TokioTimer::new())
         .keep_alive(true)
         .serve_connection(
             TokioIo::new(session.stream),
@@ -795,6 +796,7 @@ async fn handle_session<T: SessionStream>(inner: Arc<Inner>, session: SessionDat
         )
         .with_upgrades()
         .await
+        && !http_err.is_timeout()
     {
         if http_err.is_parse() {
             let server = inner.build_server();

crates/imap-proto/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "imap_proto"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/imap/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "imap"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/imap/src/op/copy_move.rs

@@ -9,6 +9,7 @@ use crate::{
     core::{MailboxId, SelectedMailbox, Session, SessionData},
     spawn_op,
 };
+use ahash::AHashMap;
 use common::{ipc::PushNotification, network::SessionStream, storage::index::ObjectIndexBuilder};
 use email::{
     cache::{MessageCacheFetch, email::MessageCacheAccess},
@@ -22,8 +23,13 @@ use email::{
 use imap_proto::{
     Command, ResponseCode, StatusResponse, protocol::copy_move::Arguments, receiver::Request,
 };
+use rand::RngExt;
 use registry::schema::enums::Permission;
-use std::{sync::Arc, time::Instant};
+use std::{
+    ops::RangeInclusive,
+    sync::Arc,
+    time::{Duration, Instant},
+};
 use store::{
     ValueKey,
     roaring::RoaringBitmap,
@@ -36,6 +42,9 @@ use types::{
     type_state::{DataType, StateChange},
 };
 
+const MAX_MOVE_RETRIES: u32 = 3;
+const MOVE_RETRY_BACKOFF_MS: RangeInclusive<u64> = 1..=15;
+
 impl<T: SessionStream> Session<T> {
     pub async fn handle_copy_move(
         &mut self,
@@ -236,9 +245,15 @@ impl<T: SessionStream> SessionData<T> {
             // Mailboxes are in the same account
             let account_id = src_mailbox.id.account_id;
             let dest_mailbox_id = UidMailbox::new_unassigned(dest_mailbox_id);
-            let mut batch = BatchBuilder::new();
+            let mut written_uids = AHashMap::with_capacity(ids.len());
+            let mut retries = 0;
 
-            for (id, imap_id) in ids {
+            loop {
+                let mut batch = BatchBuilder::new();
+                copied_ids.clear();
+                did_move = false;
+
+                for (&id, imap_id) in &ids {
                     // Obtain mailbox tags
                     let data_ = if let Some(result) = self
                         .get_message_data(account_id, id)
@@ -262,6 +277,13 @@ impl<T: SessionStream> SessionData<T> {
                         .iter()
                         .any(|mailbox| mailbox.mailbox_id == src_mailbox.id.mailbox_id)
                     {
+                        // Moved by a chunk of a previous attempt
+                        if let Some(&uid) = written_uids.get(&id)
+                            && data.inner.message_uid(dest_mailbox_id.mailbox_id) == Some(uid)
+                        {
+                            copied_ids.push((imap_id.uid, uid));
+                            did_move = true;
+                        }
                         continue;
                     }
 
@@ -272,7 +294,8 @@ impl<T: SessionStream> SessionData<T> {
                         .iter()
                         .find(|mailbox| mailbox.mailbox_id == dest_mailbox_id.mailbox_id)
                     {
-                    copied_ids.push((imap_id.uid, mailbox.uid.to_native()));
+                        let uid = mailbox.uid.to_native();
+                        copied_ids.push((imap_id.uid, uid));
 
                         if is_move {
                             let mut new_data = data.inner.to_builder();
@@ -292,6 +315,7 @@ impl<T: SessionStream> SessionData<T> {
                                     (src_mailbox.id.mailbox_id, imap_id.uid),
                                 )
                                 .commit_point();
+                            written_uids.insert(id, uid);
                             did_move = true;
                         }
 
@@ -329,6 +353,7 @@ impl<T: SessionStream> SessionData<T> {
                         .zip(ids)
                     {
                         copied_ids.push((imap_id.uid, uid));
+                        written_uids.insert(id, uid);
                         uid_mailbox.uid = uid;
                     }
 
@@ -353,14 +378,26 @@ impl<T: SessionStream> SessionData<T> {
                     // Add message to training queue
                     if dest_mailbox_id.mailbox_id == JUNK_ID {
                         self.server
-                        .add_account_spam_sample(&mut batch, account_id, id, true, self.session_id)
+                            .add_account_spam_sample(
+                                &mut batch,
+                                account_id,
+                                id,
+                                true,
+                                self.session_id,
+                            )
                             .await
                             .imap_ctx(&arguments.tag, trc::location!())?;
                     } else if src_mailbox.id.mailbox_id == JUNK_ID
                         && dest_mailbox_id.mailbox_id != TRASH_ID
                     {
                         self.server
-                        .add_account_spam_sample(&mut batch, account_id, id, false, self.session_id)
+                            .add_account_spam_sample(
+                                &mut batch,
+                                account_id,
+                                id,
+                                false,
+                                self.session_id,
+                            )
                             .await
                             .imap_ctx(&arguments.tag, trc::location!())?;
                     }
@@ -374,10 +411,14 @@ impl<T: SessionStream> SessionData<T> {
                 }
 
                 // Write changes
-            self.server
-                .commit_batch(batch)
-                .await
-                .imap_ctx(&arguments.tag, trc::location!())?;
+                match self.server.commit_batch(batch).await {
+                    Ok(_) => break,
+                    Err(err) => {
+                        retry_after_conflict(err, &mut retries, &arguments.tag, trc::location!())
+                            .await?
+                    }
+                }
+            }
         } else {
             // Obtain quota for target account
             let src_account_id = src_mailbox.id.account_id;
@@ -400,7 +441,7 @@ impl<T: SessionStream> SessionData<T> {
             let mut train_batch = BatchBuilder::new();
             let mut did_train = false;
             train_batch.with_account_id(src_account_id);
-            for (id, imap_id) in ids {
+            'next_message: for (id, imap_id) in ids {
                 match self
                     .server
                     .copy_message(
@@ -448,6 +489,9 @@ impl<T: SessionStream> SessionData<T> {
                         {
                             copied_ids.push((imap_id.uid, uid));
                         } else {
+                            let mut retries = 0;
+
+                            loop {
                                 let data_ = if let Some(data_) = self
                                     .get_message_data(dest_account_id, existing_id)
                                     .await
@@ -455,7 +499,7 @@ impl<T: SessionStream> SessionData<T> {
                                 {
                                     data_
                                 } else {
-                                continue;
+                                    continue 'next_message;
                                 };
                                 let data = data_
                                     .to_unarchived::<MessageData>()
@@ -463,7 +507,9 @@ impl<T: SessionStream> SessionData<T> {
 
                                 if let Some(uid) = data.inner.message_uid(dest_mailbox_id) {
                                     copied_ids.push((imap_id.uid, uid));
-                            } else {
+                                    break;
+                                }
+
                                 let mut new_data = data.inner.to_builder();
                                 new_data.add_mailbox(UidMailbox::new_unassigned(dest_mailbox_id));
 
@@ -504,15 +550,27 @@ impl<T: SessionStream> SessionData<T> {
                                     )
                                     .imap_ctx(&arguments.tag, trc::location!())?;
 
-                                dest_change_id = self
+                                match self
                                     .server
                                     .commit_batch(batch)
                                     .await
                                     .and_then(|ids| ids.last_change_id(dest_account_id))
-                                    .imap_ctx(&arguments.tag, trc::location!())?
-                                    .into();
-
+                                {
+                                    Ok(change_id) => {
+                                        dest_change_id = change_id.into();
                                         copied_ids.push((imap_id.uid, assigned_uid));
+                                        break;
+                                    }
+                                    Err(err) => {
+                                        retry_after_conflict(
+                                            err,
+                                            &mut retries,
+                                            &arguments.tag,
+                                            trc::location!(),
+                                        )
+                                        .await?
+                                    }
+                                }
                             }
                         }
                     }
@@ -554,6 +612,9 @@ impl<T: SessionStream> SessionData<T> {
 
             // Untag or delete emails
             if !destroy_ids.is_empty() {
+                let mut retries = 0;
+
+                loop {
                     let mut batch = BatchBuilder::new();
                     self.email_untag_or_delete(
                         src_account_id,
@@ -564,10 +625,19 @@ impl<T: SessionStream> SessionData<T> {
                     .await
                     .imap_ctx(&arguments.tag, trc::location!())?;
 
-                self.server
-                    .commit_batch(batch)
-                    .await
-                    .imap_ctx(&arguments.tag, trc::location!())?;
+                    match self.server.commit_batch(batch).await {
+                        Ok(_) => break,
+                        Err(err) => {
+                            retry_after_conflict(
+                                err,
+                                &mut retries,
+                                &arguments.tag,
+                                trc::location!(),
+                            )
+                            .await?
+                        }
+                    }
+                }
 
                 did_move = true;
             }
@@ -731,3 +801,24 @@ impl<T: SessionStream> SessionData<T> {
         }
     }
 }
+
+async fn retry_after_conflict(
+    err: trc::Error,
+    retries: &mut u32,
+    tag: &str,
+    location: &'static str,
+) -> trc::Result<()> {
+    if !err.is_assertion_failure() {
+        Err(err).imap_ctx(tag, location)
+    } else if *retries < MAX_MOVE_RETRIES {
+        *retries += 1;
+        let backoff = rand::rng().random_range(MOVE_RETRY_BACKOFF_MS);
+        tokio::time::sleep(Duration::from_millis(backoff)).await;
+        Ok(())
+    } else {
+        Err(trc::ImapEvent::Error
+            .into_err()
+            .details("Some messages were modified by another process.")
+            .id(tag.to_string()))
+    }
+}

crates/jmap-proto/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "jmap_proto"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/jmap-proto/src/method/import.rs

@@ -12,7 +12,6 @@ use crate::{
         email::{EmailProperty, EmailValue},
     },
     request::{
-        MaybeInvalid,
         deserialize::{DeserializeArguments, deserialize_request},
         reference::{MaybeIdReference, MaybeResultReference, ResultReference},
     },
@@ -33,7 +32,7 @@ pub struct ImportEmailRequest {
 
 #[derive(Debug, Clone, Default)]
 pub struct ImportEmail {
-    pub blob_id: MaybeInvalid<BlobId>,
+    pub blob_id: MaybeIdReference<BlobId>,
     pub mailbox_ids: MaybeResultReference<Vec<MaybeIdReference<Id>>>,
     pub keywords: Vec<Keyword>,
     pub received_at: Option<UTCDate>,

crates/jmap-proto/src/references/resolve.rs

@@ -338,6 +338,10 @@ impl ResolveReference for ImportEmailRequest {
     fn resolve_references(&mut self, response: &Response<'_>) -> trc::Result<()> {
         // Resolve email mailbox references
         for email in self.emails.values_mut() {
+            if let MaybeIdReference::Reference(ir) = &email.blob_id {
+                email.blob_id = MaybeIdReference::Id(response.eval_blob_id_reference(ir)?);
+            }
+
             match &mut email.mailbox_ids {
                 MaybeResultReference::Reference(reference) => {
                     email.mailbox_ids = MaybeResultReference::Value(

crates/jmap-proto/src/request/capability.rs

@@ -89,7 +89,7 @@ pub enum Capability {
     FileNode = 1 << 15,
     #[serde(rename(serialize = "urn:ietf:params:jmap:mail:share"))]
     MailShare = 1 << 16,
-    #[serde(rename(serialize = "urn:stalwart:jmap"))]
+    #[serde(rename(serialize = "urn:inbuxa:jmap:registry"))]
     Stalwart = 1 << 17,
     #[serde(rename(serialize = "urn:ietf:params:jmap:webpush-vapid"))]
     WebPushVapid = 1 << 18,
@@ -331,7 +331,7 @@ impl Capability {
             Capability::PrincipalsAvailability => "urn:ietf:params:jmap:principals:availability",
             Capability::FileNode => "urn:ietf:params:jmap:filenode",
             Capability::MailShare => "urn:ietf:params:jmap:mail:share",
-            Capability::Stalwart => "urn:stalwart:jmap",
+            Capability::Stalwart => "urn:inbuxa:jmap:registry",
             Capability::WebPushVapid => "urn:ietf:params:jmap:webpush-vapid",
             Capability::EmailPush => "urn:ietf:params:jmap:emailpush",
         }
@@ -477,7 +477,7 @@ impl Capability {
             "urn:ietf:params:jmap:contacts:parse" => Capability::ContactsParse,
             "urn:ietf:params:jmap:calendars:parse" => Capability::CalendarsParse,
             "urn:ietf:params:jmap:mail:share" => Capability::MailShare,
-            "urn:stalwart:jmap" => Capability::Stalwart,
+            "urn:inbuxa:jmap:registry" => Capability::Stalwart,
             "urn:ietf:params:jmap:webpush-vapid" => Capability::WebPushVapid,
             "urn:ietf:params:jmap:emailpush" => Capability::EmailPush,
         )

crates/jmap-proto/src/request/reference.rs

@@ -105,6 +105,12 @@ impl<V: Default> Default for MaybeResultReference<V> {
     }
 }
 
+impl<V: FromStr> Default for MaybeIdReference<V> {
+    fn default() -> Self {
+        MaybeIdReference::Invalid(String::new())
+    }
+}
+
 impl<T: Default> MaybeResultReference<T> {
     pub fn unwrap(self) -> T {
         match self {

crates/jmap/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "jmap"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/jmap/src/email/import.rs

@@ -18,7 +18,7 @@ use jmap_proto::{
     error::set::{SetError, SetErrorType},
     method::import::{ImportEmailRequest, ImportEmailResponse},
     object::email::EmailProperty,
-    request::MaybeInvalid,
+    request::reference::MaybeIdReference,
     types::state::State,
 };
 use mail_parser::{HeaderName, MessageParser};
@@ -128,7 +128,7 @@ impl EmailImport for Server {
                 }
             }
 
-            let MaybeInvalid::Value(blob_id) = email.blob_id else {
+            let MaybeIdReference::Id(blob_id) = email.blob_id else {
                 response.not_created.append(
                     id,
                     SetError::invalid_properties()

crates/jmap/src/email/set.rs

@@ -852,8 +852,11 @@ impl EmailSet for Server {
                         new_data.set_mailboxes(
                             ids.into_expanded_boolean_set()
                                 .filter_map(|id| {
-                                    UidMailbox::new_unassigned(
-                                        id.try_into_property()?.try_into_id()?.document_id(),
+                                    let mailbox_id =
+                                        id.try_into_property()?.try_into_id()?.document_id();
+                                    UidMailbox::new(
+                                        mailbox_id,
+                                        data.inner.message_uid(mailbox_id).unwrap_or(0),
                                     )
                                     .into()
                                 })

crates/jmap/src/registry/mapping/bootstrap.rs

@@ -639,6 +639,7 @@ fn map_dns_server(dns_server: &DnsServerBootstrap) -> Option<registry::schema::s
         DnsServerBootstrap::Ns1(inner) => DnsServer::Ns1(inner.clone()).into(),
         DnsServerBootstrap::OracleCloud(inner) => DnsServer::OracleCloud(inner.clone()).into(),
         DnsServerBootstrap::Plesk(inner) => DnsServer::Plesk(inner.clone()).into(),
+        DnsServerBootstrap::PowerDns(inner) => DnsServer::PowerDns(inner.clone()).into(),
         DnsServerBootstrap::Safedns(inner) => DnsServer::Safedns(inner.clone()).into(),
         DnsServerBootstrap::Scaleway(inner) => DnsServer::Scaleway(inner.clone()).into(),
         DnsServerBootstrap::TencentCloud(inner) => DnsServer::TencentCloud(inner.clone()).into(),
@@ -677,7 +678,7 @@ fn build_default_bootstrap(server: &Server) -> Bootstrap {
         directory: DirectoryBootstrap::Internal,
         tracer: Tracer::Log(TracerLog {
             path: "/var/log/stalwart/".to_string(),
-            prefix: "stalwart".to_string(),
+            prefix: "inbuxa".to_string(),
             ansi: true,
             enable: true,
             ..Default::default()

crates/main/Cargo.toml

@@ -7,7 +7,7 @@ homepage = "https://stalw.art"
 keywords = ["imap", "jmap", "smtp", "email", "mail", "webdav", "server"]
 categories = ["email"]
 license = "AGPL-3.0-only OR LicenseRef-SEL"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [[bin]]

crates/managesieve/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "managesieve"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/migration/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "migration"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/nlp/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "nlp"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/pop3/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "pop3"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/registry/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "registry"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/registry/src/schema/enums.rs

@@ -618,6 +618,7 @@ pub enum DnsServerBootstrapType {
     Vultr = 68,
     WebSupport = 69,
     YandexCloud = 70,
+    PowerDns = 71,
 }
 
 #[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Hash)]
@@ -694,6 +695,7 @@ pub enum DnsServerType {
     Vultr = 67,
     WebSupport = 68,
     YandexCloud = 69,
+    PowerDns = 70,
 }
 
 #[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Hash)]

crates/registry/src/schema/enums_impl.rs

@@ -3021,6 +3021,7 @@ impl EnumImpl for DnsServerBootstrapType {
             b"Vultr" => DnsServerBootstrapType::Vultr,
             b"WebSupport" => DnsServerBootstrapType::WebSupport,
             b"YandexCloud" => DnsServerBootstrapType::YandexCloud,
+            b"PowerDns" => DnsServerBootstrapType::PowerDns,
         }
         .copied()
     }
@@ -3098,6 +3099,7 @@ impl EnumImpl for DnsServerBootstrapType {
             DnsServerBootstrapType::Vultr => "Vultr",
             DnsServerBootstrapType::WebSupport => "WebSupport",
             DnsServerBootstrapType::YandexCloud => "YandexCloud",
+            DnsServerBootstrapType::PowerDns => "PowerDns",
         }
     }
 
@@ -3178,11 +3180,12 @@ impl EnumImpl for DnsServerBootstrapType {
             68 => Some(DnsServerBootstrapType::Vultr),
             69 => Some(DnsServerBootstrapType::WebSupport),
             70 => Some(DnsServerBootstrapType::YandexCloud),
+            71 => Some(DnsServerBootstrapType::PowerDns),
             _ => None,
         }
     }
 
-    const COUNT: usize = 71;
+    const COUNT: usize = 72;
 }
 
 impl serde::Serialize for DnsServerBootstrapType {
@@ -3279,6 +3282,7 @@ impl EnumImpl for DnsServerType {
             b"Vultr" => DnsServerType::Vultr,
             b"WebSupport" => DnsServerType::WebSupport,
             b"YandexCloud" => DnsServerType::YandexCloud,
+            b"PowerDns" => DnsServerType::PowerDns,
         }
         .copied()
     }
@@ -3355,6 +3359,7 @@ impl EnumImpl for DnsServerType {
             DnsServerType::Vultr => "Vultr",
             DnsServerType::WebSupport => "WebSupport",
             DnsServerType::YandexCloud => "YandexCloud",
+            DnsServerType::PowerDns => "PowerDns",
         }
     }
 
@@ -3434,11 +3439,12 @@ impl EnumImpl for DnsServerType {
             67 => Some(DnsServerType::Vultr),
             68 => Some(DnsServerType::WebSupport),
             69 => Some(DnsServerType::YandexCloud),
+            70 => Some(DnsServerType::PowerDns),
             _ => None,
         }
     }
 
-    const COUNT: usize = 70;
+    const COUNT: usize = 71;
 }
 
 impl serde::Serialize for DnsServerType {
@@ -10372,8 +10378,8 @@ impl EnumImpl for SieveCapability {
             b"spamtest" => SieveCapability::Spamtest,
             b"spamtestplus" => SieveCapability::Spamtestplus,
             b"virustest" => SieveCapability::Virustest,
-            b"vnd.stalwart.while" => SieveCapability::VndStalwartWhile,
-            b"vnd.stalwart.expressions" => SieveCapability::VndStalwartExpressions,
+            b"vnd.inbuxa.while" => SieveCapability::VndStalwartWhile,
+            b"vnd.inbuxa.expressions" => SieveCapability::VndStalwartExpressions,
         }
     }
 
@@ -10426,8 +10432,8 @@ impl EnumImpl for SieveCapability {
             SieveCapability::Spamtest => "spamtest",
             SieveCapability::Spamtestplus => "spamtestplus",
             SieveCapability::Virustest => "virustest",
-            SieveCapability::VndStalwartWhile => "vnd.stalwart.while",
-            SieveCapability::VndStalwartExpressions => "vnd.stalwart.expressions",
+            SieveCapability::VndStalwartWhile => "vnd.inbuxa.while",
+            SieveCapability::VndStalwartExpressions => "vnd.inbuxa.expressions",
         }
     }
 

crates/registry/src/schema/properties.rs

@@ -1042,6 +1042,7 @@ pub enum Property {
     SentinelUsername = 914,
     Separator = 97,
     ServerHostname = 121,
+    ServerId = 934,
     Servers = 308,
     ServiceAccountJson = 316,
     ServiceName = 913,

crates/registry/src/schema/properties_impl.rs

@@ -1195,6 +1195,7 @@ impl EnumImpl for Property {
             b"sentinelUsername" => Property::SentinelUsername,
             b"separator" => Property::Separator,
             b"serverHostname" => Property::ServerHostname,
+            b"serverId" => Property::ServerId,
             b"servers" => Property::Servers,
             b"serviceAccountJson" => Property::ServiceAccountJson,
             b"serviceName" => Property::ServiceName,
@@ -2134,6 +2135,7 @@ impl EnumImpl for Property {
             Property::SentinelUsername => "sentinelUsername",
             Property::Separator => "separator",
             Property::ServerHostname => "serverHostname",
+            Property::ServerId => "serverId",
             Property::Servers => "servers",
             Property::ServiceAccountJson => "serviceAccountJson",
             Property::ServiceName => "serviceName",
@@ -3077,6 +3079,7 @@ impl EnumImpl for Property {
             914 => Some(Property::SentinelUsername),
             97 => Some(Property::Separator),
             121 => Some(Property::ServerHostname),
+            934 => Some(Property::ServerId),
             308 => Some(Property::Servers),
             316 => Some(Property::ServiceAccountJson),
             913 => Some(Property::ServiceName),
@@ -3227,7 +3230,7 @@ impl EnumImpl for Property {
         }
     }
 
-    const COUNT: usize = 934;
+    const COUNT: usize = 935;
 }
 
 impl serde::Serialize for Property {
@@ -4493,6 +4496,7 @@ impl ObjectInner {
             ObjectInner::DnsServer(DnsServer::Vultr(obj)) => obj.member_tenant_id,
             ObjectInner::DnsServer(DnsServer::WebSupport(obj)) => obj.member_tenant_id,
             ObjectInner::DnsServer(DnsServer::YandexCloud(obj)) => obj.member_tenant_id,
+            ObjectInner::DnsServer(DnsServer::PowerDns(obj)) => obj.member_tenant_id,
             ObjectInner::Domain(obj) => obj.member_tenant_id,
             ObjectInner::MailingList(obj) => obj.member_tenant_id,
             ObjectInner::OAuthClient(obj) => obj.member_tenant_id,
@@ -4595,6 +4599,7 @@ impl ObjectInner {
             ObjectInner::DnsServer(DnsServer::Vultr(obj)) => obj.member_tenant_id = Some(id),
             ObjectInner::DnsServer(DnsServer::WebSupport(obj)) => obj.member_tenant_id = Some(id),
             ObjectInner::DnsServer(DnsServer::YandexCloud(obj)) => obj.member_tenant_id = Some(id),
+            ObjectInner::DnsServer(DnsServer::PowerDns(obj)) => obj.member_tenant_id = Some(id),
             ObjectInner::Domain(obj) => obj.member_tenant_id = Some(id),
             ObjectInner::MailingList(obj) => obj.member_tenant_id = Some(id),
             ObjectInner::OAuthClient(obj) => obj.member_tenant_id = Some(id),

crates/registry/src/schema/structs.rs

@@ -1457,6 +1457,7 @@ pub enum DnsServer {
     Vultr(DnsServerCloud),
     WebSupport(DnsServerWebSupport),
     YandexCloud(DnsServerYandexCloud),
+    PowerDns(DnsServerPowerDns),
 }
 
 #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
@@ -1672,6 +1673,7 @@ pub enum DnsServerBootstrap {
     Vultr(DnsServerCloud),
     WebSupport(DnsServerWebSupport),
     YandexCloud(DnsServerYandexCloud),
+    PowerDns(DnsServerPowerDns),
 }
 
 #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
@@ -2435,6 +2437,31 @@ pub struct DnsServerPorkbun {
     pub propagation_delay: Option<Duration>,
 }
 
+#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
+#[serde(default)]
+pub struct DnsServerPowerDns {
+    #[serde(rename = "apiKey")]
+    pub api_key: SecretKey,
+    #[serde(rename = "endpoint")]
+    pub endpoint: Option<String>,
+    #[serde(rename = "serverId")]
+    pub server_id: Option<String>,
+    #[serde(rename = "description")]
+    pub description: String,
+    #[serde(rename = "memberTenantId")]
+    pub member_tenant_id: Option<Id>,
+    #[serde(rename = "timeout")]
+    pub timeout: Duration,
+    #[serde(rename = "ttl")]
+    pub ttl: Duration,
+    #[serde(rename = "pollingInterval")]
+    pub polling_interval: Duration,
+    #[serde(rename = "propagationTimeout")]
+    pub propagation_timeout: Duration,
+    #[serde(rename = "propagationDelay")]
+    pub propagation_delay: Option<Duration>,
+}
+
 #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
 #[serde(default)]
 pub struct DnsServerRoute53 {

crates/registry/src/schema/structs_impl.rs

@@ -10557,6 +10557,7 @@ impl ObjectImpl for DnsServer {
             DnsServer::Vultr(inner) => inner.validate(errors),
             DnsServer::WebSupport(inner) => inner.validate(errors),
             DnsServer::YandexCloud(inner) => inner.validate(errors),
+            DnsServer::PowerDns(inner) => inner.validate(errors),
         }
     }
 
@@ -10770,6 +10771,9 @@ impl ObjectImpl for DnsServer {
             DnsServer::YandexCloud(object) => {
                 object.index(i);
             }
+            DnsServer::PowerDns(object) => {
+                object.index(i);
+            }
         }
     }
 }
@@ -11062,6 +11066,10 @@ impl Pickle for DnsServer {
                 69u16.pickle(out);
                 inner.pickle(out);
             }
+            DnsServer::PowerDns(inner) => {
+                70u16.pickle(out);
+                inner.pickle(out);
+            }
         }
     }
 
@@ -11137,6 +11145,7 @@ impl Pickle for DnsServer {
             67 => Pickle::unpickle(stream).map(DnsServer::Vultr),
             68 => Pickle::unpickle(stream).map(DnsServer::WebSupport),
             69 => Pickle::unpickle(stream).map(DnsServer::YandexCloud),
+            70 => Pickle::unpickle(stream).map(DnsServer::PowerDns),
             _ => None,
         }
     }
@@ -11633,6 +11642,13 @@ impl IntoValue for DnsServer {
                     .insert_unchecked(Property::Type, JmapValue::Str("YandexCloud".into()));
                 obj
             }
+            DnsServer::PowerDns(obj) => {
+                let mut obj = obj.into_value();
+                obj.as_object_mut()
+                    .unwrap()
+                    .insert_unchecked(Property::Type, JmapValue::Str("PowerDns".into()));
+                obj
+            }
         }
     }
 }
@@ -11717,6 +11733,7 @@ impl RegistryJsonPatch for DnsServer {
                 DnsServerType::Vultr => *self = DnsServer::Vultr(Default::default()),
                 DnsServerType::WebSupport => *self = DnsServer::WebSupport(Default::default()),
                 DnsServerType::YandexCloud => *self = DnsServer::YandexCloud(Default::default()),
+                DnsServerType::PowerDns => *self = DnsServer::PowerDns(Default::default()),
             }
         }
         match self {
@@ -11790,6 +11807,7 @@ impl RegistryJsonPatch for DnsServer {
             DnsServer::Vultr(inner) => inner.patch(pointer, value),
             DnsServer::WebSupport(inner) => inner.patch(pointer, value),
             DnsServer::YandexCloud(inner) => inner.patch(pointer, value),
+            DnsServer::PowerDns(inner) => inner.patch(pointer, value),
         }
     }
 }
@@ -11867,6 +11885,7 @@ impl DnsServer {
             DnsServer::Vultr(_) => DnsServerType::Vultr,
             DnsServer::WebSupport(_) => DnsServerType::WebSupport,
             DnsServer::YandexCloud(_) => DnsServerType::YandexCloud,
+            DnsServer::PowerDns(_) => DnsServerType::PowerDns,
         }
     }
 }
@@ -12702,6 +12721,7 @@ impl DnsServerBootstrap {
             DnsServerBootstrap::Vultr(inner) => inner.validate(errors),
             DnsServerBootstrap::WebSupport(inner) => inner.validate(errors),
             DnsServerBootstrap::YandexCloud(inner) => inner.validate(errors),
+            DnsServerBootstrap::PowerDns(inner) => inner.validate(errors),
         }
     }
 }
@@ -12997,6 +13017,10 @@ impl Pickle for DnsServerBootstrap {
                 70u16.pickle(out);
                 inner.pickle(out);
             }
+            DnsServerBootstrap::PowerDns(inner) => {
+                71u16.pickle(out);
+                inner.pickle(out);
+            }
         }
     }
 
@@ -13073,6 +13097,7 @@ impl Pickle for DnsServerBootstrap {
             68 => Pickle::unpickle(stream).map(DnsServerBootstrap::Vultr),
             69 => Pickle::unpickle(stream).map(DnsServerBootstrap::WebSupport),
             70 => Pickle::unpickle(stream).map(DnsServerBootstrap::YandexCloud),
+            71 => Pickle::unpickle(stream).map(DnsServerBootstrap::PowerDns),
             _ => None,
         }
     }
@@ -13574,6 +13599,13 @@ impl IntoValue for DnsServerBootstrap {
                     .insert_unchecked(Property::Type, JmapValue::Str("YandexCloud".into()));
                 obj
             }
+            DnsServerBootstrap::PowerDns(obj) => {
+                let mut obj = obj.into_value();
+                obj.as_object_mut()
+                    .unwrap()
+                    .insert_unchecked(Property::Type, JmapValue::Str("PowerDns".into()));
+                obj
+            }
         }
     }
 }
@@ -13791,6 +13823,9 @@ impl RegistryJsonPatch for DnsServerBootstrap {
                 DnsServerBootstrapType::YandexCloud => {
                     *self = DnsServerBootstrap::YandexCloud(Default::default())
                 }
+                DnsServerBootstrapType::PowerDns => {
+                    *self = DnsServerBootstrap::PowerDns(Default::default())
+                }
             }
         }
         match self {
@@ -13865,6 +13900,7 @@ impl RegistryJsonPatch for DnsServerBootstrap {
             DnsServerBootstrap::Vultr(inner) => inner.patch(pointer, value),
             DnsServerBootstrap::WebSupport(inner) => inner.patch(pointer, value),
             DnsServerBootstrap::YandexCloud(inner) => inner.patch(pointer, value),
+            DnsServerBootstrap::PowerDns(inner) => inner.patch(pointer, value),
         }
     }
 }
@@ -13943,6 +13979,7 @@ impl DnsServerBootstrap {
             DnsServerBootstrap::Vultr(_) => DnsServerBootstrapType::Vultr,
             DnsServerBootstrap::WebSupport(_) => DnsServerBootstrapType::WebSupport,
             DnsServerBootstrap::YandexCloud(_) => DnsServerBootstrapType::YandexCloud,
+            DnsServerBootstrap::PowerDns(_) => DnsServerBootstrapType::PowerDns,
         }
     }
 }
@@ -18226,6 +18263,148 @@ impl RegistryJsonPropertyPatch for DnsServerPorkbun {
     }
 }
 
+impl DnsServerPowerDns {
+    fn validate(&self, errors: &mut Vec<ValidationError>) -> bool {
+        let neb = errors.len();
+        let value = &self.api_key;
+        value.validate(errors);
+        if let Some(value) = &self.endpoint {
+            if value.is_empty() {
+                errors.push(ValidationError::required(Property::Endpoint));
+            }
+        }
+        if let Some(value) = &self.server_id {
+            if value.is_empty() {
+                errors.push(ValidationError::required(Property::ServerId));
+            }
+        }
+        let value = &self.description;
+        if value.is_empty() {
+            errors.push(ValidationError::required(Property::Description));
+        }
+        if let Some(value) = &self.member_tenant_id {
+            if !value.is_valid() {
+                errors.push(ValidationError::required(Property::MemberTenantId));
+            }
+        }
+        errors.len() == neb
+    }
+
+    fn index<'x>(&'x self, i: &mut IndexBuilder<'x>) {
+        i.foreign_key(ObjectType::Tenant, self.member_tenant_id, None);
+        if let Some(value) = &self.member_tenant_id {
+            i.search(Property::MemberTenantId, value);
+        }
+    }
+}
+
+impl Pickle for DnsServerPowerDns {
+    fn pickle(&self, out: &mut Vec<u8>) {
+        self.api_key.pickle(out);
+        self.endpoint.pickle(out);
+        self.server_id.pickle(out);
+        self.description.pickle(out);
+        self.member_tenant_id.pickle(out);
+        self.timeout.pickle(out);
+        self.ttl.pickle(out);
+        self.polling_interval.pickle(out);
+        self.propagation_timeout.pickle(out);
+        self.propagation_delay.pickle(out);
+    }
+
+    fn unpickle(stream: &mut crate::pickle::PickledStream<'_>) -> Option<Self> {
+        let mut this = Self::default();
+        this.api_key = Pickle::unpickle(stream)?;
+        this.endpoint = Pickle::unpickle(stream)?;
+        this.server_id = Pickle::unpickle(stream)?;
+        this.description = Pickle::unpickle(stream)?;
+        this.member_tenant_id = Pickle::unpickle(stream)?;
+        this.timeout = Pickle::unpickle(stream)?;
+        this.ttl = Pickle::unpickle(stream)?;
+        this.polling_interval = Pickle::unpickle(stream)?;
+        this.propagation_timeout = Pickle::unpickle(stream)?;
+        this.propagation_delay = Pickle::unpickle(stream)?;
+        Some(this)
+    }
+}
+
+impl Default for DnsServerPowerDns {
+    fn default() -> Self {
+        Self {
+            api_key: Default::default(),
+            endpoint: Default::default(),
+            server_id: Default::default(),
+            description: Default::default(),
+            member_tenant_id: Default::default(),
+            timeout: Duration::from_millis(30000),
+            ttl: Duration::from_millis(300000),
+            polling_interval: Duration::from_millis(15000),
+            propagation_timeout: Duration::from_millis(60000),
+            propagation_delay: Default::default(),
+        }
+    }
+}
+
+impl IntoValue for DnsServerPowerDns {
+    fn into_value(self) -> JmapValue<'static> {
+        let mut map = jmap_tools::Map::with_capacity(12);
+        map.insert_unchecked(Property::ApiKey, self.api_key.into_value());
+        map.insert_unchecked(Property::Endpoint, self.endpoint.into_value());
+        map.insert_unchecked(Property::ServerId, self.server_id.into_value());
+        map.insert_unchecked(Property::Description, self.description.into_value());
+        map.insert_unchecked(Property::MemberTenantId, self.member_tenant_id.into_value());
+        map.insert_unchecked(Property::Timeout, self.timeout.into_value());
+        map.insert_unchecked(Property::Ttl, self.ttl.into_value());
+        map.insert_unchecked(
+            Property::PollingInterval,
+            self.polling_interval.into_value(),
+        );
+        map.insert_unchecked(
+            Property::PropagationTimeout,
+            self.propagation_timeout.into_value(),
+        );
+        map.insert_unchecked(
+            Property::PropagationDelay,
+            self.propagation_delay.into_value(),
+        );
+        JmapValue::Object(map)
+    }
+}
+
+impl RegistryJsonPropertyPatch for DnsServerPowerDns {
+    fn patch_property<'x>(
+        &mut self,
+        mut pointer: JsonPointerPatch<'_>,
+        value: JmapValue<'x>,
+    ) -> PatchResult<'x> {
+        match pointer.next_property() {
+            Some(Property::ApiKey) => self.api_key.patch(pointer, value),
+            Some(Property::Endpoint) => self
+                .endpoint
+                .patch(pointer.with_validators(&[StringValidator::Trim]), value),
+            Some(Property::ServerId) => self
+                .server_id
+                .patch(pointer.with_validators(&[StringValidator::Trim]), value),
+            Some(Property::Description) => self
+                .description
+                .patch(pointer.with_validators(&[StringValidator::Trim]), value),
+            Some(Property::MemberTenantId) => self
+                .member_tenant_id
+                .patch(pointer.assert_can_set_tenant()?, value),
+            Some(Property::Timeout) => self.timeout.patch(pointer, value),
+            Some(Property::Ttl) => self.ttl.patch(pointer, value),
+            Some(Property::PollingInterval) => self.polling_interval.patch(pointer, value),
+            Some(Property::PropagationTimeout) => self.propagation_timeout.patch(pointer, value),
+            Some(Property::PropagationDelay) => self.propagation_delay.patch(pointer, value),
+            Some(Property::Type) => Ok(MaybeUnpatched::Unpatched {
+                property: Property::Type,
+                value,
+            }),
+            _ => Err(PatchError::new(pointer, "Invalid property")),
+        }
+    }
+}
+
 impl DnsServerRoute53 {
     fn validate(&self, errors: &mut Vec<ValidationError>) -> bool {
         let neb = errors.len();
@@ -29620,8 +29799,8 @@ impl Default for MySqlSettings {
         Self {
             host: Default::default(),
             port: 3306u64,
-            database: "stalwart".to_string(),
-            auth_username: Some("stalwart".to_string()),
+            database: "inbuxa".to_string(),
+            auth_username: Some("inbuxa".to_string()),
             auth_secret: Default::default(),
         }
     }
@@ -29778,8 +29957,8 @@ impl Default for MySqlStore {
             read_replicas: Default::default(),
             host: Default::default(),
             port: 3306u64,
-            database: "stalwart".to_string(),
-            auth_username: Some("stalwart".to_string()),
+            database: "inbuxa".to_string(),
+            auth_username: Some("inbuxa".to_string()),
             auth_secret: Default::default(),
         }
     }
@@ -29940,7 +30119,7 @@ impl Default for NatsCoordinator {
             no_echo: true,
             use_tls: false,
             auth_secret: Default::default(),
-            auth_username: Some("stalwart".to_string()),
+            auth_username: Some("inbuxa".to_string()),
             credentials: Default::default(),
         }
     }
@@ -31122,8 +31301,8 @@ impl Default for PostgreSqlSettings {
         Self {
             host: Default::default(),
             port: 5432u64,
-            database: "stalwart".to_string(),
-            auth_username: Some("stalwart".to_string()),
+            database: "inbuxa".to_string(),
+            auth_username: Some("inbuxa".to_string()),
             auth_secret: Default::default(),
             options: Default::default(),
         }
@@ -31267,8 +31446,8 @@ impl Default for PostgreSqlStore {
             read_replicas: Default::default(),
             host: Default::default(),
             port: 5432u64,
-            database: "stalwart".to_string(),
-            auth_username: Some("stalwart".to_string()),
+            database: "inbuxa".to_string(),
+            auth_username: Some("inbuxa".to_string()),
             auth_secret: Default::default(),
             options: Default::default(),
         }
@@ -32573,7 +32752,7 @@ impl Default for RedisClusterStore {
         Self {
             urls: Map::new(vec!["redis://127.0.0.1".to_string()]),
             timeout: Duration::from_millis(10000),
-            auth_username: Some("stalwart".to_string()),
+            auth_username: Some("inbuxa".to_string()),
             auth_secret: Default::default(),
             max_retry_wait: Default::default(),
             min_retry_wait: Default::default(),
@@ -32740,7 +32919,7 @@ impl Default for RedisSentinelStore {
             urls: Map::new(vec!["redis://127.0.0.1:26379".to_string()]),
             service_name: "mymaster".to_string(),
             timeout: Duration::from_millis(10000),
-            auth_username: Some("stalwart".to_string()),
+            auth_username: Some("inbuxa".to_string()),
             auth_secret: Default::default(),
             sentinel_username: Default::default(),
             sentinel_secret: Default::default(),
@@ -40212,7 +40391,7 @@ impl Default for SpamSettings {
             score_reject: Float::new(0.0f64),
             score_spam: Float::new(5.0f64),
             trust_replies: true,
-            spam_filter_rules_url: Some("https://github.com/stalwartlabs/spam-filter/releases/latest/download/spam-filter-rules.json.gz".to_string()),
+            spam_filter_rules_url: None,
         }
     }
 }
@@ -46305,7 +46484,7 @@ impl Default for TracerLog {
     fn default() -> Self {
         Self {
             path: Default::default(),
-            prefix: "stalwart".to_string(),
+            prefix: "inbuxa".to_string(),
             rotate: LogRotateFrequency::Daily,
             ansi: true,
             multiline: false,

crates/registry/src/utils/mod.rs

@@ -14,6 +14,7 @@ pub mod dkim;
 pub mod http;
 pub mod report;
 pub mod secret;
+pub mod spam;
 pub mod task;
 
 impl Roles {

crates/registry/src/utils/spam.rs

@@ -0,0 +1,59 @@
+/*
+ * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
+ *
+ * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
+ */
+
+use crate::schema::prelude::{SpamDnsblServer, SpamRule};
+
+impl SpamRule {
+    pub fn enable(&self) -> bool {
+        match self {
+            SpamRule::Any(rule) => rule.enable,
+            SpamRule::Url(rule) => rule.enable,
+            SpamRule::Domain(rule) => rule.enable,
+            SpamRule::Email(rule) => rule.enable,
+            SpamRule::Ip(rule) => rule.enable,
+            SpamRule::Header(rule) => rule.enable,
+            SpamRule::Body(rule) => rule.enable,
+        }
+    }
+
+    pub fn set_enable(&mut self, enable: bool) {
+        match self {
+            SpamRule::Any(rule) => rule.enable = enable,
+            SpamRule::Url(rule) => rule.enable = enable,
+            SpamRule::Domain(rule) => rule.enable = enable,
+            SpamRule::Email(rule) => rule.enable = enable,
+            SpamRule::Ip(rule) => rule.enable = enable,
+            SpamRule::Header(rule) => rule.enable = enable,
+            SpamRule::Body(rule) => rule.enable = enable,
+        }
+    }
+}
+
+impl SpamDnsblServer {
+    pub fn enable(&self) -> bool {
+        match self {
+            SpamDnsblServer::Any(server) => server.enable,
+            SpamDnsblServer::Url(server) => server.enable,
+            SpamDnsblServer::Domain(server) => server.enable,
+            SpamDnsblServer::Email(server) => server.enable,
+            SpamDnsblServer::Ip(server) => server.enable,
+            SpamDnsblServer::Header(server) => server.enable,
+            SpamDnsblServer::Body(server) => server.enable,
+        }
+    }
+
+    pub fn set_enable(&mut self, enable: bool) {
+        match self {
+            SpamDnsblServer::Any(server) => server.enable = enable,
+            SpamDnsblServer::Url(server) => server.enable = enable,
+            SpamDnsblServer::Domain(server) => server.enable = enable,
+            SpamDnsblServer::Email(server) => server.enable = enable,
+            SpamDnsblServer::Ip(server) => server.enable = enable,
+            SpamDnsblServer::Header(server) => server.enable = enable,
+            SpamDnsblServer::Body(server) => server.enable = enable,
+        }
+    }
+}

crates/scim-proto/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "scim-proto"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/scim/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "scim"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/services/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "services"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/services/src/state_manager/http.rs

@@ -9,7 +9,7 @@ use super::{
     ece::{ECE_WEBPUSH_MAX_PLAINTEXT_SIZE, WEBPUSH_MAX_BODY_SIZE, ece_encrypt},
     email_push::build_email_push_object,
 };
-use crate::state_manager::PushRegistration;
+use crate::state_manager::{PushBatch, PushRegistration};
 use calcard::jscalendar::JSCalendarDateTime;
 use common::{Server, ipc::PushNotification, network::webpush::Vapid};
 use email::push::{PushSubscription, Urgency};
@@ -29,7 +29,7 @@ use std::time::{Duration, Instant};
 use store::write::now;
 use tokio::sync::mpsc;
 use trc::PushSubscriptionEvent;
-use types::{id::Id, type_state::DataType};
+use types::id::Id;
 use utils::map::vec_map::VecMap;
 
 const MAX_ERROR_RESPONSE_LEN: usize = 1024;
@@ -48,34 +48,29 @@ impl PushRegistration {
     pub fn send(
         &mut self,
         id: Id,
+        push_client: &Client,
         push_tx: mpsc::Sender<Event>,
         push_timeout: Duration,
         server: Server,
     ) {
         let subscription = self.server.clone();
-        let push_client = self.client.clone();
-        let notifications = std::mem::take(&mut self.notifications);
+        let push_client = push_client.clone();
+        let batch = std::mem::take(&mut self.pending);
 
         self.in_flight = true;
         self.last_request = Instant::now();
 
         tokio::spawn(async move {
-            let mut changed: VecMap<Id, VecMap<DataType, State>> = VecMap::new();
+            let vapid = server.core.jmap.vapid.as_ref();
             let mut email_pushes: VecMap<Id, EmailPushObject> = VecMap::new();
 
-            let mut failed_state_change = false;
+            let mut failed = PushBatch::default();
             let mut failed_email_pushes = Vec::new();
             let mut failed_calendar_alerts = Vec::new();
 
-            for notification in &notifications {
+            for notification in &batch.notifications {
                 match notification {
-                    PushNotification::StateChange(state_change) => {
-                        for type_state in state_change.types {
-                            changed
-                                .get_mut_or_insert(state_change.account_id.into())
-                                .set(type_state, State::Exact(state_change.change_id));
-                        }
-                    }
+                    PushNotification::StateChange(_) => {}
                     PushNotification::CalendarAlert(calendar_alert) => {
                         let payload = PushObject::CalendarAlert {
                             account_id: calendar_alert.account_id.into(),
@@ -86,12 +81,12 @@ impl PushRegistration {
                             }),
                             alert_id: calendar_alert.alert_id.clone(),
                         };
-                        if !http_request(
+                        if !post_object(
                             &push_client,
                             &subscription,
-                            serde_json::to_string(&payload).unwrap().into_bytes(),
+                            &payload,
                             push_timeout,
-                            server.core.jmap.vapid.as_ref(),
+                            vapid,
                             Urgency::Normal,
                         )
                         .await
@@ -155,18 +150,23 @@ impl PushRegistration {
                 }
             }
 
-            if !changed.is_empty() {
-                failed_state_change = !http_request(
+            if !batch.state_changes.is_empty() {
+                let payload = PushObject::StateChange {
+                    changed: batch.state_changes,
+                };
+                if !post_object(
                     &push_client,
                     &subscription,
-                    serde_json::to_string(&PushObject::StateChange { changed })
-                        .unwrap()
-                        .into_bytes(),
+                    &payload,
                     push_timeout,
-                    server.core.jmap.vapid.as_ref(),
+                    vapid,
                     Urgency::Normal,
                 )
-                .await;
+                .await
+                    && let PushObject::StateChange { changed } = payload
+                {
+                    failed.state_changes = changed;
+                }
             }
 
             for (account_id, email_push) in email_pushes {
@@ -180,12 +180,12 @@ impl PushRegistration {
                     state: email_push.change_id.map(State::Exact),
                 };
 
-                if !http_request(
+                if !post_object(
                     &push_client,
                     &subscription,
-                    serde_json::to_string(&payload).unwrap().into_bytes(),
+                    &payload,
                     push_timeout,
-                    server.core.jmap.vapid.as_ref(),
+                    vapid,
                     email_push.urgency,
                 )
                 .await
@@ -194,44 +194,26 @@ impl PushRegistration {
                 }
             }
 
-            let result = if !failed_state_change
+            let result = if failed.state_changes.is_empty()
                 && failed_email_pushes.is_empty()
                 && failed_calendar_alerts.is_empty()
             {
                 Event::DeliverySuccess { id }
             } else {
-                let mut failed_notifications = Vec::with_capacity(
-                    failed_state_change as usize
-                        + failed_email_pushes.len()
-                        + failed_calendar_alerts.len(),
-                );
-
-                for notification in notifications {
-                    match &notification {
-                        PushNotification::StateChange(_) => {
-                            if failed_state_change {
-                                failed_notifications.push(notification);
-                            }
-                        }
+                failed.notifications = batch
+                    .notifications
+                    .into_iter()
+                    .filter(|notification| match notification {
+                        PushNotification::StateChange(_) => false,
                         PushNotification::EmailPush(email_push) => {
-                            if failed_email_pushes.contains(&email_push.account_id) {
-                                failed_notifications.push(notification);
-                            }
-                        }
-                        PushNotification::CalendarAlert(calendar_alert) => {
-                            if failed_calendar_alerts
-                                .contains(&(calendar_alert.account_id, calendar_alert.event_id))
-                            {
-                                failed_notifications.push(notification);
-                            }
-                        }
-                    }
+                            failed_email_pushes.contains(&email_push.account_id)
                         }
+                        PushNotification::CalendarAlert(calendar_alert) => failed_calendar_alerts
+                            .contains(&(calendar_alert.account_id, calendar_alert.event_id)),
+                    })
+                    .collect();
 
-                Event::DeliveryFailure {
-                    id,
-                    notifications: failed_notifications,
-                }
+                Event::DeliveryFailure { id, failed }
             };
 
             push_tx.send(result).await.ok();
@@ -239,6 +221,38 @@ impl PushRegistration {
     }
 }
 
+async fn post_object(
+    push_client: &Client,
+    subscription: &PushSubscription,
+    object: &PushObject,
+    push_timeout: Duration,
+    vapid: Option<&Vapid>,
+    urgency: Urgency,
+) -> bool {
+    match serde_json::to_vec(object) {
+        Ok(body) => {
+            http_request(
+                push_client,
+                subscription,
+                body,
+                push_timeout,
+                vapid,
+                urgency,
+            )
+            .await
+        }
+        Err(err) => {
+            trc::event!(
+                PushSubscription(PushSubscriptionEvent::Error),
+                Details = "Failed to serialize push object",
+                Url = subscription.url.to_string(),
+                Reason = err.to_string()
+            );
+            true
+        }
+    }
+}
+
 pub(crate) fn build_push_client() -> Client {
     utils::http::http_client_builder(cfg!(feature = "test_mode"))
         .redirect(Policy::custom(|attempt| match attempt.previous().last() {

crates/services/src/state_manager/manager.rs

@@ -12,7 +12,7 @@ use common::{
 };
 use std::{sync::Arc, time::Instant};
 use store::ahash::AHashMap;
-use tokio::sync::mpsc;
+use tokio::sync::mpsc::{self, error::TrySendError};
 use trc::ServerEvent;
 
 #[derive(Default)]
@@ -84,7 +84,7 @@ pub fn spawn_push_router(inner: Arc<Inner>, mut change_rx: mpsc::Receiver<PushEv
                 } => {
                     // Publish event to cluster
                     if broadcast
-                        && let Some(broadcast_tx) = &inner.ipc.broadcast_tx.clone()
+                        && let Some(broadcast_tx) = inner.ipc.broadcast_tx.as_ref()
                         && broadcast_tx
                             .send(BroadcastEvent::PushNotification(notification.clone()))
                             .await
@@ -102,7 +102,9 @@ pub fn spawn_push_router(inner: Arc<Inner>, mut change_rx: mpsc::Receiver<PushEv
                         for subscriber in &subscribers.ipc {
                             if let Some(notification) = notification.filter_types(&subscriber.types)
                             {
-                                if subscriber.is_valid() {
+                                match subscriber.tx.try_send(notification) {
+                                    Ok(()) => {}
+                                    Err(TrySendError::Full(notification)) => {
                                         let subscriber_tx = subscriber.tx.clone();
 
                                         tokio::spawn(async move {
@@ -120,11 +122,13 @@ pub fn spawn_push_router(inner: Arc<Inner>, mut change_rx: mpsc::Receiver<PushEv
                                                 );
                                             }
                                         });
-                                } else {
+                                    }
+                                    Err(TrySendError::Closed(_)) => {
                                         purge_needed = true;
                                     }
                                 }
                             }
+                        }
 
                         if subscribers.is_push
                             && push_tx.send(Event::Push { notification }).await.is_err()
@@ -144,7 +148,7 @@ pub fn spawn_push_router(inner: Arc<Inner>, mut change_rx: mpsc::Receiver<PushEv
                 } => {
                     // Publish event to cluster
                     if broadcast
-                        && let Some(broadcast_tx) = &inner.ipc.broadcast_tx.clone()
+                        && let Some(broadcast_tx) = inner.ipc.broadcast_tx.as_ref()
                         && broadcast_tx
                             .send(BroadcastEvent::PushServerUpdate(account_id))
                             .await

crates/services/src/state_manager/mod.rs

@@ -12,14 +12,14 @@ pub mod push;
 
 use common::ipc::PushNotification;
 use email::push::PushSubscription;
-use reqwest::Client;
+use jmap_proto::types::state::State;
 use std::{
     sync::Arc,
     time::{Duration, Instant},
 };
 use tokio::sync::mpsc;
 use types::{id::Id, type_state::DataType};
-use utils::map::bitmap::Bitmap;
+use utils::map::{bitmap::Bitmap, vec_map::VecMap};
 
 const PURGE_EVERY: Duration = Duration::from_secs(3600);
 const SEND_TIMEOUT: Duration = Duration::from_millis(500);
@@ -36,26 +36,22 @@ pub struct PushRegistration {
     member_account_ids: Vec<u32>,
     num_attempts: u32,
     last_request: Instant,
-    notifications: Vec<PushNotification>,
+    pending: PushBatch,
     in_flight: bool,
-    client: Client,
+}
+
+#[derive(Debug, Default)]
+pub struct PushBatch {
+    state_changes: VecMap<Id, VecMap<DataType, State>>,
+    notifications: Vec<PushNotification>,
 }
 
 #[derive(Debug)]
 pub enum Event {
-    Push {
-        notification: PushNotification,
-    },
-    Update {
-        account_id: u32,
-    },
-    DeliverySuccess {
-        id: Id,
-    },
-    DeliveryFailure {
-        id: Id,
-        notifications: Vec<PushNotification>,
-    },
+    Push { notification: PushNotification },
+    Update { account_id: u32 },
+    DeliverySuccess { id: Id },
+    DeliveryFailure { id: Id, failed: PushBatch },
     Reset,
 }
 
@@ -64,3 +60,130 @@ impl IpcSubscriber {
         !self.tx.is_closed()
     }
 }
+
+impl PushBatch {
+    pub fn push(&mut self, notification: PushNotification) {
+        match notification {
+            PushNotification::StateChange(state_change) => {
+                if !state_change.types.is_empty() {
+                    let states = self
+                        .state_changes
+                        .get_mut_or_insert(Id::from(state_change.account_id));
+                    for data_type in state_change.types {
+                        merge_state(states, data_type, state_change.change_id);
+                    }
+                }
+            }
+            notification => self.notifications.push(notification),
+        }
+    }
+
+    pub fn merge_failed(&mut self, failed: PushBatch) {
+        for (account_id, failed_states) in failed.state_changes {
+            let states = self.state_changes.get_mut_or_insert(account_id);
+            for (data_type, state) in failed_states {
+                if let State::Exact(change_id) = state {
+                    merge_state(states, data_type, change_id);
+                }
+            }
+        }
+
+        if !failed.notifications.is_empty() {
+            let mut notifications = failed.notifications;
+            notifications.append(&mut self.notifications);
+            self.notifications = notifications;
+        }
+    }
+
+    pub fn is_empty(&self) -> bool {
+        self.state_changes.is_empty() && self.notifications.is_empty()
+    }
+
+    pub fn clear(&mut self) {
+        self.state_changes.clear();
+        self.notifications.clear();
+    }
+}
+
+fn merge_state(states: &mut VecMap<DataType, State>, data_type: DataType, change_id: u64) {
+    match states.get_mut(&data_type) {
+        Some(State::Exact(current)) if *current >= change_id => {}
+        Some(state) => *state = State::Exact(change_id),
+        None => states.append(data_type, State::Exact(change_id)),
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::PushBatch;
+    use common::ipc::{EmailPush, PushNotification};
+    use jmap_proto::types::state::State;
+    use types::{
+        id::Id,
+        type_state::{DataType, StateChange},
+    };
+    use utils::map::bitmap::Bitmap;
+
+    fn state_change<const N: usize>(change_id: u64, types: [DataType; N]) -> PushNotification {
+        PushNotification::StateChange(StateChange {
+            account_id: 1,
+            change_id,
+            types: Bitmap::from_iter(types),
+        })
+    }
+
+    fn email_push(email_id: u32) -> PushNotification {
+        PushNotification::EmailPush(EmailPush {
+            account_id: 1,
+            email_id,
+            change_id: email_id.into(),
+        })
+    }
+
+    #[test]
+    fn batch_keeps_newest_state_per_type() {
+        let mut batch = PushBatch::default();
+        batch.push(state_change(5, [DataType::Email]));
+        batch.push(state_change(7, [DataType::Email, DataType::Mailbox]));
+        batch.push(state_change(6, [DataType::Mailbox]));
+
+        let mut failed = PushBatch::default();
+        failed.push(state_change(
+            4,
+            [DataType::Email, DataType::Mailbox, DataType::Thread],
+        ));
+        batch.merge_failed(failed);
+
+        let states = batch.state_changes.get(&Id::from(1u32)).unwrap();
+        assert_eq!(states.len(), 3);
+        assert_eq!(states.get(&DataType::Email), Some(&State::Exact(7)));
+        assert_eq!(states.get(&DataType::Mailbox), Some(&State::Exact(7)));
+        assert_eq!(states.get(&DataType::Thread), Some(&State::Exact(4)));
+
+        assert!(!batch.is_empty());
+        batch.clear();
+        assert!(batch.is_empty());
+    }
+
+    #[test]
+    fn failed_notifications_are_retried_first() {
+        let mut batch = PushBatch::default();
+        batch.push(email_push(3));
+
+        let mut failed = PushBatch::default();
+        failed.push(email_push(1));
+        failed.push(email_push(2));
+        batch.merge_failed(failed);
+
+        let email_ids = batch
+            .notifications
+            .iter()
+            .filter_map(|notification| match notification {
+                PushNotification::EmailPush(email_push) => Some(email_push.email_id),
+                _ => None,
+            })
+            .collect::<Vec<_>>();
+        assert_eq!(email_ids, [1, 2, 3]);
+        assert!(batch.state_changes.is_empty());
+    }
+}

crates/services/src/state_manager/push.rs

@@ -8,13 +8,14 @@ use super::{
     Event,
     http::{build_push_client, http_request},
 };
-use crate::state_manager::PushRegistration;
+use crate::state_manager::{PushBatch, PushRegistration};
 use common::{
     BuildServer, IPC_CHANNEL_BUFFER, Inner, LONG_1Y_SLUMBER, Server,
     auth::BuildAccessToken,
     ipc::{PushEvent, PushNotification},
 };
 use email::push::{PushSubscription, PushSubscriptions, Urgency};
+use reqwest::Client;
 use std::{
     collections::hash_map::Entry,
     sync::Arc,
@@ -36,7 +37,10 @@ pub fn spawn_push_manager(inner: Arc<Inner>) -> mpsc::Sender<Event> {
     tokio::spawn(async move {
         let mut push_servers: AHashMap<Id, PushRegistration> = AHashMap::default();
         let mut account_push_ids: AHashMap<u32, AHashSet<Id>> = AHashMap::default();
-        let mut last_verify: AHashMap<u32, Instant> = AHashMap::default();
+        let mut last_verify: AHashMap<u32, (Instant, u32)> = AHashMap::default();
+        let mut pending_verify: AHashMap<u32, (Instant, Arc<PushSubscription>)> =
+            AHashMap::default();
+        let mut next_verify: Option<Instant> = None;
         let mut last_retry = Instant::now();
         let mut retry_timeout = LONG_1Y_SLUMBER;
         let mut retry_ids = AHashSet::default();
@@ -90,10 +94,9 @@ pub fn spawn_push_manager(inner: Arc<Inner>) -> mpsc::Sender<Event> {
                                             last_request: Instant::now()
                                                 - (server.core.jmap.push_throttle
                                                     + Duration::from_millis(1)),
-                                            notifications: Vec::new(),
+                                            pending: PushBatch::default(),
                                             server: subscription.clone(),
                                             in_flight: false,
-                                            client: push_client.clone(),
                                         },
                                     );
                                 }
@@ -129,8 +132,39 @@ pub fn spawn_push_manager(inner: Arc<Inner>) -> mpsc::Sender<Event> {
         }
 
         loop {
+            if let Some(verify_due) = next_verify {
+                let current_instant = Instant::now();
+                if verify_due <= current_instant {
+                    let server = inner.build_server();
+                    let push_timeout = server.core.jmap.push_timeout;
+                    let current_time = now();
+                    next_verify = None;
+                    pending_verify.retain(|account_id, (verify_due, subscription)| {
+                        if *verify_due > current_instant {
+                            next_verify =
+                                Some(next_verify.map_or(*verify_due, |next| next.min(*verify_due)));
+                            true
+                        } else {
+                            if subscription.expires > current_time {
+                                last_verify.insert(*account_id, (current_instant, subscription.id));
+                                send_verification(
+                                    &push_client,
+                                    subscription.clone(),
+                                    &server,
+                                    push_timeout,
+                                );
+                            }
+                            false
+                        }
+                    });
+                }
+            }
+
             // Wait for the next event or timeout
-            let event_or_timeout = tokio::time::timeout(retry_timeout, push_rx.recv()).await;
+            let wait_timeout = next_verify.map_or(retry_timeout, |verify_due| {
+                retry_timeout.min(verify_due.saturating_duration_since(Instant::now()))
+            });
+            let event_or_timeout = tokio::time::timeout(wait_timeout, push_rx.recv()).await;
 
             // Load settings
             let server = inner.build_server();
@@ -193,10 +227,9 @@ pub fn spawn_push_manager(inner: Arc<Inner>) -> mpsc::Sender<Event> {
                                             num_attempts: 0,
                                             last_request: Instant::now()
                                                 - (push_throttle + Duration::from_millis(1)),
-                                            notifications: Vec::new(),
+                                            pending: PushBatch::default(),
                                             server: subscription.clone(),
                                             in_flight: false,
-                                            client: push_client.clone(),
                                         });
                                     }
                                 }
@@ -213,44 +246,31 @@ pub fn spawn_push_manager(inner: Arc<Inner>) -> mpsc::Sender<Event> {
 
                             #[cfg(feature = "test_mode")]
                             if subscription.url.contains("skip_checks") {
-                                last_verify.insert(
-                                    account_id,
-                                    current_time - (push_verify_timeout + Duration::from_millis(1)),
-                                );
+                                last_verify.remove(&account_id);
                             }
 
-                            if last_verify
+                            match last_verify
                                 .get(&account_id)
-                                .map(|last_verify| {
-                                    current_time - *last_verify > push_verify_timeout
+                                .map(|(verified_at, verified_id)| {
+                                    (*verified_at + push_verify_timeout, *verified_id)
                                 })
-                                .unwrap_or(true)
+                                .filter(|(verify_due, _)| *verify_due >= current_time)
                             {
-                                let core = server.core.clone();
-                                let push_client = push_client.clone();
-                                tokio::spawn(async move {
-                                    http_request(
-                                        &push_client,
-                                        &subscription,
-                                        format!(
-                                            concat!(
-                                                "{{\"@type\":\"PushVerification\",",
-                                                "\"pushSubscriptionId\":\"{}\",",
-                                                "\"verificationCode\":\"{}\"}}"
-                                            ),
-                                            Id::from(subscription.id),
-                                            subscription.verification_code
-                                        )
-                                        .into_bytes(),
-                                        push_timeout,
-                                        core.jmap.vapid.as_ref(),
-                                        Urgency::Normal,
-                                    )
-                                    .await;
+                                None => {
+                                    last_verify.retain(|_, (verified_at, _)| {
+                                        current_time.duration_since(*verified_at)
+                                            <= push_verify_timeout
                                     });
-
-                                last_verify.insert(account_id, current_time);
-                            } else {
+                                    last_verify.insert(account_id, (current_time, subscription.id));
+                                    pending_verify.remove(&account_id);
+                                    send_verification(
+                                        &push_client,
+                                        subscription,
+                                        &server,
+                                        push_timeout,
+                                    );
+                                }
+                                Some((_, verified_id)) if verified_id == subscription.id => {
                                     trc::event!(
                                         PushSubscription(PushSubscriptionEvent::Error),
                                         Details = "Failed to verify push subscription",
@@ -258,7 +278,24 @@ pub fn spawn_push_manager(inner: Arc<Inner>) -> mpsc::Sender<Event> {
                                         AccountId = account_id,
                                         Reason = "Too many requests"
                                     );
+                                    pending_verify.remove(&account_id);
                                 }
+                                Some((verify_due, _)) => {
+                                    trc::event!(
+                                        PushSubscription(PushSubscriptionEvent::Error),
+                                        Details = "Push subscription verification deferred",
+                                        Url = subscription.url.clone(),
+                                        AccountId = account_id,
+                                        Reason = "Too many requests"
+                                    );
+                                    next_verify = Some(
+                                        next_verify.map_or(verify_due, |next| next.min(verify_due)),
+                                    );
+                                    pending_verify.insert(account_id, (verify_due, subscription));
+                                }
+                            }
+                        } else {
+                            pending_verify.remove(&account_id);
                         }
 
                         // Update subscriptions
@@ -342,7 +379,7 @@ pub fn spawn_push_manager(inner: Arc<Inner>) -> mpsc::Sender<Event> {
                                                 );
                                             }
 
-                                            subscription.notifications.push(notification);
+                                            subscription.pending.push(notification);
                                             let last_request = subscription.last_request.elapsed();
 
                                             if !subscription.in_flight
@@ -354,6 +391,7 @@ pub fn spawn_push_manager(inner: Arc<Inner>) -> mpsc::Sender<Event> {
                                             {
                                                 subscription.send(
                                                     *id,
+                                                    &push_client,
                                                     push_tx.clone(),
                                                     push_timeout,
                                                     server.clone(),
@@ -402,19 +440,25 @@ pub fn spawn_push_manager(inner: Arc<Inner>) -> mpsc::Sender<Event> {
                     Event::Reset => {
                         push_servers.clear();
                         account_push_ids.clear();
+                        pending_verify.clear();
+                        next_verify = None;
                     }
                     Event::DeliverySuccess { id } => {
                         if let Some(subscription) = push_servers.get_mut(&id) {
                             subscription.num_attempts = 0;
                             subscription.in_flight = false;
+                            if subscription.pending.is_empty() {
                                 retry_ids.remove(&id);
+                            } else {
+                                retry_ids.insert(id);
                             }
                         }
-                    Event::DeliveryFailure { id, notifications } => {
+                    }
+                    Event::DeliveryFailure { id, failed } => {
                         if let Some(subscription) = push_servers.get_mut(&id) {
                             subscription.last_request = Instant::now();
                             subscription.num_attempts += 1;
-                            subscription.notifications.extend(notifications);
+                            subscription.pending.merge_failed(failed);
                             subscription.in_flight = false;
                             retry_ids.insert(id);
                         }
@@ -430,21 +474,23 @@ pub fn spawn_push_manager(inner: Arc<Inner>) -> mpsc::Sender<Event> {
                 let last_retry_elapsed = last_retry.elapsed();
 
                 if last_retry_elapsed >= push_retry_interval {
-                    let mut remove_ids = Vec::with_capacity(retry_ids.len());
-
-                    for retry_id in &retry_ids {
-                        if let Some(subscription) = push_servers.get_mut(retry_id) {
+                    retry_ids.retain(|retry_id| {
+                        let Some(subscription) = push_servers.get_mut(retry_id) else {
+                            return false;
+                        };
                         let last_request = subscription.last_request.elapsed();
-
-                            if !subscription.in_flight
-                                && ((subscription.num_attempts == 0
-                                    && last_request >= push_throttle)
+                        let is_due = !subscription.in_flight
+                            && ((subscription.num_attempts == 0 && last_request >= push_throttle)
                                 || (subscription.num_attempts > 0
-                                        && last_request >= push_attempt_interval))
-                            {
+                                    && last_request >= push_attempt_interval));
+                        if !is_due {
+                            return true;
+                        }
+
                         if subscription.num_attempts < push_attempts_max {
                             subscription.send(
                                 *retry_id,
+                                &push_client,
                                 push_tx.clone(),
                                 push_timeout,
                                 server.clone(),
@@ -457,25 +503,17 @@ pub fn spawn_push_manager(inner: Arc<Inner>) -> mpsc::Sender<Event> {
                                 Reason = "Too many failed attempts"
                             );
 
-                                    subscription.notifications.clear();
+                            subscription.pending.clear();
                             subscription.num_attempts = 0;
                         }
-                                remove_ids.push(*retry_id);
-                            }
-                        } else {
-                            remove_ids.push(*retry_id);
-                        }
-                    }
+                        false
+                    });
 
-                    if remove_ids.len() < retry_ids.len() {
-                        for remove_id in remove_ids {
-                            retry_ids.remove(&remove_id);
-                        }
+                    if retry_ids.is_empty() {
+                        LONG_1Y_SLUMBER
+                    } else {
                         last_retry = Instant::now();
                         push_retry_interval
-                    } else {
-                        retry_ids.clear();
-                        LONG_1Y_SLUMBER
                     }
                 } else {
                     push_retry_interval - last_retry_elapsed
@@ -489,6 +527,36 @@ pub fn spawn_push_manager(inner: Arc<Inner>) -> mpsc::Sender<Event> {
     push_tx_
 }
 
+fn send_verification(
+    push_client: &Client,
+    subscription: Arc<PushSubscription>,
+    server: &Server,
+    push_timeout: Duration,
+) {
+    let core = server.core.clone();
+    let push_client = push_client.clone();
+    tokio::spawn(async move {
+        http_request(
+            &push_client,
+            &subscription,
+            format!(
+                concat!(
+                    "{{\"@type\":\"PushVerification\",",
+                    "\"pushSubscriptionId\":\"{}\",",
+                    "\"verificationCode\":\"{}\"}}"
+                ),
+                Id::from(subscription.id),
+                subscription.verification_code
+            )
+            .into_bytes(),
+            push_timeout,
+            core.jmap.vapid.as_ref(),
+            Urgency::Normal,
+        )
+        .await;
+    });
+}
+
 async fn load_push_subscriptions(
     server: &Server,
     account_id: u32,

crates/services/src/task_manager/spam_classifier.rs

@@ -13,17 +13,18 @@ use common::{
 use registry::{
     schema::{
         enums::TaskSpamFilterMaintenanceType,
-        prelude::ObjectType,
+        prelude::{Object, ObjectInner, ObjectType},
         structs::{
             HttpLookup, MemoryLookupKey, SpamDnsblServer, SpamFileExtension, SpamRule, SpamTag,
             TaskSpamFilterMaintenance,
         },
     },
-    types::EnumImpl,
+    types::{EnumImpl, ObjectImpl},
 };
 use spam_filter::modules::classifier::SpamClassifier;
 use std::time::{Duration, Instant};
 use store::{
+    RegistryStore,
     ahash::AHashMap,
     registry::write::{RegistryWrite, RegistryWriteResult},
 };
@@ -97,13 +98,67 @@ struct Rules {
     file_exts: Vec<SpamFileExtension>,
 }
 
-#[derive(Default)]
+trait UpstreamObject: ObjectImpl + PartialEq + From<Object> + Into<ObjectInner> {
+    fn replacement_for(self, _local: &Self) -> Option<Self> {
+        Some(self)
+    }
+}
+
+impl UpstreamObject for SpamRule {
+    fn replacement_for(mut self, local: &Self) -> Option<Self> {
+        self.set_enable(local.enable());
+        Some(self)
+    }
+}
+
+impl UpstreamObject for SpamDnsblServer {
+    fn replacement_for(mut self, local: &Self) -> Option<Self> {
+        self.set_enable(local.enable());
+        Some(self)
+    }
+}
+
+impl UpstreamObject for HttpLookup {
+    fn replacement_for(mut self, local: &Self) -> Option<Self> {
+        self.enable = local.enable;
+        Some(self)
+    }
+}
+
+impl UpstreamObject for SpamTag {
+    fn replacement_for(self, _local: &Self) -> Option<Self> {
+        None
+    }
+}
+
+impl UpstreamObject for MemoryLookupKey {}
+
+impl UpstreamObject for SpamFileExtension {}
+
 struct RuleUpdateResult {
-    success: usize,
-    already_exists: usize,
+    object_type: ObjectType,
+    added: usize,
+    updated: usize,
+    unchanged: usize,
     failed: usize,
 }
 
+impl RuleUpdateResult {
+    fn new(object_type: ObjectType) -> Self {
+        RuleUpdateResult {
+            object_type,
+            added: 0,
+            updated: 0,
+            unchanged: 0,
+            failed: 0,
+        }
+    }
+
+    fn has_changes(&self) -> bool {
+        self.added + self.updated > 0
+    }
+}
+
 async fn update_spam_rules(server: &Server) -> trc::Result<TaskResult> {
     let started = Instant::now();
     let rules = match fetch_spam_rules(server).await {
@@ -118,170 +173,52 @@ async fn update_spam_rules(server: &Server) -> trc::Result<TaskResult> {
     };
 
     let registry = server.registry();
-    let mut stats: AHashMap<ObjectType, RuleUpdateResult> = AHashMap::new();
+    let settings = [
+        apply_upstream(registry, rules.rules).await?,
+        apply_upstream(registry, rules.dnsbls).await?,
+        apply_upstream(registry, rules.tags).await?,
+        apply_upstream(registry, rules.file_exts).await?,
+    ];
+    let lookups = [
+        apply_upstream(registry, rules.http_lookups).await?,
+        apply_upstream(registry, rules.key_lookups).await?,
+    ];
 
-    let mut reload_settings = false;
-    let mut reload_lookups = false;
-
-    for rule in rules.rules {
-        match registry.write(RegistryWrite::insert(&rule.into())).await? {
-            RegistryWriteResult::Success(_) => {
-                stats.entry(ObjectType::SpamRule).or_default().success += 1;
-                reload_settings = true;
-            }
-            RegistryWriteResult::PrimaryKeyConflict { .. } => {
-                stats
-                    .entry(ObjectType::SpamRule)
-                    .or_default()
-                    .already_exists += 1;
-            }
-            _ => {
-                stats.entry(ObjectType::SpamRule).or_default().failed += 1;
-            }
-        }
-    }
-
-    for dnsbl in rules.dnsbls {
-        match registry.write(RegistryWrite::insert(&dnsbl.into())).await? {
-            RegistryWriteResult::Success(_) => {
-                stats
-                    .entry(ObjectType::SpamDnsblServer)
-                    .or_default()
-                    .success += 1;
-                reload_settings = true;
-            }
-            RegistryWriteResult::PrimaryKeyConflict { .. } => {
-                stats
-                    .entry(ObjectType::SpamDnsblServer)
-                    .or_default()
-                    .already_exists += 1;
-            }
-            _ => {
-                stats.entry(ObjectType::SpamDnsblServer).or_default().failed += 1;
-            }
-        }
-    }
-
-    for tag in rules.tags {
-        match registry.write(RegistryWrite::insert(&tag.into())).await? {
-            RegistryWriteResult::Success(_) => {
-                stats.entry(ObjectType::SpamTag).or_default().success += 1;
-                reload_settings = true;
-            }
-            RegistryWriteResult::PrimaryKeyConflict { .. } => {
-                stats.entry(ObjectType::SpamTag).or_default().already_exists += 1;
-            }
-            _ => {
-                stats.entry(ObjectType::SpamTag).or_default().failed += 1;
-            }
-        }
-    }
-
-    for lookup in rules.http_lookups {
-        match registry
-            .write(RegistryWrite::insert(&lookup.into()))
-            .await?
+    let mut reload_errors = Vec::new();
+    for object in [
+        settings
+            .iter()
+            .any(RuleUpdateResult::has_changes)
+            .then_some(ObjectType::SpamRule),
+        lookups
+            .iter()
+            .any(RuleUpdateResult::has_changes)
+            .then_some(ObjectType::MemoryLookupKey),
+    ]
+    .into_iter()
+    .flatten()
     {
-            RegistryWriteResult::Success(_) => {
-                stats.entry(ObjectType::HttpLookup).or_default().success += 1;
-                reload_lookups = true;
-            }
-            RegistryWriteResult::PrimaryKeyConflict { .. } => {
-                stats
-                    .entry(ObjectType::HttpLookup)
-                    .or_default()
-                    .already_exists += 1;
-            }
-            _ => {
-                stats.entry(ObjectType::HttpLookup).or_default().failed += 1;
+        if let Err(reason) = reload_and_broadcast(server, object).await {
+            reload_errors.push(reason);
         }
     }
-    }
-
-    for key_lookup in rules.key_lookups {
-        match registry
-            .write(RegistryWrite::insert(&key_lookup.into()))
-            .await?
-        {
-            RegistryWriteResult::Success(_) => {
-                stats
-                    .entry(ObjectType::MemoryLookupKey)
-                    .or_default()
-                    .success += 1;
-                reload_lookups = true;
-            }
-            RegistryWriteResult::PrimaryKeyConflict { .. } => {
-                stats
-                    .entry(ObjectType::MemoryLookupKey)
-                    .or_default()
-                    .already_exists += 1;
-            }
-            _ => {
-                stats.entry(ObjectType::MemoryLookupKey).or_default().failed += 1;
-            }
-        }
-    }
-
-    for ext in rules.file_exts {
-        match registry.write(RegistryWrite::insert(&ext.into())).await? {
-            RegistryWriteResult::Success(_) => {
-                stats
-                    .entry(ObjectType::SpamFileExtension)
-                    .or_default()
-                    .success += 1;
-                reload_settings = true;
-            }
-            RegistryWriteResult::PrimaryKeyConflict { .. } => {
-                stats
-                    .entry(ObjectType::SpamFileExtension)
-                    .or_default()
-                    .already_exists += 1;
-            }
-            _ => {
-                stats
-                    .entry(ObjectType::SpamFileExtension)
-                    .or_default()
-                    .failed += 1;
-            }
-        }
-    }
-
-    if reload_settings {
-        if let Err(err) =
-            Box::pin(server.reload_registry(RegistryChange::Reload(ObjectType::SpamRule))).await
-        {
-            trc::error!(err.details("Failed to reload registry after updating spam rules"));
-        }
-        server
-            .cluster_broadcast(BroadcastEvent::RegistryChange(RegistryChange::Reload(
-                ObjectType::SpamRule,
-            )))
-            .await;
-    }
-
-    if reload_lookups {
-        if let Err(err) =
-            Box::pin(server.reload_registry(RegistryChange::Reload(ObjectType::MemoryLookupKey)))
-                .await
-        {
-            trc::error!(err.details("Failed to reload registry after updating spam rules"));
-        }
-        server
-            .cluster_broadcast(BroadcastEvent::RegistryChange(RegistryChange::Reload(
-                ObjectType::MemoryLookupKey,
-            )))
-            .await;
-    }
+    let failed: usize = settings
+        .iter()
+        .chain(&lookups)
+        .map(|result| result.failed)
+        .sum();
 
     trc::event!(
         Spam(SpamEvent::RulesUpdated),
-        Details = stats
+        Details = settings
             .into_iter()
-            .map(|(object_type, result)| {
+            .chain(lookups)
+            .map(|result| {
                 Value::Array(vec![
-                    Value::String(object_type.as_str().into()),
-                    Value::from(result.success),
-                    Value::from(result.already_exists),
+                    Value::String(result.object_type.as_str().into()),
+                    Value::from(result.added),
+                    Value::from(result.updated),
+                    Value::from(result.unchanged),
                     Value::from(result.failed),
                 ])
             })
@@ -289,7 +226,100 @@ async fn update_spam_rules(server: &Server) -> trc::Result<TaskResult> {
         Elapsed = started.elapsed(),
     );
 
+    if !reload_errors.is_empty() {
+        Ok(TaskResult::permanent(format!(
+            "Spam rules were stored but not activated ({}); fix the logged errors and run Reload settings",
+            reload_errors.join("; ")
+        )))
+    } else if failed > 0 {
+        Ok(TaskResult::permanent(format!(
+            "{failed} spam filter objects failed to import or update"
+        )))
+    } else {
         Ok(TaskResult::Success(vec![]))
+    }
+}
+
+async fn reload_and_broadcast(server: &Server, object: ObjectType) -> Result<(), String> {
+    match Box::pin(server.reload_registry(RegistryChange::Reload(object))).await {
+        Ok(result) => {
+            result.log();
+            if result.has_errors() {
+                return Err(format!("{} configuration errors", result.errors.len()));
+            }
+            server
+                .cluster_broadcast(BroadcastEvent::RegistryChange(RegistryChange::Reload(
+                    object,
+                )))
+                .await;
+            Ok(())
+        }
+        Err(err) => {
+            let reason = err.to_string();
+            trc::error!(err.details("Failed to reload registry after updating spam rules"));
+            Err(reason)
+        }
+    }
+}
+
+async fn apply_upstream<T: UpstreamObject>(
+    registry: &RegistryStore,
+    objects: Vec<T>,
+) -> trc::Result<RuleUpdateResult> {
+    let mut result = RuleUpdateResult::new(T::OBJECT);
+
+    for upstream in objects {
+        let upstream = Object::from(upstream);
+        let existing_id = match registry.write(RegistryWrite::insert(&upstream)).await? {
+            RegistryWriteResult::Success(_) => {
+                result.added += 1;
+                continue;
+            }
+            RegistryWriteResult::PrimaryKeyConflict { existing_id, .. }
+                if existing_id.object() == T::OBJECT =>
+            {
+                existing_id
+            }
+            RegistryWriteResult::PrimaryKeyConflict { .. } => {
+                result.unchanged += 1;
+                continue;
+            }
+            _ => {
+                result.failed += 1;
+                continue;
+            }
+        };
+
+        let Some(local) = registry.get(existing_id).await? else {
+            result.failed += 1;
+            continue;
+        };
+        let revision = local.revision;
+        let local = T::from(local);
+        let Some(replacement) = T::from(upstream)
+            .replacement_for(&local)
+            .filter(|replacement| replacement != &local)
+        else {
+            result.unchanged += 1;
+            continue;
+        };
+
+        let replacement = Object::from(replacement);
+        let local = Object::with_revision(local.into(), revision);
+        match registry
+            .write(RegistryWrite::update(
+                existing_id.id(),
+                &replacement,
+                &local,
+            ))
+            .await?
+        {
+            RegistryWriteResult::Success(_) => result.updated += 1,
+            _ => result.failed += 1,
+        }
+    }
+
+    Ok(result)
 }
 
 async fn fetch_spam_rules(server: &Server) -> Result<Rules, RuleUpdateError> {

crates/smtp/Cargo.toml

@@ -7,7 +7,7 @@ homepage = "https://stalw.art/smtp"
 keywords = ["smtp", "email", "mail", "server"]
 categories = ["email"]
 license = "AGPL-3.0-only OR LicenseRef-SEL"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/smtp/src/inbound/data.rs

@@ -694,7 +694,12 @@ impl<T: SessionStream> Session<T> {
                         .map(|a| a.as_str())
                         .unwrap_or_default(),
                 )
-                .with_message(parsed_message);
+                .with_message(
+                    edited_message
+                        .as_deref()
+                        .and_then(|message| MessageParser::new().parse(message))
+                        .unwrap_or(parsed_message),
+                );
 
             let modifications = match self.run_script(script_id, script.clone(), params).await {
                 ScriptResult::Accept { modifications } => modifications,

crates/smtp/src/inbound/hooks/message.rs

@@ -132,7 +132,7 @@ impl<T: SessionStream> Session<T> {
                                 name: "X-Quarantine".into(),
                                 value: "true".into(),
                             });
-                            FilterResponse::accept()
+                            continue;
                         }
                     };
 

crates/smtp/src/lib.rs

@@ -38,12 +38,19 @@ impl StartQueueManager for BootManager {
 impl SpawnQueueManager for IpcReceivers {
     fn spawn_queue_manager(&mut self, inner: Arc<Inner>) {
         let core = inner.shared_core.load();
-        if !core.storage.registry.is_recovery_mode() && core.network.roles.outbound_mta {
+        if core.storage.registry.is_recovery_mode() {
+            return;
+        }
+
         // Spawn queue manager
-            self.queue_rx.take().unwrap().spawn(inner.clone());
+        let queue_rx = self.queue_rx.take().unwrap();
+        if core.network.roles.outbound_mta {
+            queue_rx.spawn(inner.clone());
+        } else {
+            queue_rx.discard();
+        }
 
         // Spawn report manager
         self.report_rx.take().unwrap().spawn(inner);
     }
-    }
 }

crates/smtp/src/outbound/delivery.rs

@@ -531,11 +531,23 @@ impl QueuedMessage {
             };
 
             // Obtain remote hosts list
+            let mx_unvalidated = mx_config.is_some() && !tls_strategy.try_dane();
             let mx_list;
             if let Some(mx_config) = mx_config {
                 // Lookup MX
                 let time = Instant::now();
-                mx_list = match server.mx_lookup(domain).await {
+                let mx_lookup = if mx_unvalidated {
+                    server
+                        .core
+                        .smtp
+                        .resolvers
+                        .dns
+                        .mx_lookup(domain, Some(&server.inner.cache.dns_mx))
+                        .await
+                } else {
+                    server.mx_lookup(domain).await
+                };
+                mx_list = match mx_lookup {
                     Ok(mx) => mx,
                     Err(mail_auth::Error::Dns(mail_auth::DnsError::RecordNotFound(_))) => {
                         trc::event!(
@@ -674,6 +686,33 @@ impl QueuedMessage {
                     message.span_id,
                 );
 
+                let validated_host;
+                let remote_host = if mx_unvalidated && tls_strategy.try_dane() {
+                    let time = Instant::now();
+                    let dnssec_status = match server.mx_lookup(domain).await {
+                        Ok(mx) => mx.dnssec_status,
+                        Err(mail_auth::Error::Dns(mail_auth::DnsError::RecordNotFound(_))) => {
+                            DnssecStatus::Indeterminate
+                        }
+                        Err(err) => {
+                            trc::event!(
+                                Delivery(DeliveryEvent::MxLookupFailed),
+                                SpanId = message.span_id,
+                                Domain = domain.to_string(),
+                                CausedBy = trc::Error::from(err.clone()),
+                                Elapsed = time.elapsed(),
+                            );
+
+                            last_status = Status::from_mail_auth_error(domain, err);
+                            continue 'next_host;
+                        }
+                    };
+                    validated_host = remote_host.with_dnssec_status(dnssec_status);
+                    &validated_host
+                } else {
+                    remote_host
+                };
+
                 // Obtain source and remote IPs
                 let time = Instant::now();
                 let validate_addresses = server.core.smtp.resolvers.dnssec_available
@@ -722,15 +761,8 @@ impl QueuedMessage {
                     let time = Instant::now();
                     let strict = tls_strategy.is_dane_required();
 
-                    let (dnssec_status, dnssec_entity) = match remote_host.dnssec_status() {
-                        DnssecStatus::Secure => match addresses_dnssec_status {
-                            status @ (DnssecStatus::Insecure | DnssecStatus::Bogus) => {
-                                (status, "A/AAAA")
-                            }
-                            _ => (DnssecStatus::Secure, "MX"),
-                        },
-                        status => (status, "MX"),
-                    };
+                    let (dnssec_status, dnssec_entity) =
+                        remote_host.dane_status(addresses_dnssec_status);
 
                     match dnssec_status {
                         DnssecStatus::Secure => {

crates/smtp/src/outbound/mod.rs

@@ -350,12 +350,39 @@ impl NextHop<'_> {
         }
     }
 
-    fn dnssec_status(&self) -> DnssecStatus {
+    pub fn dnssec_status(&self) -> DnssecStatus {
         match self {
             NextHop::MX { dnssec_status, .. } => *dnssec_status,
             NextHop::Relay(_) => DnssecStatus::Indeterminate,
         }
     }
+
+    fn with_dnssec_status(&self, dnssec_status: DnssecStatus) -> Self {
+        match self {
+            NextHop::MX {
+                is_implicit,
+                host,
+                config,
+                ..
+            } => NextHop::MX {
+                is_implicit: *is_implicit,
+                host,
+                config,
+                dnssec_status,
+            },
+            NextHop::Relay(relay) => NextHop::Relay(relay),
+        }
+    }
+
+    pub fn dane_status(&self, addresses: DnssecStatus) -> (DnssecStatus, &'static str) {
+        match self.dnssec_status() {
+            DnssecStatus::Secure => match addresses {
+                status @ (DnssecStatus::Insecure | DnssecStatus::Bogus) => (status, "A/AAAA"),
+                _ => (DnssecStatus::Secure, "MX"),
+            },
+            status => (status, "MX"),
+        }
+    }
 }
 
 impl DeliveryResult {

crates/smtp/src/queue/manager.rs

@@ -62,6 +62,10 @@ impl SpawnQueue for mpsc::Receiver<QueueEvent> {
             Queue::new(core, self).start().await;
         });
     }
+
+    fn discard(mut self) {
+        tokio::spawn(async move { while self.recv().await.is_some() {} });
+    }
 }
 
 const BACK_PRESSURE_WARN_INTERVAL: Duration = Duration::from_secs(60);
@@ -480,6 +484,7 @@ impl Recipient {
 
 pub trait SpawnQueue {
     fn spawn(self, core: Arc<Inner>);
+    fn discard(self);
 }
 
 impl QueueStats {

crates/spam-filter/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "spam-filter"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/spam-filter/src/analysis/url.rs

@@ -203,7 +203,7 @@ impl SpamFilterAnalyzeUrl for Server {
 
                     if !ctx.result.has_tag("URL_REDIRECTOR_NESTED") {
                         let mut redirect_count = 1;
-                        let mut url_redirect = Cow::Borrowed(url.element.url.as_str());
+                        let mut url_redirect = url.element.request_url();
 
                         while redirect_count <= 3 {
                             match http_get_header(
@@ -224,7 +224,8 @@ impl SpamFilterAnalyzeUrl for Server {
                                         )
                                         .await
                                         {
-                                            url_redirect = Cow::Owned(location.url);
+                                            url_redirect =
+                                                Cow::Owned(location.request_url().into_owned());
                                             redirect_count += 1;
                                             continue;
                                         } else {
@@ -487,6 +488,15 @@ impl<'x> UrlParts<'x> {
                 .is_some_and(|url| url.host.fqdn.starts_with("www."))
     }
 
+    pub fn request_url(&self) -> Cow<'_, str> {
+        let url = self.url_original.trim();
+        if self.has_scheme {
+            Cow::Borrowed(url)
+        } else {
+            Cow::Owned([HTTPS_SCHEME, url].concat())
+        }
+    }
+
     fn parse(url: &str) -> Option<UrlParsed> {
         url.parse::<Uri>().ok().and_then(|parts| {
             parts
@@ -505,3 +515,19 @@ impl<'x> UrlParts<'x> {
         }
     }
 }
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn request_url_keeps_case() {
+        let url = UrlParts::new(" https://Bit.ly/3AbCdEf ");
+        assert_eq!(url.url, "https://bit.ly/3abcdef");
+        assert_eq!(url.request_url(), "https://Bit.ly/3AbCdEf");
+
+        let url = UrlParts::no_scheme("Bit.ly/3AbCdEf");
+        assert_eq!(url.url, "https://bit.ly/3abcdef");
+        assert_eq!(url.request_url(), "https://Bit.ly/3AbCdEf");
+    }
+}

crates/spam-filter/src/modules/classifier.rs

@@ -257,9 +257,7 @@ impl SpamClassifier for Server {
                         remove_entries = true;
                     }
 
-                    if trainer.last_id == 0 {
-                        trainer.last_id = id;
-                    }
+                    trainer.last_id = trainer.last_id.max(id);
 
                     Ok(true)
                 },

crates/store/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "store"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/store/src/backend/http/lookup.rs

@@ -4,21 +4,19 @@
  * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
  */
 
+use super::{HttpStore, HttpStoreConfig};
+use crate::{Value, backend::http::HttpStoreFormat, write::now};
+use ahash::AHashMap;
+use compact_str::ToCompactString;
+use rand::seq::IndexedRandom;
 use std::{
+    borrow::Cow,
     io::{BufRead, BufReader},
     sync::{Arc, atomic::Ordering},
     time::Instant,
 };
-
-use ahash::AHashMap;
-use compact_str::ToCompactString;
-use rand::seq::IndexedRandom;
 use utils::HttpLimitResponse;
 
-use crate::{Value, backend::http::HttpStoreFormat, write::now};
-
-use super::HttpStore;
-
 const BROWSER_USER_AGENTS: [&str; 5] = [
     "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
     "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Edge/120.0.0.0 Safari/537.36",
@@ -36,7 +34,7 @@ pub(crate) trait HttpStoreGet {
 impl HttpStoreGet for Arc<HttpStore> {
     fn get(&self, key: &str) -> Option<Value<'static>> {
         self.refresh();
-        self.entries.load().get(key).cloned()
+        self.entries.load().get(lookup_key(key).as_ref()).cloned()
     }
 
     fn contains(&self, key: &str) -> bool {
@@ -59,7 +57,7 @@ impl HttpStoreGet for Arc<HttpStore> {
         }
 
         self.refresh();
-        self.entries.load().contains_key(key)
+        self.entries.load().contains_key(lookup_key(key).as_ref())
     }
 
     fn refresh(&self) {
@@ -142,9 +140,10 @@ impl HttpStore {
             Box::new(&bytes[..])
         };
 
-        let mut entries = AHashMap::new();
-        for (pos, line) in BufReader::new(reader).lines().enumerate() {
-            let line_ = line.map_err(|err| {
+        let entries = self
+            .config
+            .parse_entries(BufReader::new(reader))
+            .map_err(|err| {
                 trc::StoreEvent::HttpStoreError
                     .into_err()
                     .reason(err)
@@ -153,11 +152,31 @@ impl HttpStore {
                     .details("Failed to read line")
             })?;
 
-            match &self.config.format {
+        trc::event!(
+            Store(trc::StoreEvent::HttpStoreFetch),
+            Url = self.config.url.to_compact_string(),
+            Total = entries.len(),
+            Elapsed = time.elapsed(),
+        );
+
+        Ok(entries)
+    }
+}
+
+impl HttpStoreConfig {
+    fn parse_entries(
+        &self,
+        reader: impl BufRead,
+    ) -> std::io::Result<AHashMap<String, Value<'static>>> {
+        let mut entries = AHashMap::new();
+        for (pos, line) in reader.lines().enumerate() {
+            let line_ = line?;
+
+            match &self.format {
                 HttpStoreFormat::List => {
                     let line = line_.trim();
                     if !line.is_empty() {
-                        entries.insert(line.to_string(), Value::Integer(1));
+                        entries.insert(lookup_key(line).into_owned(), Value::Integer(1));
                     }
                 }
                 HttpStoreFormat::Csv {
@@ -188,12 +207,12 @@ impl HttpStore {
                                     }
                                 } else if col_num == *index_key {
                                     entry_key.push(ch);
-                                    if entry_key.len() > self.config.max_entry_size {
+                                    if entry_key.len() > self.max_entry_size {
                                         break;
                                     }
                                 } else if index_value.is_some_and(|v| col_num == v) {
                                     entry_value.push(ch);
-                                    if entry_value.len() > self.config.max_entry_size {
+                                    if entry_value.len() > self.max_entry_size {
                                         break;
                                     }
                                 }
@@ -209,24 +228,122 @@ impl HttpStore {
                         } else {
                             Value::Integer(1)
                         };
+                        let entry_key = match lookup_key(&entry_key) {
+                            Cow::Owned(key) => key,
+                            Cow::Borrowed(_) => entry_key,
+                        };
                         entries.insert(entry_key, entry_value);
                     }
                 }
                 _ => (),
             }
 
-            if entries.len() == self.config.max_entries {
+            if entries.len() == self.max_entries {
                 break;
             }
         }
 
-        trc::event!(
-            Store(trc::StoreEvent::HttpStoreFetch),
-            Url = self.config.url.to_compact_string(),
-            Total = entries.len(),
-            Elapsed = time.elapsed(),
-        );
-
         Ok(entries)
     }
 }
+
+fn lookup_key(key: &str) -> Cow<'_, str> {
+    if key.bytes().any(|b| !b.is_ascii() || b.is_ascii_uppercase()) {
+        Cow::Owned(key.to_lowercase())
+    } else {
+        Cow::Borrowed(key)
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use arc_swap::ArcSwap;
+    use reqwest::Client;
+    use std::{
+        sync::atomic::{AtomicBool, AtomicU64},
+        time::Duration,
+    };
+
+    fn http_store(format: HttpStoreFormat, feed: &str) -> Arc<HttpStore> {
+        let config = HttpStoreConfig {
+            id: "test".into(),
+            url: "https://lists.example.org/feed".into(),
+            retry: 0,
+            refresh: 0,
+            timeout: Duration::from_secs(1),
+            gzipped: false,
+            max_size: 1024 * 1024,
+            max_entries: 100,
+            max_entry_size: 512,
+            format,
+        };
+        let entries = config
+            .parse_entries(feed.as_bytes())
+            .expect("feed is readable");
+
+        Arc::new(HttpStore {
+            entries: ArcSwap::from_pointee(entries),
+            expires: AtomicU64::new(u64::MAX),
+            in_flight: AtomicBool::new(false),
+            config,
+            client: Client::new(),
+        })
+    }
+
+    #[test]
+    fn list_keys_ignore_case() {
+        let store = http_store(
+            HttpStoreFormat::List,
+            "https://phish.example.org/Account/Verify?Token=AbC123\n\
+             https://PHISH.example.net/lower\n",
+        );
+
+        assert!(store.contains("https://phish.example.org/account/verify?token=abc123"));
+        assert!(store.contains("https://phish.example.org/Account/Verify?Token=AbC123"));
+        assert!(store.contains("https://phish.example.net/lower"));
+        assert!(store.contains("HTTPS://PHISH.EXAMPLE.NET/LOWER"));
+        assert!(!store.contains("https://phish.example.org/account/verify"));
+    }
+
+    #[test]
+    fn csv_keys_ignore_case() {
+        let store = http_store(
+            HttpStoreFormat::Csv {
+                index_key: 1,
+                index_value: None,
+                separator: ',',
+                skip_first: true,
+            },
+            "phish_id,url,phish_detail_url\n\
+             1,\"https://phish.example.org/Login.PHP?Id=Xy\",https://phishtank.example/1\n",
+        );
+
+        assert!(store.contains("https://phish.example.org/login.php?id=xy"));
+        assert!(store.contains("https://phish.example.org/Login.PHP?Id=Xy"));
+        assert!(!store.contains("phish_id"));
+        assert!(!store.contains("url"));
+    }
+
+    #[test]
+    fn csv_values_keep_case() {
+        let store = http_store(
+            HttpStoreFormat::Csv {
+                index_key: 0,
+                index_value: Some(1),
+                separator: ',',
+                skip_first: false,
+            },
+            "Example.ORG,Some Value\n",
+        );
+
+        assert_eq!(
+            store.get("example.org"),
+            Some(Value::Text("Some Value".into()))
+        );
+        assert_eq!(
+            store.get("EXAMPLE.org"),
+            Some(Value::Text("Some Value".into()))
+        );
+    }
+}

crates/store/src/backend/mysql/mod.rs

@@ -30,7 +30,10 @@ fn into_error(err: impl Display) -> trc::Error {
     trc::StoreEvent::MysqlError.reason(err)
 }
 
+const ER_UNKNOWN_ERROR: u16 = 1105;
+const ER_TRANS_CACHE_FULL: u16 = 1197;
 const ER_LOCK_WAIT_TIMEOUT: u16 = 1205;
+const ER_LOCK_TABLE_FULL: u16 = 1206;
 const ER_STATEMENT_TIMEOUT: u16 = 1969;
 const ER_QUERY_TIMEOUT: u16 = 3024;
 
@@ -47,6 +50,17 @@ pub(crate) fn is_timeout_error(err: &mysql_async::Error) -> bool {
     )
 }
 
+#[inline(always)]
+pub(crate) fn is_chunk_too_large_error(err: &mysql_async::Error) -> bool {
+    is_timeout_error(err)
+        || matches!(err, mysql_async::Error::Server(err)
+            if matches!(
+                err.code,
+                ER_UNKNOWN_ERROR | ER_TRANS_CACHE_FULL | ER_LOCK_TABLE_FULL
+            )
+        )
+}
+
 impl SearchIndex {
     pub fn mysql_table(&self) -> &'static str {
         match self {

crates/store/src/backend/mysql/search.rs

@@ -9,7 +9,7 @@ use crate::{
         MAX_TOKEN_LENGTH,
         mysql::{
             DELETE_CHUNK_SIZE, MIN_DELETE_CHUNK_SIZE, MysqlSearchField, MysqlStore, into_error,
-            is_timeout_error,
+            is_chunk_too_large_error,
         },
     },
     search::{
@@ -109,14 +109,6 @@ impl MysqlStore {
         let params = build_filter(&mut query, &filter.filters);
 
         let mut conn = self.conn_pool.get_conn().await.map_err(into_error)?;
-        let s = conn.prep(&query).await.map_err(into_error)?;
-
-        match conn.exec_drop(s, params.clone()).await {
-            Ok(_) => return Ok(conn.affected_rows()),
-            Err(err) if is_timeout_error(&err) => (),
-            Err(err) => return Err(into_error(err)),
-        }
-
         let mut chunk_size = DELETE_CHUNK_SIZE;
         let mut deleted = 0;
 
@@ -130,12 +122,14 @@ impl MysqlStore {
                 match conn.exec_drop(&s, params.clone()).await {
                     Ok(_) => {
                         let affected = conn.affected_rows();
-                        if affected == 0 {
+                        deleted += affected;
+                        if affected < chunk_size as u64 {
                             return Ok(deleted);
                         }
-                        deleted += affected;
                     }
-                    Err(err) if is_timeout_error(&err) && chunk_size > MIN_DELETE_CHUNK_SIZE => {
+                    Err(err)
+                        if is_chunk_too_large_error(&err) && chunk_size > MIN_DELETE_CHUNK_SIZE =>
+                    {
                         chunk_size = (chunk_size / 2).max(MIN_DELETE_CHUNK_SIZE);
                         break;
                     }

crates/store/src/backend/mysql/write.rs

@@ -4,7 +4,9 @@
  * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
  */
 
-use super::{DELETE_CHUNK_SIZE, MIN_DELETE_CHUNK_SIZE, MysqlStore, into_error, is_timeout_error};
+use super::{
+    DELETE_CHUNK_SIZE, MIN_DELETE_CHUNK_SIZE, MysqlStore, into_error, is_chunk_too_large_error,
+};
 use crate::{
     IndexKey, Key, LogKey, SUBSPACE_COUNTER, SUBSPACE_IN_MEMORY_COUNTER, SUBSPACE_QUOTA,
     SUBSPACE_REGISTRY_IDX,
@@ -400,13 +402,6 @@ impl MysqlStore {
             .prep(format!("DELETE FROM {table} WHERE k >= ? AND k < ?"))
             .await
             .map_err(into_error)?;
-
-        match conn.exec_drop(&delete, (&from, &to)).await {
-            Ok(_) => return Ok(()),
-            Err(err) if is_timeout_error(&err) => (),
-            Err(err) => return Err(into_error(err)),
-        }
-
         let mut chunk_size = DELETE_CHUNK_SIZE;
 
         loop {
@@ -423,7 +418,9 @@ impl MysqlStore {
                     .await
                 {
                     Ok(next) => next,
-                    Err(err) if is_timeout_error(&err) && chunk_size > MIN_DELETE_CHUNK_SIZE => {
+                    Err(err)
+                        if is_chunk_too_large_error(&err) && chunk_size > MIN_DELETE_CHUNK_SIZE =>
+                    {
                         chunk_size = (chunk_size / 2).max(MIN_DELETE_CHUNK_SIZE);
                         break;
                     }
@@ -435,7 +432,9 @@ impl MysqlStore {
                     .await
                 {
                     Ok(_) => (),
-                    Err(err) if is_timeout_error(&err) && chunk_size > MIN_DELETE_CHUNK_SIZE => {
+                    Err(err)
+                        if is_chunk_too_large_error(&err) && chunk_size > MIN_DELETE_CHUNK_SIZE =>
+                    {
                         chunk_size = (chunk_size / 2).max(MIN_DELETE_CHUNK_SIZE);
                         break;
                     }
@@ -459,7 +458,7 @@ async fn purge_table(conn: &mut Conn, table: char) -> trc::Result<()> {
 
     match conn.exec_drop(&s, ()).await {
         Ok(_) => return Ok(()),
-        Err(err) if is_timeout_error(&err) => (),
+        Err(err) if is_chunk_too_large_error(&err) => (),
         Err(err) => return Err(into_error(err)),
     }
 
@@ -487,7 +486,9 @@ async fn purge_table(conn: &mut Conn, table: char) -> trc::Result<()> {
         loop {
             let next = match conn.exec_first::<Vec<u8>, _, _>(&boundary, (&from,)).await {
                 Ok(next) => next,
-                Err(err) if is_timeout_error(&err) && chunk_size > MIN_DELETE_CHUNK_SIZE => {
+                Err(err)
+                    if is_chunk_too_large_error(&err) && chunk_size > MIN_DELETE_CHUNK_SIZE =>
+                {
                     chunk_size = (chunk_size / 2).max(MIN_DELETE_CHUNK_SIZE);
                     break;
                 }
@@ -501,7 +502,9 @@ async fn purge_table(conn: &mut Conn, table: char) -> trc::Result<()> {
 
             match result {
                 Ok(_) => (),
-                Err(err) if is_timeout_error(&err) && chunk_size > MIN_DELETE_CHUNK_SIZE => {
+                Err(err)
+                    if is_chunk_too_large_error(&err) && chunk_size > MIN_DELETE_CHUNK_SIZE =>
+                {
                     chunk_size = (chunk_size / 2).max(MIN_DELETE_CHUNK_SIZE);
                     break;
                 }

crates/store/src/backend/redis/lookup.rs

@@ -7,16 +7,7 @@
 use super::{RedisPool, RedisStore, into_error};
 use crate::{Deserialize, write::now};
 use deadpool::managed::{Manager, Object, Pool};
-use redis::{AsyncCommands, RedisError, RedisResult, RetryMethod, Script};
-use std::sync::LazyLock;
-
-static INCR_EXPIRE: LazyLock<Script> = LazyLock::new(|| {
-    Script::new(
-        "redis.call('INCRBY', KEYS[1], ARGV[1])
-         redis.call('EXPIRE', KEYS[1], ARGV[2])
-         return redis.call('GET', KEYS[1])",
-    )
-});
+use redis::{AsyncCommands, RedisError, RedisResult, RetryMethod};
 
 impl RedisStore {
     pub async fn key_set(&self, key: &[u8], value: &[u8], expires: Option<u64>) -> trc::Result<()> {
@@ -46,19 +37,19 @@ impl RedisStore {
         match &self.pool {
             RedisPool::Single(pool) => {
                 with_conn(pool, async |conn| {
-                    Self::key_incr_(conn, key, value, expires).await
+                    self.key_incr_(conn, key, value, expires).await
                 })
                 .await
             }
             RedisPool::Cluster(pool) => {
                 with_conn(pool, async |conn| {
-                    Self::key_incr_(conn, key, value, expires).await
+                    self.key_incr_(conn, key, value, expires).await
                 })
                 .await
             }
             RedisPool::Sentinel(pool) => {
                 with_conn(pool, async |conn| {
-                    Self::key_incr_(conn, key, value, expires).await
+                    self.key_incr_(conn, key, value, expires).await
                 })
                 .await
             }
@@ -193,13 +184,14 @@ impl RedisStore {
     }
 
     async fn key_incr_(
+        &self,
         conn: &mut impl AsyncCommands,
         key: &[u8],
         value: i64,
         expires: Option<u64>,
     ) -> RedisResult<i64> {
         if let Some(expires) = expires {
-            INCR_EXPIRE
+            self.incr_expire
                 .key(key)
                 .arg(value)
                 .arg(expires as i64)

crates/store/src/backend/redis/mod.rs

@@ -10,7 +10,7 @@ use deadpool::{
     managed::{Manager, Pool},
 };
 use redis::{
-    Client, ConnectionAddr, IntoConnectionInfo, ProtocolVersion, TlsMode,
+    Client, ConnectionAddr, IntoConnectionInfo, ProtocolVersion, Script, TlsMode,
     cluster::{ClusterClient, ClusterClientBuilder},
     cluster_read_routing::RandomReplicaStrategy,
     sentinel::{SentinelClient, SentinelClientBuilder, SentinelServerType},
@@ -27,6 +27,7 @@ pub mod pool;
 #[derive(Debug)]
 pub struct RedisStore {
     pub pool: RedisPool,
+    incr_expire: Script,
 }
 
 pub struct RedisConnectionManager {
@@ -51,9 +52,20 @@ pub enum RedisPool {
 }
 
 impl RedisStore {
+    fn new(pool: RedisPool) -> Self {
+        RedisStore {
+            pool,
+            incr_expire: Script::new(
+                "redis.call('INCRBY', KEYS[1], ARGV[1])
+                 redis.call('EXPIRE', KEYS[1], ARGV[2])
+                 return redis.call('GET', KEYS[1])",
+            ),
+        }
+    }
+
     pub async fn open_single(config: structs::RedisStore) -> Result<InMemoryStore, String> {
-        Ok(InMemoryStore::Redis(Arc::new(RedisStore {
-            pool: RedisPool::Single(build_pool(
+        Ok(InMemoryStore::Redis(Arc::new(RedisStore::new(
+            RedisPool::Single(build_pool(
                 RedisConnectionManager {
                     client: Client::open(config.url)
                         .map_err(|err| format!("Failed to open Redis client: {err:?}"))?,
@@ -64,7 +76,7 @@ impl RedisStore {
                 config.pool_timeout_wait,
                 config.pool_timeout_recycle,
             )?),
-        })))
+        ))))
     }
 
     pub async fn open_cluster(config: structs::RedisClusterStore) -> Result<InMemoryStore, String> {
@@ -95,8 +107,8 @@ impl RedisStore {
             .build()
             .map_err(|err| format!("Failed to open Redis client: {err:?}"))?;
 
-        Ok(InMemoryStore::Redis(Arc::new(RedisStore {
-            pool: RedisPool::Cluster(build_pool(
+        Ok(InMemoryStore::Redis(Arc::new(RedisStore::new(
+            RedisPool::Cluster(build_pool(
                 RedisClusterConnectionManager {
                     client,
                     timeout: config.timeout.into_inner(),
@@ -106,7 +118,7 @@ impl RedisStore {
                 config.pool_timeout_wait,
                 config.pool_timeout_recycle,
             )?),
-        })))
+        ))))
     }
 
     pub async fn open_sentinel(
@@ -167,8 +179,8 @@ impl RedisStore {
             .build()
             .map_err(|err| format!("Failed to open Redis Sentinel client: {err:?}"))?;
 
-        Ok(InMemoryStore::Redis(Arc::new(RedisStore {
-            pool: RedisPool::Sentinel(build_pool(
+        Ok(InMemoryStore::Redis(Arc::new(RedisStore::new(
+            RedisPool::Sentinel(build_pool(
                 RedisSentinelConnectionManager {
                     client: tokio::sync::Mutex::new(client),
                     timeout: config.timeout.into_inner(),
@@ -178,7 +190,7 @@ impl RedisStore {
                 config.pool_timeout_wait,
                 config.pool_timeout_recycle,
             )?),
-        })))
+        ))))
     }
 }
 

crates/trc/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "trc"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/trc/event-macro/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "event_macro"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [lib]

crates/types/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "types"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/utils/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "utils"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [dependencies]

crates/utils/proc-macros/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "proc_macros"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [lib]

resources/schema/schema.json.sha256

@@ -1 +1 @@
-zUWyYdvOBMVeP1H7DNb7OqUThdpIaKjph4RyqBSKZAA
+lOFYe4x1vf5EPY-GcNW1BVwDo2coqV-oAiO0uFlOP14

tests/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "tests"
-version = "0.16.23"
+version = "0.16.24"
 edition = "2024"
 
 [features]

tests/resources/jmap/sieve/test_mailbox.sieve

@@ -1,4 +1,4 @@
-require ["fileinto", "mailbox", "mailboxid", "special-use", "ihave", "imap4flags", "vnd.stalwart.expressions"];
+require ["fileinto", "mailbox", "mailboxid", "special-use", "ihave", "imap4flags", "vnd.inbuxa.expressions"];
 
 # SpecialUse extension tests
 if not specialuse_exists ["inbox", "trash"] {

tests/resources/scripts/stress_test_prepare.py

@@ -8,7 +8,7 @@ import urllib.error
 import urllib.request
 
 CORE = "urn:ietf:params:jmap:core"
-STALWART = "urn:stalwart:jmap"
+STALWART = "urn:inbuxa:jmap:registry"
 USING = [CORE, STALWART]
 
 DEFAULT_BASE_URL = "https://127.0.0.1"

tests/resources/smtp/sieve/awl.sieve

@@ -1,4 +1,4 @@
-require ["variables", "include", "vnd.stalwart.expressions", "reject"];
+require ["variables", "include", "vnd.inbuxa.expressions", "reject"];
 
 global "score";
 

tests/resources/smtp/sieve/awl_include.sieve

@@ -1,4 +1,4 @@
-require ["variables", "include", "vnd.stalwart.expressions", "reject"];
+require ["variables", "include", "vnd.inbuxa.expressions", "reject"];
 
 global "score";
 set "awl_factor" "0.5";

tests/resources/smtp/sieve/stage_mail.sieve

@@ -1,4 +1,4 @@
-require ["variables", "envelope", "reject", "vnd.stalwart.expressions"];
+require ["variables", "envelope", "reject", "vnd.inbuxa.expressions"];
 
 if envelope :localpart :is "from" "spammer" {
     reject "450 4.1.1 Invalid address";

tests/resources/smtp/sieve/stage_rcpt.sieve

@@ -1,4 +1,4 @@
-require ["variables", "envelope", "reject", "vnd.stalwart.expressions"];
+require ["variables", "envelope", "reject", "vnd.inbuxa.expressions"];
 
 if envelope :domain :is "to" "foobar.org" {
     eval "query('sql', 'CREATE TABLE IF NOT EXISTS greylist (addr TEXT PRIMARY KEY)', [])";

tests/src/imap/antispam.rs

@@ -129,6 +129,21 @@ pub async fn test(test: &TestServer) {
     assert_eq!(samples.iter().filter(|x| !x.1.is_spam).count(), 11);
     assert_eq!(samples.iter().filter(|x| x.1.is_spam).count(), 11);
 
+    let last_id = samples.iter().map(|(id, _)| id.id()).max().unwrap();
+    for _ in 0..2 {
+        admin
+            .registry_create_object(Task::SpamFilterMaintenance(TaskSpamFilterMaintenance {
+                maintenance_type: TaskSpamFilterMaintenanceType::Train,
+                status: TaskStatus::now(),
+            }))
+            .await;
+        test.wait_for_tasks().await;
+        let model = spam_classifier_model(&test.server).await;
+        assert_eq!(model.reservoir.ham.total_seen, 11);
+        assert_eq!(model.reservoir.spam.total_seen, 11);
+        assert_eq!(model.last_id, last_id);
+    }
+
     // Global spam samples should not appear in the account
     let samples = account.spam_training_samples().await;
     assert_eq!(samples.iter().filter(|x| !x.1.is_spam).count(), 11);

tests/src/imap/copy_move.rs

@@ -174,4 +174,53 @@ pub async fn test(imap: &mut ImapConnection, imap_check: &mut ImapConnection) {
     imap.assert_read(Type::Tagged, ResponseType::Ok)
         .await
         .assert_contains("COPYUID");
+
+    move_store_race(imap, imap_check).await;
+}
+
+async fn move_store_race(imap: &mut ImapConnection, imap_check: &mut ImapConnection) {
+    const RACE_MESSAGES: usize = 10;
+    const RACE_ROUNDS: usize = 20;
+
+    // A MOVE that loses a race with a STORE on the same messages in another
+    // session is retried instead of failing with CONTACTADMIN
+    imap.send_ok("CREATE \"Race Left\"").await;
+    imap.send_ok("CREATE \"Race Right\"").await;
+    for i in 0..RACE_MESSAGES {
+        imap.append(
+            "Race Left",
+            &format!("From: [email protected]\r\nSubject: Move race {i}\r\n\r\nrace\r\n"),
+        )
+        .await;
+    }
+
+    for round in 0..RACE_ROUNDS {
+        let (src, dest, store) = if round % 2 == 0 {
+            ("Race Left", "Race Right", "UID STORE 1:* +FLAGS (\\Seen)")
+        } else {
+            ("Race Right", "Race Left", "UID STORE 1:* -FLAGS (\\Seen)")
+        };
+        imap.send_ok(&format!("SELECT \"{src}\"")).await;
+        imap_check.send_ok(&format!("SELECT \"{src}\"")).await;
+
+        // The STORE may lose the race instead, so only the MOVE is checked
+        imap.send(&format!("UID MOVE 1:* \"{dest}\"")).await;
+        imap_check.send(store).await;
+        let (moved, _) = tokio::join!(
+            imap.assert_read(Type::Tagged, ResponseType::Ok),
+            imap_check.read(Type::Tagged)
+        );
+        moved.assert_contains("COPYUID");
+
+        imap.send(&format!("STATUS \"{dest}\" (MESSAGES)")).await;
+        imap.assert_read(Type::Tagged, ResponseType::Ok)
+            .await
+            .assert_contains(&format!("(MESSAGES {RACE_MESSAGES})"));
+    }
+
+    // Restore the state the following tests expect
+    imap.send_ok("SELECT \"Burrata al Tartufo\"").await;
+    imap_check.send_ok("SELECT \"Burrata al Tartufo\"").await;
+    imap.send_ok("DELETE \"Race Left\"").await;
+    imap.send_ok("DELETE \"Race Right\"").await;
 }

tests/src/jmap/core/blob.rs

@@ -367,6 +367,82 @@ pub async fn test(test: &TestServer) {
     }
     test.blob_expire_all().await;
 
+    let inbox_id = Id::from(INBOX_ID).to_string();
+    let response = account
+        .jmap_method_calls(json!([
+          [
+            "Blob/upload",
+            {
+              "accountId": account.id_string(),
+              "create": {
+                "m0": {
+                  "data": [
+                    {
+                      "data:asText": concat!(
+                          "From: [email protected]\r\n",
+                          "To: [email protected]\r\n",
+                          "Subject: Blob reference import\r\n",
+                          "\r\n",
+                          "Imported through a Blob/upload creation id."
+                      )
+                    }
+                  ],
+                  "type": "message/rfc822"
+                }
+              }
+            },
+            "U0"
+          ],
+          [
+            "Email/import",
+            {
+              "accountId": account.id_string(),
+              "emails": {
+                "i0": {
+                  "blobId": "#m0",
+                  "mailboxIds": { (inbox_id.as_str()): true }
+                }
+              }
+            },
+            "I0"
+          ],
+          [
+            "Email/import",
+            {
+              "accountId": account.id_string(),
+              "emails": {
+                "i1": {
+                  "blobId": "#missing",
+                  "mailboxIds": { (inbox_id.as_str()): true }
+                }
+              }
+            },
+            "I1"
+          ]
+        ]))
+        .await;
+
+    assert_eq!(response.name_at(1), "Email/import", "{response:?}");
+    assert!(
+        response
+            .pointer("/methodResponses/1/1/created/i0/id")
+            .and_then(|v| v.as_str())
+            .is_some(),
+        "{response:?}"
+    );
+    assert!(
+        response
+            .pointer("/methodResponses/1/1/notCreated")
+            .is_none(),
+        "{response:?}"
+    );
+    assert_eq!(
+        response.error_type_at(2),
+        Some("invalidResultReference"),
+        "{response:?}"
+    );
+    test.blob_expire_all().await;
+
     // Blob/lookup
     let client = account.jmap_client().await;
     let blob_id = client

tests/src/jmap/core/push_subscription.rs

@@ -188,13 +188,37 @@ pub async fn test(test: &TestServer) {
     client.push_subscription_destroy(&push_id).await.unwrap();
 
     // Only one verification per minute is allowed
-    let push_id = client
-        .push_subscription_create("invalid", "https://127.0.0.1:19000/push", None)
+    let first_id = client
+        .push_subscription_create(
+            "first",
+            "https://127.0.0.1:19000/push?skip_checks=true",
+            None,
+        )
+        .await
+        .unwrap()
+        .take_id();
+    let verification = expect_push(&mut event_rx).await.unwrap_verification();
+    assert_eq!(verification.push_subscription_id, first_id);
+    let deferred_id = client
+        .push_subscription_create("deferred", "https://127.0.0.1:19000/push", None)
         .await
         .unwrap()
         .take_id();
     expect_nothing(&mut event_rx).await;
-    client.push_subscription_destroy(&push_id).await.unwrap();
+    let verification = expect_push_within(&mut event_rx, Duration::from_secs(8))
+        .await
+        .unwrap_verification();
+    assert_eq!(verification.push_subscription_id, deferred_id);
+    account
+        .jmap_request(
+            &["urn:ietf:params:jmap:core"],
+            json!([[
+                "PushSubscription/set",
+                { "destroy": [first_id, deferred_id] },
+                "0"
+            ]]),
+        )
+        .await;
 
     // Register push notification (with encryption)
     let push_id = client
@@ -731,7 +755,14 @@ fn assert_vapid_authorization(header: &str, expected_key: &str, expected_origin:
 }
 
 async fn expect_push(event_rx: &mut mpsc::Receiver<PushMessage>) -> PushMessage {
-    match tokio::time::timeout(Duration::from_millis(1500), event_rx.recv()).await {
+    expect_push_within(event_rx, Duration::from_millis(1500)).await
+}
+
+async fn expect_push_within(
+    event_rx: &mut mpsc::Receiver<PushMessage>,
+    wait: Duration,
+) -> PushMessage {
+    match tokio::time::timeout(wait, event_rx.recv()).await {
         Ok(Some(push)) => {
             //println!("Push received: {:?}", push);
             push

tests/src/jmap/mail/set.rs

@@ -8,7 +8,7 @@ use crate::{
     jmap::{find_values, replace_blob_ids, replace_boundaries, replace_values},
     utils::server::TestServer,
 };
-use ::email::mailbox::INBOX_ID;
+use ::email::{mailbox::INBOX_ID, message::metadata::MessageData};
 use ahash::AHashSet;
 use jmap_client::{
     Error, Set,
@@ -18,8 +18,12 @@ use jmap_client::{
     mailbox::Role,
 };
 use registry::schema::prelude::ObjectType;
-use std::{fs, path::PathBuf};
-use types::id::Id;
+use std::{fs, path::PathBuf, str::FromStr};
+use store::{
+    ValueKey,
+    write::{AlignedBytes, Archive},
+};
+use types::{collection::Collection, id::Id};
 
 pub async fn test(test: &TestServer) {
     println!("Running Email Set tests...");
@@ -29,6 +33,7 @@ pub async fn test(test: &TestServer) {
 
     create(&client, &mailbox_id).await;
     update(&client, &mailbox_id).await;
+    update_preserves_uids(test, &client, account.id().document_id(), &mailbox_id).await;
 
     test.destroy_all_mailboxes(account).await;
     test.account("[email protected]")
@@ -296,6 +301,94 @@ async fn update(client: &Client, root_mailbox_id: &str) {
         .unwrap();
 }
 
+async fn update_preserves_uids(
+    test: &TestServer,
+    client: &Client,
+    account_id: u32,
+    root_mailbox_id: &str,
+) {
+    let email_id = client
+        .email_query(
+            email::query::Filter::in_mailbox(root_mailbox_id).into(),
+            None::<Vec<_>>,
+        )
+        .await
+        .unwrap()
+        .take_ids()
+        .pop()
+        .unwrap();
+    let document_id = Id::from_str(&email_id).unwrap().document_id();
+    let test_mailbox_id = client
+        .mailbox_create("UID Test", None::<String>, Role::None)
+        .await
+        .unwrap()
+        .take_id();
+    let test_mailbox_document_id = Id::from_str(&test_mailbox_id).unwrap().document_id();
+    let uids = message_uids(test, account_id, document_id).await;
+    let inbox_uid = uids[&INBOX_ID];
+    assert_ne!(inbox_uid, 0);
+
+    // Full mailboxIds identical to the current ones plus a keyword change must keep the UID
+    let mut request = client.build();
+    request
+        .set_email()
+        .update(&email_id)
+        .mailbox_ids([root_mailbox_id])
+        .keywords(["uid-test"]);
+    request
+        .send_set_email()
+        .await
+        .unwrap()
+        .updated(&email_id)
+        .unwrap();
+    assert_eq!(
+        message_uids(test, account_id, document_id).await,
+        [(INBOX_ID, inbox_uid)].into_iter().collect()
+    );
+
+    // Full mailboxIds that keeps a mailbox and adds another must only assign a UID to the new one
+    let mut request = client.build();
+    request
+        .set_email()
+        .update(&email_id)
+        .mailbox_ids([root_mailbox_id, test_mailbox_id.as_str()]);
+    request
+        .send_set_email()
+        .await
+        .unwrap()
+        .updated(&email_id)
+        .unwrap();
+    let uids = message_uids(test, account_id, document_id).await;
+    assert_eq!(uids.len(), 2);
+    assert_eq!(uids[&INBOX_ID], inbox_uid);
+    assert_ne!(uids[&test_mailbox_document_id], 0);
+
+    client.mailbox_destroy(&test_mailbox_id, true).await.unwrap();
+}
+
+async fn message_uids(
+    test: &TestServer,
+    account_id: u32,
+    document_id: u32,
+) -> std::collections::BTreeMap<u32, u32> {
+    test.server
+        .store()
+        .get_value::<Archive<AlignedBytes>>(ValueKey::archive(
+            account_id,
+            Collection::Email,
+            document_id,
+        ))
+        .await
+        .unwrap()
+        .unwrap()
+        .deserialize::<MessageData>()
+        .unwrap()
+        .mailboxes
+        .iter()
+        .map(|m| (m.mailbox_id, m.uid))
+        .collect()
+}
+
 pub async fn assert_email_properties(
     client: &Client,
     message_id: &str,

tests/src/jmap/mod.rs

@@ -99,6 +99,7 @@ pub async fn jmap_tests() {
                 push_throttle: 500u64.into(),
                 websocket_throttle: 500u64.into(),
                 push_attempt_wait: 500u64.into(),
+                push_verify_timeout: 5000u64.into(),
                 ..Default::default()
             },
             &[
@@ -108,6 +109,7 @@ pub async fn jmap_tests() {
                 Property::PushThrottle,
                 Property::WebsocketThrottle,
                 Property::PushAttemptWait,
+                Property::PushVerifyTimeout,
             ],
         )
         .await;

tests/src/jmap/principal/get.rs

@@ -244,7 +244,7 @@ pub async fn test(test: &TestServer) {
               "webWriteUrlTemplate": null
             },
             "urn:ietf:params:jmap:mail:share": {},
-            "urn:stalwart:jmap": {}
+            "urn:inbuxa:jmap:registry": {}
           }
         }
       },
@@ -265,7 +265,7 @@ pub async fn test(test: &TestServer) {
         "urn:ietf:params:jmap:principals:availability": john_id,
         "urn:ietf:params:jmap:filenode": john_id,
         "urn:ietf:params:jmap:mail:share": john_id,
-        "urn:stalwart:jmap": john_id
+        "urn:inbuxa:jmap:registry": john_id
       },
       "username": "[email protected]",
       "apiUrl": "https://127.0.0.1:8899/jmap/",

tests/src/smtp/inbound/milter.rs

@@ -26,9 +26,12 @@ use registry::{
     schema::{
         enums::{self, MtaStage},
         prelude::{ObjectType, Property},
-        structs::{Expression, MtaHook, MtaMilter, MtaStageRcpt},
+        structs::{
+            Expression, ExpressionMatch, MtaHook, MtaMilter, MtaStageData, MtaStageRcpt,
+            SieveSystemScript,
         },
-    types::map::Map,
+    },
+    types::{list::List, map::Map},
 };
 use serde::Deserialize;
 use smtp::{
@@ -236,6 +239,34 @@ async fn mta_hook_session() {
             ..Default::default()
         })
         .await;
+    admin
+        .registry_create_object(MtaStageData {
+            script: Expression {
+                match_: List::from_iter([ExpressionMatch {
+                    if_: "sender = '[email protected]'".into(),
+                    then: "'tag_quarantine'".into(),
+                }]),
+                else_: "false".into(),
+            },
+            ..Default::default()
+        })
+        .await;
+    admin
+        .registry_create_object(SieveSystemScript {
+            contents: concat!(
+                "require [\"editheader\"];\n",
+                "addheader \"X-Sieve\" \"Seen\";\n",
+                "if exists \"X-Quarantine\" {\n",
+                "  deleteheader \"Subject\";\n",
+                "  addheader \"Subject\" \"INFECTED\";\n",
+                "}\n"
+            )
+            .into(),
+            description: None,
+            is_active: true,
+            name: "tag_quarantine".into(),
+        })
+        .await;
     admin.reload_settings().await;
     test.reload_core();
     test.expect_reload_settings().await;
@@ -353,6 +384,41 @@ async fn mta_hook_session() {
         .await
         .assert_contains("X-Spam: Yes")
         .assert_contains("123456");
+
+    // Test quarantine
+    session
+        .send_message(
+            "[email protected]",
+            &["[email protected]"],
+            "test:no_dkim",
+            "250 2.0.0",
+        )
+        .await;
+    test.expect_message()
+        .await
+        .read_lines(&test)
+        .await
+        .assert_contains("X-Quarantine: true")
+        .assert_contains("Subject: Is dinner ready?")
+        .assert_contains("Are you hungry yet?");
+
+    // Test that a DATA stage Sieve script sees and preserves hook modifications
+    session
+        .send_message(
+            "[email protected]",
+            &["[email protected]"],
+            "test:no_dkim",
+            "250 2.0.0",
+        )
+        .await;
+    test.expect_message()
+        .await
+        .read_lines(&test)
+        .await
+        .assert_contains("X-Quarantine: true")
+        .assert_contains("X-Sieve: Seen")
+        .assert_contains("Subject: INFECTED")
+        .assert_contains("Are you hungry yet?");
 }
 
 #[test]
@@ -863,6 +929,11 @@ fn handle_mta_hook(request: Request, tests: Arc<Vec<HeaderTest>>) -> hooks::Resp
             response: None,
             modifications: vec![],
         },
+        "quarantine" | "quarantine_only" => hooks::Response {
+            action: hooks::Action::Quarantine,
+            response: None,
+            modifications: vec![],
+        },
         "temp_fail" => hooks::Response {
             action: hooks::Action::Reject,
             response: SmtpResponse {

tests/src/smtp/inbound/mod.rs

@@ -34,6 +34,7 @@ pub mod rcpt;
 pub mod rewrite;
 pub mod scripts;
 pub mod sign;
+pub mod spam_rules;
 pub mod throttle;
 pub mod vrfy;
 

tests/src/smtp/inbound/spam_rules.rs

@@ -0,0 +1,364 @@
+/*
+ * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
+ *
+ * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
+ */
+
+use crate::utils::server::{TestServer, TestServerBuilder};
+use registry::{
+    schema::{
+        enums::TaskSpamFilterMaintenanceType,
+        prelude::{Object, ObjectType},
+        structs::{
+            HttpLookup, MemoryLookupKey, MemoryLookupKeyValue, SpamDnsblServer, SpamFileExtension,
+            SpamRule, SpamSettings, SpamTag, SpamTagScore, Task, TaskSpamFilterMaintenance,
+            TaskStatus,
+        },
+    },
+    types::{ObjectImpl, float::Float},
+};
+use serde_json::{Value, json};
+use std::fs;
+use store::registry::RegistryObject;
+
+#[tokio::test(flavor = "multi_thread")]
+async fn spam_rules_update() {
+    let test = TestServerBuilder::new("spam_rules_update_test")
+        .await
+        .with_http_listener(19057)
+        .await
+        .build()
+        .await;
+    let admin = test.account("admin");
+    let rules_path = test.temp_dir.path.join("spam-filter-rules.json");
+    admin
+        .registry_create_object(SpamSettings {
+            spam_filter_rules_url: format!("file://{}", rules_path.display()).into(),
+            ..Default::default()
+        })
+        .await;
+    admin
+        .registry_create_object(MemoryLookupKeyValue {
+            namespace: "test-keys".into(),
+            key: "conflict.org".into(),
+            value: "local".into(),
+            is_glob_pattern: false,
+        })
+        .await;
+    admin.reload_settings().await;
+
+    update_rules(&test, &release_1()).await;
+
+    for name in ["STWT_TEST_RULE", "STWT_TEST_DISABLED"] {
+        assert!(
+            stored::<SpamRule>(&test, "name", name)
+                .await
+                .unwrap()
+                .object
+                .enable()
+        );
+    }
+    assert!(
+        stored::<SpamDnsblServer>(&test, "name", "STWT_TEST_DNSBL")
+            .await
+            .unwrap()
+            .object
+            .enable()
+    );
+    assert!(
+        stored::<MemoryLookupKey>(&test, "key", "conflict.org")
+            .await
+            .is_none()
+    );
+
+    let disabled_rule = stored::<SpamRule>(&test, "name", "STWT_TEST_DISABLED")
+        .await
+        .unwrap();
+    admin
+        .registry_update_object(
+            ObjectType::SpamRule,
+            disabled_rule.id.id(),
+            json!({"enable": false}),
+        )
+        .await;
+    let local_tag = stored::<SpamTag>(&test, "tag", "TEST_TAG_LOCAL")
+        .await
+        .unwrap();
+    admin
+        .registry_update_object(
+            ObjectType::SpamTag,
+            local_tag.id.id(),
+            json!({"score": 2.0}),
+        )
+        .await;
+    let feed = stored::<HttpLookup>(&test, "namespace", "stwt_test_feed")
+        .await
+        .unwrap();
+    admin
+        .registry_update_object(
+            ObjectType::HttpLookup,
+            feed.id.id(),
+            json!({"enable": false}),
+        )
+        .await;
+
+    let release_2 = release_2();
+    for _ in 0..2 {
+        update_rules(&test, &release_2).await;
+
+        let rule = stored::<SpamRule>(&test, "name", "STWT_TEST_RULE")
+            .await
+            .unwrap()
+            .object;
+        assert!(rule.enable());
+        assert_eq!(object_json(&rule)["priority"], 400);
+
+        let rule = stored::<SpamRule>(&test, "name", "STWT_TEST_DISABLED")
+            .await
+            .unwrap()
+            .object;
+        assert!(!rule.enable());
+        assert_eq!(object_json(&rule)["priority"], 450);
+
+        assert!(
+            stored::<SpamRule>(&test, "name", "STWT_TEST_NEW")
+                .await
+                .is_some()
+        );
+
+        for upstream in release_2["SpamRule"].as_array().unwrap() {
+            let mut upstream: SpamRule = serde_json::from_value(upstream.clone()).unwrap();
+            let local = stored::<SpamRule>(
+                &test,
+                "name",
+                object_json(&upstream)["name"].as_str().unwrap(),
+            )
+            .await
+            .unwrap()
+            .object;
+            upstream.set_enable(local.enable());
+            assert_eq!(local, upstream);
+        }
+        for upstream in release_2["SpamDnsblServer"].as_array().unwrap() {
+            let upstream: SpamDnsblServer = serde_json::from_value(upstream.clone()).unwrap();
+            let local = stored::<SpamDnsblServer>(
+                &test,
+                "name",
+                object_json(&upstream)["name"].as_str().unwrap(),
+            )
+            .await
+            .unwrap()
+            .object;
+            assert_eq!(local, upstream);
+        }
+
+        for (tag, score) in [("TEST_TAG", 6.5), ("TEST_TAG_LOCAL", 2.0)] {
+            assert_eq!(
+                stored::<SpamTag>(&test, "tag", tag).await.unwrap().object,
+                SpamTag::Score(SpamTagScore {
+                    tag: tag.into(),
+                    score: Float::new(score),
+                })
+            );
+        }
+
+        let feed = stored::<HttpLookup>(&test, "namespace", "stwt_test_feed")
+            .await
+            .unwrap()
+            .object;
+        assert!(!feed.enable);
+        assert_eq!(feed.max_entries, 2000);
+
+        assert!(
+            stored::<MemoryLookupKey>(&test, "key", "example.org")
+                .await
+                .unwrap()
+                .object
+                .is_glob_pattern
+        );
+        assert!(
+            stored::<MemoryLookupKey>(&test, "key", "conflict.org")
+                .await
+                .is_none()
+        );
+        assert_eq!(
+            stored::<MemoryLookupKeyValue>(&test, "key", "conflict.org")
+                .await
+                .unwrap()
+                .object
+                .value,
+            "local"
+        );
+
+        assert!(
+            stored::<SpamFileExtension>(&test, "extension", "tst")
+                .await
+                .unwrap()
+                .object
+                .is_bad
+        );
+    }
+}
+
+async fn update_rules(test: &TestServer, rules: &Value) {
+    fs::write(
+        test.temp_dir.path.join("spam-filter-rules.json"),
+        rules.to_string(),
+    )
+    .unwrap();
+    test.account("admin")
+        .registry_create_object(Task::SpamFilterMaintenance(TaskSpamFilterMaintenance {
+            maintenance_type: TaskSpamFilterMaintenanceType::UpdateRules,
+            status: TaskStatus::now(),
+        }))
+        .await;
+    test.wait_for_tasks().await;
+}
+
+async fn stored<T: ObjectImpl + From<Object>>(
+    test: &TestServer,
+    property: &str,
+    value: &str,
+) -> Option<RegistryObject<T>> {
+    test.server
+        .registry()
+        .list::<T>()
+        .await
+        .unwrap()
+        .into_iter()
+        .find(|item| object_json(&item.object)[property] == value)
+}
+
+fn object_json<T: ObjectImpl>(object: &T) -> Value {
+    serde_json::to_value(object).unwrap()
+}
+
+fn release_1() -> Value {
+    json!({
+        "SpamRule": [
+            {
+                "@type": "Any",
+                "name": "STWT_TEST_RULE",
+                "enable": true,
+                "priority": 500,
+                "condition": {"else": "false", "match": {"0": {"if": "$MISSING_ESSENTIAL_HEADERS && $SINGLE_SHORT_PART", "then": "'SHORT_PART_BAD_HEADERS'"}}}
+            },
+            {
+                "@type": "Any",
+                "name": "STWT_TEST_DISABLED",
+                "enable": true,
+                "priority": 500,
+                "condition": {"else": "false", "match": {"0": {"if": "$MISSING_ESSENTIAL_HEADERS && $SINGLE_SHORT_PART", "then": "'SHORT_PART_BAD_HEADERS'"}}}
+            }
+        ],
+        "SpamDnsblServer": [
+            {
+                "@type": "Domain",
+                "name": "STWT_TEST_DNSBL",
+                "enable": true,
+                "tag": {"else": "false", "match": {
+                    "0": {"if": "octets[3] == 1", "then": "'URIBL_BLOCKED'"},
+                    "1": {"if": "octets[3] == 2", "then": "'URIBL_BLACK'"},
+                    "2": {"if": "octets[3] == 4", "then": "'URIBL_GREY'"},
+                    "3": {"if": "octets[3] == 8", "then": "'URIBL_RED'"}
+                }},
+                "zone": {"else": "value + '.multi.uribl.com'", "match": {}}
+            }
+        ],
+        "SpamTag": [
+            {"@type": "Score", "tag": "TEST_TAG", "score": 6.5},
+            {"@type": "Score", "tag": "TEST_TAG_LOCAL", "score": 1.0}
+        ],
+        "HttpLookup": [
+            {
+                "namespace": "stwt_test_feed",
+                "url": "http://127.0.0.1:1/feed.txt",
+                "format": {"@type": "List"},
+                "enable": true,
+                "isGzipped": false,
+                "maxSize": 1048576,
+                "maxEntries": 1000,
+                "maxEntrySize": 512,
+                "refresh": 43200000,
+                "retry": 3600000,
+                "timeout": 30000
+            }
+        ],
+        "MemoryLookupKey": [
+            {"namespace": "test-keys", "key": "example.org", "isGlobPattern": false},
+            {"namespace": "test-keys", "key": "conflict.org", "isGlobPattern": false}
+        ],
+        "SpamFileExtension": [
+            {"extension": "tst", "contentTypes": {}, "isArchive": false, "isBad": false, "isNz": false}
+        ]
+    })
+}
+
+fn release_2() -> Value {
+    json!({
+        "SpamRule": [
+            {
+                "@type": "Any",
+                "name": "STWT_TEST_RULE",
+                "enable": true,
+                "priority": 400,
+                "condition": {"else": "false", "match": {"0": {"if": "$MISSING_ESSENTIAL_HEADERS && $SINGLE_SHORT_PART", "then": "'SHORT_PART_BAD_HEADERS'"}}}
+            },
+            {
+                "@type": "Any",
+                "name": "STWT_TEST_DISABLED",
+                "enable": true,
+                "priority": 450,
+                "condition": {"else": "false", "match": {"0": {"if": "$MISSING_ESSENTIAL_HEADERS && $SINGLE_SHORT_PART", "then": "'SHORT_PART_BAD_HEADERS'"}}}
+            },
+            {
+                "@type": "Any",
+                "name": "STWT_TEST_NEW",
+                "enable": true,
+                "priority": 500,
+                "condition": {"else": "false", "match": {"0": {"if": "$MISSING_ESSENTIAL_HEADERS && $SINGLE_SHORT_PART", "then": "'SHORT_PART_BAD_HEADERS'"}}}
+            }
+        ],
+        "SpamDnsblServer": [
+            {
+                "@type": "Domain",
+                "name": "STWT_TEST_DNSBL",
+                "enable": true,
+                "tag": {"else": "false", "match": {
+                    "0": {"if": "octets[0] != 127", "then": "false"},
+                    "1": {"if": "octets[3] == 1", "then": "'URIBL_BLOCKED'"},
+                    "2": {"if": "bit_and(octets[3], 2) != 0", "then": "'URIBL_BLACK'"},
+                    "3": {"if": "bit_and(octets[3], 4) != 0", "then": "'URIBL_GREY'"},
+                    "4": {"if": "bit_and(octets[3], 8) != 0", "then": "'URIBL_RED'"}
+                }},
+                "zone": {"else": "value + '.multi.uribl.com'", "match": {}}
+            }
+        ],
+        "SpamTag": [
+            {"@type": "Score", "tag": "TEST_TAG", "score": 4.5},
+            {"@type": "Score", "tag": "TEST_TAG_LOCAL", "score": 1.0}
+        ],
+        "HttpLookup": [
+            {
+                "namespace": "stwt_test_feed",
+                "url": "http://127.0.0.1:1/feed.txt",
+                "format": {"@type": "List"},
+                "enable": true,
+                "isGzipped": false,
+                "maxSize": 1048576,
+                "maxEntries": 2000,
+                "maxEntrySize": 512,
+                "refresh": 43200000,
+                "retry": 3600000,
+                "timeout": 30000
+            }
+        ],
+        "MemoryLookupKey": [
+            {"namespace": "test-keys", "key": "example.org", "isGlobPattern": true},
+            {"namespace": "test-keys", "key": "conflict.org", "isGlobPattern": false}
+        ],
+        "SpamFileExtension": [
+            {"extension": "tst", "contentTypes": {}, "isArchive": false, "isBad": true, "isNz": false}
+        ]
+    })
+}

tests/src/system/task.rs

@@ -4,7 +4,7 @@
  * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
  */
 
-use crate::utils::{account::Account, server::TestServer};
+use crate::utils::{account::Account, jmap::JmapUtils, server::TestServer};
 use registry::{
     schema::{
         enums::TaskStoreMaintenanceType,
@@ -297,14 +297,29 @@ impl Account {
         .await
     }
 
-    pub async fn tasks(&self) -> Vec<TaskId> {
+    pub async fn tasks(&self) -> Option<Vec<TaskId>> {
         let ids = self.task_ids().await;
-        let mut results = Vec::with_capacity(ids.len());
-        for id in ids {
-            let sample = self.registry_get::<Task>(id).await;
-            results.push(TaskId { id, task: sample });
+        if ids.is_empty() {
+            return Some(Vec::new());
         }
-        results
+
+        let response = self.registry_get_many(ObjectType::Task, &ids).await;
+        if response.not_found().next().is_some() {
+            return None;
+        }
+
+        Some(
+            response
+                .list()
+                .iter()
+                .map(|item| TaskId {
+                    id: item.object_id(),
+                    task: serde_json::from_str(&item.to_string()).unwrap_or_else(|err| {
+                        panic!("Failed to deserialize {item}: {err}");
+                    }),
+                })
+                .collect(),
+        )
     }
 
     async fn assert_no_tasks(&self) {
@@ -340,11 +355,11 @@ impl Account {
     ) -> Vec<TaskId> {
         let mut attempt = 0;
         loop {
-            let tasks = self.tasks().await;
-            if tasks.len() == count && tasks.iter().all(&is_expected) {
+            match self.tasks().await {
+                Some(tasks) if tasks.len() == count && tasks.iter().all(&is_expected) => {
                     return tasks;
                 }
-
+                Some(tasks) => {
                     attempt += 1;
                     assert!(
                         attempt < TASK_WAIT_ATTEMPTS,
@@ -353,6 +368,15 @@ impl Account {
                         tasks.len(),
                         tasks
                     );
+                }
+                None => {
+                    attempt += 1;
+                    assert!(
+                        attempt < TASK_WAIT_ATTEMPTS,
+                        "Task/query keeps returning ids that Task/get reports as not found"
+                    );
+                }
+            }
             tokio::time::sleep(TASK_WAIT_INTERVAL).await;
         }
     }

tests/src/utils/jmap.rs

@@ -408,7 +408,7 @@ impl Account {
             "urn:ietf:params:jmap:principals:availability",
             "urn:ietf:params:jmap:filenode",
             "urn:ietf:params:jmap:mail:share",
-            "urn:stalwart:jmap"
+            "urn:inbuxa:jmap:registry"
           ],
           "methodCalls": calls
         });

tests/src/webdav/lock.rs

@@ -45,7 +45,7 @@ pub async fn test(test: &TestServer) {
 
         // Test 2: Refreshing a lock token with an invalid a lock token should fail
         client
-            .lock_refresh(&path, "urn:stalwart:davlock:1234", "infinity", "Second-456")
+            .lock_refresh(&path, "urn:inbuxa:davlock:1234", "infinity", "Second-456")
             .await
             .with_status(StatusCode::PRECONDITION_FAILED);
 
@@ -148,7 +148,7 @@ pub async fn test(test: &TestServer) {
 
         // Test 10: Unlock with and without a lock token
         client
-            .unlock(&path, "urn:stalwart:davlock:1234")
+            .unlock(&path, "urn:inbuxa:davlock:1234")
             .await
             .with_status(StatusCode::CONFLICT)
             .with_value("D:error.D:lock-token-matches-request-uri", "");