Cluster role changes apply to delivery and tasks without a restart

In cluster rehearsal 3, turning outboundMta off on node1's role was
reported applied (x:settingsReload applied: true), yet node1 kept
delivering mail, a report message included, until it was restarted.
The queue and report managers were started at boot only when the
node's role included outboundMta (crates/smtp/src/lib.rs), and the task
manager only when the role had some task type (spawn_task_manager).
After that nothing looked at the role again: a queue manager that was
running kept claiming and delivering, and one that wasn't never
started.

They now start on every node (outside recovery mode) and follow the
role live:

- Queue manager: before each scan it reads the role from the running
  settings. Without outboundMta it claims nothing new; deliveries
  already running finish and report back as usual, which releases
  their locks. When the role comes back (a reload wakes the manager
  with ReloadSettings, and it looks again every 30 s regardless) it
  logs queue.started and scans the whole queue at once.
- Report scheduler: DMARC and TLS report events are handled only while
  the role has outboundMta, as at boot; events arriving without it are
  dropped, as they were on a node started without the role.
- Task manager: task_enabled already read the current role on every
  scan. It now also runs on nodes whose role has no task type (the
  scan returns at once until one is added), a job claimed before a
  role change is handed back at once rather than run or held until
  its lease lapses, and a settings reload wakes the manager so a role
  that gained task types starts claiming them straight away.

Starting the queue manager on every node also drains the queue channel
on nodes without outboundMta. Upstream left that channel unread, so
each message queued there parked a refresh in it, and by the code,
queueing would block once 1024 had piled up (not reproduced here).

A role object edit reaches the nodes that name that role in
INBUXA_ROLE. Moving a node to another role still means changing its
environment, and so a restart. Listener changes in a role still need a
restart too (listeners bind at boot); this change is about tasks and
delivery.

cluster::live_roles::live_role_tests (new; PostgreSQL, two nodes over
one store):
1. A node started with outboundMta delivers and runs a TLS report
   task; after its role loses outboundMta and the settings reload, a
   new message isn't attempted and a new report task stays pending;
   with the role back, both are taken up.
2. A node started with no task type at all gains outboundMta: a
   waiting message is attempted and a report task runs.
On main the test fails at step 1 ("delivery attempted without
outboundMta"); with step 1 bypassed, step 2 fails (nothing picked the
message up in 20 s).

crates/smtp/src/lib.rs

@@ -44,7 +44,14 @@ impl StartQueueManager for BootManager {
 impl SpawnQueueManager for IpcReceivers {
     fn spawn_queue_manager(&mut self, inner: Arc<Inner>) {
         let core = inner.shared_core.load();
-        if !core.storage.registry.is_recovery_mode() && core.network.roles.outbound_mta {
+        // inbuxa: upstream started these only when the node's role included
+        // outboundMta at boot, so turning the role on later did nothing and
+        // turning it off left them delivering until a restart. They now run
+        // on every node and follow the role live (see Queue::start and the
+        // report scheduler). This also drains the queue channel on nodes
+        // without the role, where every queued message's refresh used to sit
+        // in a channel nobody read until it filled and queueing blocked.
+        if !core.storage.registry.is_recovery_mode() {
             // Spawn queue manager
             self.queue_rx.take().unwrap().spawn(inner.clone());
 

crates/smtp/src/queue/manager.rs

@@ -2,6 +2,8 @@
  * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
  *
  * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
+ *
+ * Modified by Coffey Labs in 2026 for INBUXA.
  */
 
 use super::{Message, QueueId, Status, spool::SmtpSpool};
@@ -39,6 +41,9 @@ pub struct Queue {
     pub urgent_refresh: bool,
     pub last_scan: Instant,
     pub last_full_scan: Instant,
+    /// inbuxa: whether this node's role included outboundMta when last
+    /// checked (None before the first check)
+    pub role_enabled: Option<bool>,
 }
 
 #[derive(Debug)]
@@ -67,6 +72,9 @@ impl SpawnQueue for mpsc::Receiver<QueueEvent> {
 const BACK_PRESSURE_WARN_INTERVAL: Duration = Duration::from_secs(60);
 const MIN_SCAN_INTERVAL: Duration = Duration::from_millis(100);
 const FULL_SCAN_INTERVAL: Duration = Duration::from_secs(QUEUE_REFRESH / 2);
+/// inbuxa: how often a node without the outbound MTA role looks at its role
+/// again when nothing else wakes it (a settings reload does)
+const ROLE_RECHECK_INTERVAL: Duration = Duration::from_secs(30);
 
 impl Queue {
     pub fn new(core: Arc<Inner>, rx: mpsc::Receiver<QueueEvent>) -> Self {
@@ -87,6 +95,7 @@ impl Queue {
             urgent_refresh: false,
             last_scan: now.checked_sub(MIN_SCAN_INTERVAL).unwrap_or(now),
             last_full_scan: now,
+            role_enabled: None,
         }
     }
 
@@ -123,6 +132,27 @@ impl Queue {
                 continue;
             }
 
+            // inbuxa: follow the node's role live. Without outboundMta the
+            // queue claims nothing new; deliveries already running finish
+            // and report back as usual, releasing their locks. When the role
+            // comes back, the whole queue is scanned at once.
+            let role_enabled = self.core.shared_core.load().network.roles.outbound_mta;
+            if self.role_enabled.replace(role_enabled) == Some(false) && role_enabled {
+                trc::event!(
+                    Queue(trc::QueueEvent::Started),
+                    Details = "This node's cluster role now includes outboundMta",
+                );
+                self.scan_from = 0;
+                self.pending_refresh = true;
+                self.urgent_refresh = true;
+            }
+            if !role_enabled {
+                self.pending_refresh = false;
+                self.urgent_refresh = false;
+                self.next_refresh = Instant::now() + ROLE_RECHECK_INTERVAL;
+                continue;
+            }
+
             self.pending_refresh |= refresh_queue;
             if !self.pending_refresh && self.next_refresh > Instant::now() {
                 continue;

crates/smtp/src/reporting/scheduler.rs

@@ -2,6 +2,8 @@
  * SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
  *
  * SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
+ *
+ * Modified by Coffey Labs in 2026 for INBUXA.
  */
 
 use super::{dmarc::DmarcReporting, tls::TlsReporting};
@@ -18,6 +20,14 @@ impl SpawnReport for mpsc::Receiver<ReportingEvent> {
         tokio::spawn(async move {
             while let Some(event) = self.recv().await {
                 let server = inner.build_server();
+                // inbuxa: reports are the outbound MTA's business, as at
+                // boot, but the role is read per event so a change applies
+                // without a restart. Events that arrive while the role is
+                // off are dropped, as they were on a node started without it
+                if !matches!(event, ReportingEvent::Stop) && !server.core.network.roles.outbound_mta
+                {
+                    continue;
+                }
                 match event {
                     ReportingEvent::Dmarc(event) => server.schedule_dmarc(event).await,
                     ReportingEvent::Tls(event) => server.schedule_tls(event).await,