Give IMAP, POP3 and ManageSieve a switch each

The legacy-protocols switch was all or nothing. An operator can now stop
POP3 and keep IMAP: each of IMAP, POP3 and ManageSieve has its own
switch, server-wide on inbuxa:ProtocolPolicy and per tenant on
inbuxa:TenantProtocolPolicy (properties imap, pop3, manageSieve).

legacyProtocols stays as the kill-all: setting it sets all three, and it
reads "disabled" exactly when all three are off. A policy stored before
this has only legacyProtocols and reads as all three at that value, so
existing servers and tenants carry over unchanged. In one /set, a
protocol named beside legacyProtocols overrides it.

SMTP submission keeps no switch of its own: sign-in over it is refused
only when all three are off, as the single switch did (LP-6), so
turning one protocol off never stops a mail app sending. For a tenant,
the server's switches and the tenant's count together.

Server-wide, a change closes the listeners of whatever is now off and
puts back the saved listeners of whatever is on again, both in one
change if asked; listeners of a protocol still off stay saved. Sign-in,
autoconfig, autodiscover, PACC (now prepared once per combination) and
the suggested DNS records all follow each protocol separately. A tenant
may turn a protocol on only while the server has it on (LP-9), and the
refusal names which. The JMAP session adds legacyAllowed, the protocols
still allowed for the account; legacyProtocols there keeps its meaning
for older webmail builds. Events name the switches ("pop3 disabled"),
and audit before/after reads every switch even from an older policy.

Tested: unit tests for the switches, the old-policy reading, the
server/tenant combination, the tenant refusal and listener refusal; and
tests/e2e/legacy_protocols.py against a running server, all 100 checks,
including new ones: POP3 alone off closes only its port and refuses
only its sign-in while IMAP and sending go on; only POP3 stops being
advertised; one change closes IMAP and reopens POP3; a tenant turns
POP3 off for itself, and can't turn IMAP on while the server has it off.

crates/features/src/security/protocol_policy.rs

@@ -8,6 +8,17 @@
 //! (legacy-protocols spec, data model and LP-1 to LP-8). Stored as JSON under
 //! `P` + `p` in the fork's subspace; unset fields read as the defaults.
 //!
+//! Each mail-app protocol has its own switch (legacy-protocols spec,
+//! "Revisit: one switch per protocol"): IMAP, POP3 and ManageSieve.
+//! `legacyProtocols` is the kill-all: setting it sets all three, and it reads
+//! `disabled` exactly when all three are off. A policy stored before the
+//! per-protocol switches has only `legacyProtocols`, and reads as all three
+//! at that value.
+//!
+//! SMTP submission has no switch of its own here: sign-in over it is refused
+//! only when all three are off, as it was by the single switch (LP-6), so
+//! turning off one protocol never stops a mail app sending.
+//!
 //! This module is the fact, not the act. It holds what the operator chose and
 //! which listeners were taken away to honour it. Closing sockets belongs to
 //! `common`, which owns the listener registry, and removing the listener
@@ -59,12 +70,30 @@ pub struct SavedListener {
     pub object: serde_json::Value,
 }
 
-/// The server-wide switch.
+/// The protocols with a switch of their own, as the schema and JMAP spell
+/// them.
+pub const SWITCHED: &[&str] = &["imap", "pop3", "manageSieve"];
+
+/// The name sign-in uses for SMTP AUTH, which follows the kill-all.
+pub const SUBMISSION: &str = "submission";
+
+/// The server-wide switches.
 #[derive(Debug, Clone, PartialEq, SerdeSerialize, SerdeDeserialize)]
 #[serde(rename_all = "camelCase", default)]
 pub struct ProtocolPolicy {
-    /// The switch itself.
+    /// The kill-all: `disabled` exactly when all three protocols are off,
+    /// once [`ProtocolPolicy::normalize`] has run. In a policy stored before
+    /// the per-protocol switches, it is the value of all three.
     pub legacy_protocols: LegacyProtocols,
+    /// IMAP's switch. Unset reads as `legacy_protocols`.
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub imap: Option<LegacyProtocols>,
+    /// POP3's switch. Unset reads as `legacy_protocols`.
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub pop3: Option<LegacyProtocols>,
+    /// ManageSieve's switch. Unset reads as `legacy_protocols`.
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub manage_sieve: Option<LegacyProtocols>,
     /// With `disabled`, also close SMTP submission (LP-3). The inbound
     /// listener on port 25 is never closed, whatever this says.
     pub close_submission: bool,
@@ -80,6 +109,9 @@ impl Default for ProtocolPolicy {
     fn default() -> Self {
         ProtocolPolicy {
             legacy_protocols: LegacyProtocols::Enabled,
+            imap: None,
+            pop3: None,
+            manage_sieve: None,
             close_submission: true,
             saved_listeners: Vec::new(),
             changed_at: None,
@@ -91,6 +123,9 @@ impl Default for ProtocolPolicy {
 /// The properties `inbuxa:ProtocolPolicy` has, as they appear over JMAP.
 pub const PROPERTIES: &[&str] = &[
     "legacyProtocols",
+    "imap",
+    "pop3",
+    "manageSieve",
     "closeSubmission",
     "savedListeners",
     "changedAt",
@@ -129,23 +164,137 @@ pub fn is_locked(protocol: &str) -> bool {
         .any(|locked| locked.eq_ignore_ascii_case(protocol))
 }
 
-impl ProtocolPolicy {
-    /// Whether a listener of this protocol and these ports is one the switch
-    /// closes. A listener bound to port 25 is inbound whatever its name, and
-    /// any other SMTP listener counts as submission (LP-3).
-    pub fn closes(&self, protocol: &str, ports: &[u16]) -> bool {
-        if !self.legacy_protocols.is_disabled() {
-            return false;
+/// The switch fields, by protocol name.
+pub trait Switches {
+    /// The kill-all, which an unset per-protocol switch reads as.
+    fn all(&self) -> LegacyProtocols;
+    fn slot(&self, protocol: &str) -> Option<&Option<LegacyProtocols>>;
+    fn slot_mut(&mut self, protocol: &str) -> Option<&mut Option<LegacyProtocols>>;
+    fn set_all_field(&mut self, value: LegacyProtocols);
+
+    /// One protocol's switch. `submission` follows the kill-all: it is off
+    /// only when all three are. Anything else has no switch and is on.
+    fn switch(&self, protocol: &str) -> LegacyProtocols {
+        if protocol == SUBMISSION {
+            return if self.all_off() {
+                LegacyProtocols::Disabled
+            } else {
+                LegacyProtocols::Enabled
+            };
         }
+        match self.slot(protocol) {
+            Some(value) => value.unwrap_or(self.all()),
+            None => LegacyProtocols::Enabled,
+        }
+    }
+
+    /// Whether this protocol is off.
+    fn is_off(&self, protocol: &str) -> bool {
+        self.switch(protocol).is_disabled()
+    }
+
+    /// Whether all three protocols are off.
+    fn all_off(&self) -> bool {
+        SWITCHED.iter().all(|protocol| {
+            self.slot(protocol)
+                .and_then(|value| *value)
+                .unwrap_or(self.all())
+                .is_disabled()
+        })
+    }
+
+    /// Sets one protocol's switch; false if it has none.
+    fn set(&mut self, protocol: &str, value: LegacyProtocols) -> bool {
+        match self.slot_mut(protocol) {
+            Some(slot) => {
+                *slot = Some(value);
+                true
+            }
+            None => false,
+        }
+    }
+
+    /// The kill-all: all three at once.
+    fn set_all(&mut self, value: LegacyProtocols) {
+        for protocol in SWITCHED {
+            self.set(protocol, value);
+        }
+        self.set_all_field(value);
+    }
+
+    /// Writes out every switch and derives the kill-all from them, so what is
+    /// stored and shown never depends on how it was reached.
+    fn normalize(&mut self) {
+        let values: Vec<_> = SWITCHED.iter().map(|p| self.switch(p)).collect();
+        for (protocol, value) in SWITCHED.iter().zip(values) {
+            self.set(protocol, value);
+        }
+        let all = if self.all_off() {
+            LegacyProtocols::Disabled
+        } else {
+            LegacyProtocols::Enabled
+        };
+        self.set_all_field(all);
+    }
+
+    /// The protocols that are off.
+    fn off(&self) -> Vec<&'static str> {
+        SWITCHED
+            .iter()
+            .copied()
+            .filter(|p| self.is_off(p))
+            .collect()
+    }
+}
+
+macro_rules! switches {
+    ($t:ty) => {
+        impl Switches for $t {
+            fn all(&self) -> LegacyProtocols {
+                self.legacy_protocols
+            }
+            fn slot(&self, protocol: &str) -> Option<&Option<LegacyProtocols>> {
+                match protocol {
+                    "imap" => Some(&self.imap),
+                    "pop3" => Some(&self.pop3),
+                    "manageSieve" => Some(&self.manage_sieve),
+                    _ => None,
+                }
+            }
+            fn slot_mut(&mut self, protocol: &str) -> Option<&mut Option<LegacyProtocols>> {
+                match protocol {
+                    "imap" => Some(&mut self.imap),
+                    "pop3" => Some(&mut self.pop3),
+                    "manageSieve" => Some(&mut self.manage_sieve),
+                    _ => None,
+                }
+            }
+            fn set_all_field(&mut self, value: LegacyProtocols) {
+                self.legacy_protocols = value;
+            }
+        }
+    };
+}
+pub(crate) use switches;
+
+switches!(ProtocolPolicy);
+
+impl ProtocolPolicy {
+    /// Whether a listener of this protocol and these ports is one the
+    /// switches close. A listener bound to port 25 is inbound whatever its
+    /// name, and any other SMTP listener counts as submission (LP-3), closed
+    /// only with all three off and `closeSubmission`.
+    pub fn closes(&self, protocol: &str, ports: &[u16]) -> bool {
         // The lock is checked first and answers for every caller, so no
         // request phrasing can reach past it (LP-21).
         if is_locked(protocol) {
             return false;
         }
         if LEGACY_PROTOCOLS.contains(&protocol) {
-            return true;
+            return self.is_off(protocol);
         }
         protocol.eq_ignore_ascii_case("smtp")
+            && self.all_off()
             && self.close_submission
             && !ports.contains(&INBOUND_SMTP_PORT)
     }
@@ -258,7 +407,10 @@ mod tests {
             "an unset closeSubmission reads as the default, true"
         );
 
-        let json = serde_json::to_value(&policy).unwrap();
+        // As shown: normalized, every switch written out.
+        let mut shown = policy.clone();
+        shown.normalize();
+        let json = serde_json::to_value(&shown).unwrap();
         for property in PROPERTIES {
             assert!(json.get(property).is_some(), "{property}");
         }
@@ -410,6 +562,72 @@ mod tests {
         );
     }
 
+    /// A policy stored before the per-protocol switches reads as all three
+    /// at its one value.
+    #[test]
+    fn an_old_policy_reads_as_all_three() {
+        let old: ProtocolPolicy =
+            serde_json::from_str(r#"{"legacyProtocols": "disabled"}"#).unwrap();
+        for p in SWITCHED {
+            assert!(old.is_off(p), "{p}");
+        }
+        assert!(old.all_off() && old.is_off(SUBMISSION));
+        let old: ProtocolPolicy =
+            serde_json::from_str(r#"{"legacyProtocols": "enabled"}"#).unwrap();
+        assert!(old.off().is_empty() && !old.is_off(SUBMISSION));
+    }
+
+    /// One protocol off closes only its listeners, and leaves sending alone.
+    #[test]
+    fn one_protocol_off() {
+        let mut policy = ProtocolPolicy::default();
+        policy.set("pop3", LegacyProtocols::Disabled);
+        policy.normalize();
+        assert!(policy.closes("pop3", &[995]));
+        assert!(!policy.closes("imap", &[993]));
+        assert!(!policy.closes("manageSieve", &[4190]));
+        assert!(!policy.is_off(SUBMISSION), "sending goes on");
+        assert_eq!(policy.legacy_protocols, LegacyProtocols::Enabled);
+        assert_eq!(policy.off(), vec!["pop3"]);
+        let json = serde_json::to_value(&policy).unwrap();
+        assert_eq!(json["pop3"], "disabled");
+        assert_eq!(json["imap"], "enabled");
+    }
+
+    /// Turning the three off one at a time is the kill-all, and the kill-all
+    /// back on turns all three on.
+    #[test]
+    fn the_kill_all_is_all_three() {
+        let mut policy = ProtocolPolicy::default();
+        for p in SWITCHED {
+            policy.set(p, LegacyProtocols::Disabled);
+        }
+        policy.normalize();
+        assert!(policy.legacy_protocols.is_disabled());
+        assert!(policy.is_off(SUBMISSION));
+
+        policy.set_all(LegacyProtocols::Enabled);
+        policy.normalize();
+        assert!(policy.off().is_empty());
+        assert!(!policy.legacy_protocols.is_disabled());
+
+        // The kill-all then one back on: no longer all off.
+        policy.set_all(LegacyProtocols::Disabled);
+        policy.set("imap", LegacyProtocols::Enabled);
+        policy.normalize();
+        assert!(!policy.legacy_protocols.is_disabled());
+        assert_eq!(policy.off(), vec!["pop3", "manageSieve"]);
+    }
+
+    /// Protocols without a switch are never off.
+    #[test]
+    fn unswitched_protocols_are_on() {
+        let policy = disabled();
+        for p in ["smtp", "http", "lmtp", "jmap"] {
+            assert!(!policy.is_off(p), "{p}");
+        }
+    }
+
     /// A saved listener with no id is refused, naming the property.
     #[test]
     fn a_nameless_saved_listener_is_refused() {

crates/features/src/security/tenant_protocol_policy.rs

@@ -9,12 +9,18 @@
 //! the tenant id in the fork's subspace; a tenant with nothing stored has
 //! legacy protocols on.
 //!
+//! A tenant has the same three switches as the server (IMAP, POP3,
+//! ManageSieve) and the same kill-all; a protocol off server-wide is off for
+//! every tenant whatever the tenant's own switch says.
+//!
 //! A tenant's switch closes no port -- other tenants share them (LP-13). It
 //! refuses sign-in on the tenant's domains, and keeps client configuration
 //! for them from offering what's refused. That is all it is: one fact per
 //! tenant, easy to turn back, touching no listener, role or permission.
 
-use crate::security::protocol_policy::{LegacyProtocols, ProtocolPolicy};
+use crate::security::protocol_policy::{
+    LegacyProtocols, ProtocolPolicy, SUBMISSION, SWITCHED, Switches, switches,
+};
 use serde::{Deserialize as SerdeDeserialize, Serialize as SerdeSerialize};
 use store::{
     Deserialize, SUBSPACE_INBUXA, Store, ValueKey,
@@ -26,24 +32,92 @@ use trc::AddContext;
 #[derive(Debug, Clone, PartialEq, Default, SerdeSerialize, SerdeDeserialize)]
 #[serde(rename_all = "camelCase", default)]
 pub struct TenantProtocolPolicy {
-    /// The switch itself.
+    /// The kill-all, as on the server's policy.
     pub legacy_protocols: LegacyProtocols,
+    /// IMAP's switch. Unset reads as `legacy_protocols`.
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub imap: Option<LegacyProtocols>,
+    /// POP3's switch. Unset reads as `legacy_protocols`.
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub pop3: Option<LegacyProtocols>,
+    /// ManageSieve's switch. Unset reads as `legacy_protocols`.
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub manage_sieve: Option<LegacyProtocols>,
     /// When it last changed, in milliseconds since the epoch.
     pub changed_at: Option<u64>,
     /// The account that last changed it.
     pub changed_by: Option<String>,
 }
 
-/// Why a tenant's switch can't be set this way, if it can't (LP-9).
+switches!(TenantProtocolPolicy);
+
+/// Why a tenant's switches can't be set this way, if they can't (LP-9).
 ///
-/// A tenant can always turn legacy protocols off for itself. It can turn
-/// them back on only while the server has them on: server off means off for
-/// everyone.
-pub fn refusal(server: &ProtocolPolicy, requested: LegacyProtocols) -> Option<&'static str> {
-    (server.legacy_protocols.is_disabled() && !requested.is_disabled()).then_some(
-        "Legacy mail protocols are off for the whole server (inbuxa:ProtocolPolicy), \
-         so they can't be turned back on for one organization.",
-    )
+/// A tenant can always turn a protocol off for itself. It can turn one on
+/// only while the server has it on: server off means off for everyone.
+/// `turned_on` is what the request sets to `enabled`, by protocol name.
+pub fn refusal(server: &ProtocolPolicy, turned_on: &[&str]) -> Option<String> {
+    let blocked: Vec<&str> = turned_on
+        .iter()
+        .copied()
+        .filter(|protocol| server.is_off(protocol))
+        .collect();
+    (!blocked.is_empty()).then(|| {
+        format!(
+            "{} off for the whole server (inbuxa:ProtocolPolicy), so {} can't be turned \
+             back on for one organization.",
+            names(&blocked),
+            if blocked.len() == 1 { "it" } else { "they" }
+        )
+    })
+}
+
+/// Protocol names as people read them: "IMAP and POP3 are", "POP3 is".
+fn names(protocols: &[&str]) -> String {
+    let named: Vec<&str> = protocols
+        .iter()
+        .map(|p| match *p {
+            "imap" => "IMAP",
+            "pop3" => "POP3",
+            "manageSieve" => "ManageSieve",
+            other => other,
+        })
+        .collect();
+    let list = match named.as_slice() {
+        [one] => one.to_string(),
+        [rest @ .., last] => format!("{} and {last}", rest.join(", ")),
+        [] => String::new(),
+    };
+    format!("{list} {}", if named.len() == 1 { "is" } else { "are" })
+}
+
+/// Whose switch turns a protocol off, if any.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum OffBy {
+    Server,
+    Tenant,
+}
+
+/// Whether this protocol is off for an account or domain, and by whose
+/// switch: the server's first (LP-6), then the tenant's (LP-10). Submission
+/// is off when all three protocols are, counting both switches together.
+pub fn off_by(
+    server: &ProtocolPolicy,
+    tenant: Option<&TenantProtocolPolicy>,
+    protocol: &str,
+) -> Option<OffBy> {
+    if server.is_off(protocol) {
+        return Some(OffBy::Server);
+    }
+    let tenant = tenant?;
+    let off = if protocol == SUBMISSION {
+        SWITCHED
+            .iter()
+            .all(|p| server.is_off(p) || tenant.is_off(p))
+    } else {
+        tenant.is_off(protocol)
+    };
+    off.then_some(OffBy::Tenant)
 }
 
 fn key(tenant_id: u32) -> ValueClass {
@@ -115,6 +189,15 @@ mod tests {
         }
     }
 
+    fn tenant_off(protocols: &[&str]) -> TenantProtocolPolicy {
+        let mut policy = TenantProtocolPolicy::default();
+        for p in protocols {
+            policy.set(p, LegacyProtocols::Disabled);
+        }
+        policy.normalize();
+        policy
+    }
+
     #[test]
     fn a_tenant_starts_with_legacy_protocols_on() {
         assert!(
@@ -127,20 +210,59 @@ mod tests {
     #[test]
     fn a_tenant_can_always_turn_them_off() {
         for s in [LegacyProtocols::Enabled, LegacyProtocols::Disabled] {
-            assert_eq!(refusal(&server(s), LegacyProtocols::Disabled), None);
+            assert_eq!(refusal(&server(s), &[]), None);
         }
     }
 
     #[test]
     fn a_tenant_can_turn_them_on_only_while_the_server_has_them_on() {
         // LP-9, acceptance test 9.
-        assert_eq!(
-            refusal(&server(LegacyProtocols::Enabled), LegacyProtocols::Enabled),
-            None
-        );
-        let why =
-            refusal(&server(LegacyProtocols::Disabled), LegacyProtocols::Enabled).expect("refused");
+        assert_eq!(refusal(&server(LegacyProtocols::Enabled), SWITCHED), None);
+        let why = refusal(&server(LegacyProtocols::Disabled), SWITCHED).expect("refused");
         assert!(why.contains("inbuxa:ProtocolPolicy"), "{why}");
+        assert!(
+            why.starts_with("IMAP, POP3 and ManageSieve are off"),
+            "{why}"
+        );
+    }
+
+    #[test]
+    fn a_tenant_can_turn_on_what_the_server_allows() {
+        // The server has only POP3 off: IMAP may come back, POP3 may not.
+        let mut s = ProtocolPolicy::default();
+        s.set("pop3", LegacyProtocols::Disabled);
+        assert_eq!(refusal(&s, &["imap"]), None);
+        let why = refusal(&s, &["imap", "pop3"]).expect("refused");
+        assert!(why.starts_with("POP3 is off"), "{why}");
+    }
+
+    #[test]
+    fn whose_switch_turns_a_protocol_off() {
+        let mut s = ProtocolPolicy::default();
+        s.set("pop3", LegacyProtocols::Disabled);
+        let t = tenant_off(&["imap"]);
+        assert_eq!(off_by(&s, Some(&t), "pop3"), Some(OffBy::Server));
+        assert_eq!(off_by(&s, Some(&t), "imap"), Some(OffBy::Tenant));
+        assert_eq!(off_by(&s, Some(&t), "manageSieve"), None);
+        assert_eq!(off_by(&s, None, "imap"), None);
+        // Sending goes on while any protocol is still allowed.
+        assert_eq!(off_by(&s, Some(&t), SUBMISSION), None);
+        // Between them, all three off: submission follows (LP-6, LP-10).
+        let t = tenant_off(&["imap", "manageSieve"]);
+        assert_eq!(off_by(&s, Some(&t), SUBMISSION), Some(OffBy::Tenant));
+        assert_eq!(
+            off_by(&server(LegacyProtocols::Disabled), None, SUBMISSION),
+            Some(OffBy::Server)
+        );
+    }
+
+    #[test]
+    fn an_old_tenant_policy_reads_as_all_three() {
+        let Json(old) = Json::deserialize(br#"{"legacyProtocols":"disabled"}"#).unwrap();
+        for p in SWITCHED {
+            assert!(old.is_off(p), "{p}");
+        }
+        assert!(old.is_off(SUBMISSION));
     }
 
     #[test]
@@ -158,6 +280,7 @@ mod tests {
             legacy_protocols: LegacyProtocols::Disabled,
             changed_at: Some(1),
             changed_by: Some("b".into()),
+            ..Default::default()
         };
         let Json(back) = Json::deserialize(&serde_json::to_vec(&policy).unwrap()).unwrap();
         assert_eq!(back, policy);