DLP phase 3: hold for review
ci / fork-checks (pull_request) Successful in 33s
ci / build (pull_request) Successful in 9m58s

The hold action now holds (dlp-and-mail-flow-rules spec, §2.6), where
until now it blocked.

- At DATA a hold decision queues the message with its release a century
  off (the queue's future-release mechanism, so the stored format is
  unchanged and an older node just never sends it), transport rules
  still applied, and replies 250 Held for review. A review record under
  R/h + queue id keeps the sender, recipients, subject, size, rules and
  detector counts. The sender is told when the rule asks.
- smtp/queue/held.rs: release (each recipient due now, its next notice
  as far off as it was, its lifetime counted from the release), reject
  (removed from the queue, the sender told, with the reviewer's note),
  and expiry: the daily clean-up rejects what nobody reviewed in 7 days,
  recorded as the server's doing.
- inbuxa:HeldMessage get/set: the review queue, sysDlpReviewGet to list
  and read (preview, 64 KB of text, only when asked for and recorded as
  blobAccess), sysDlpReviewUpdate to release or reject, a reason
  required and audited by the request layer; no create or destroy;
  server-level only.
- Guards: Emails > Queue refuses to change or delete held mail; the
  sender can't unsend it.
- Privacy catalog entry for inbuxa:HeldMessage; spec §2.6 as built.

Tests: mail_rules_tests gains the whole flow (held and listed with
counts, sender notified and nothing delivered, queue and unsend
refused, preview recorded, reject needs a reason and tells the sender
the note, release delivers, expiry returns it, decisions audited with
reasons). smtp inbound, system_tests (after one BlobNotFound in
antispam, the known flake, then clean), features and common unit tests.
This commit is contained in:
2026-09-28 18:33:12 -07:00
parent dd73e0ad74
commit f44382fb09
25 changed files with 1561 additions and 57 deletions
+11
View File
@@ -106,6 +106,8 @@ impl JmapAuthorization for AccessToken {
// inbuxa: DLP and mail flow rules share an object; either
// permission reaches it, and the handler shows each kind
// only to those who may see it
// inbuxa: mail held for review (§2.8)
GetRequestMethod::HeldMessage(_) => Permission::SysDlpReviewGet,
GetRequestMethod::MailRule(_) => {
if self.has_permission(Permission::SysMailRuleGet) {
Permission::SysMailRuleGet
@@ -257,6 +259,14 @@ impl JmapAuthorization for AccessToken {
Permission::SysLegalHoldUpdate,
Permission::SysLegalHoldUpdate,
),
// inbuxa: releasing or rejecting held mail (§2.8)
SetRequestMethod::HeldMessage(s) => validate_set(
s,
self,
Permission::SysDlpReviewUpdate,
Permission::SysDlpReviewUpdate,
Permission::SysDlpReviewUpdate,
),
// inbuxa: DLP and mail flow rules: either change
// permission gets in; the handler checks each rule's kind
SetRequestMethod::MailRule(_) => {
@@ -431,6 +441,7 @@ impl JmapAuthorization for AccessToken {
| MethodObject::LegalHold
| MethodObject::HoldExport
| MethodObject::MailRule
| MethodObject::HeldMessage
| MethodObject::ProtocolPolicy
| MethodObject::TenantProtocolPolicy => Permission::JmapEmailChanges,
// inbuxa: x:MaskedEmail/changes reads what /get reads
+24
View File
@@ -282,6 +282,9 @@ impl RequestHandler for Server {
SetResponseMethod::MailRule(set_response) => {
set_response.update_created_ids(&mut response);
}
SetResponseMethod::HeldMessage(set_response) => {
set_response.update_created_ids(&mut response);
}
SetResponseMethod::HoldExport(set_response) => {
set_response.update_created_ids(&mut response);
}
@@ -489,6 +492,11 @@ impl RequestHandler for Server {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
crate::inbuxa::legal_hold::get(self, *req).await?.into()
}
// inbuxa: mail held for review
GetRequestMethod::HeldMessage(mut req) => {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
crate::inbuxa::held_message::get(self, access_token, *req).await?.into()
}
// inbuxa: DLP and mail flow rules
GetRequestMethod::MailRule(mut req) => {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
@@ -918,6 +926,22 @@ impl RequestHandler for Server {
.await?
.into()
}
SetRequestMethod::HeldMessage(mut req) => {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
let reason = req.arguments.reason.clone();
crate::inbuxa::audit::recorded(
self,
access_token,
session,
&method_name.obj.to_string(),
None,
reason,
*req,
|req| Box::pin(crate::inbuxa::held_message::set(self, access_token, req)),
)
.await?
.into()
}
SetRequestMethod::MailRule(mut req) => {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
let reason = req.arguments.reason.clone();
+1
View File
@@ -430,6 +430,7 @@ impl IntermediateChangesResponse {
| MethodObject::LegalHold
| MethodObject::HoldExport
| MethodObject::MailRule
| MethodObject::HeldMessage
| MethodObject::ProtocolPolicy
| MethodObject::TenantProtocolPolicy
| MethodObject::Registry(_) => unreachable!(),
+325
View File
@@ -0,0 +1,325 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! `inbuxa:HeldMessage` (dlp-and-mail-flow-rules spec, §2.6, §2.8): the
//! review queue. `sysDlpReviewGet` lists held mail and reads it;
//! `sysDlpReviewUpdate` releases or rejects it, with a reason the request
//! layer records. Reading a held message's text is recorded as access to
//! the sender's mail. Nobody in a tenant reaches this (settled answer 3).
use common::{Server, auth::AccessToken, config::smtp::queue::QueueName};
use inbuxa_features::{
audit::{Action, Outcome, Record, Target},
mailflow::held::{self, Held},
};
use jmap_proto::{
error::set::SetError,
method::{
get::{GetRequest, GetResponse},
set::{SetRequest, SetResponse},
},
object::inbuxa_held_message::{
HeldMessage, HeldMessageProperty as P, HeldMessageSetArguments, HeldMessageValue,
},
request::IntoValid,
types::date::UTCDate,
};
use jmap_tools::{Key, Map, Value};
use mail_parser::{MessageParser, MimeHeaders, PartType};
use smtp::queue::spool::SmtpSpool;
use std::borrow::Cow;
use types::id::Id;
type HValue = Value<'static, P, HeldMessageValue>;
const ALL: &[P] = &[
P::Id,
P::Sender,
P::Recipients,
P::Subject,
P::Size,
P::Rules,
P::Counts,
P::HeldAt,
P::ExpiresAt,
];
/// How much of a held message's text a preview shows.
const PREVIEW_LIMIT: usize = 64 * 1024;
fn server_level(access_token: &AccessToken) -> trc::Result<()> {
if access_token.tenant_id().is_some() {
Err(trc::JmapEvent::Forbidden
.into_err()
.details("Held mail is the server's to review."))
} else {
Ok(())
}
}
fn date(seconds: u64) -> HValue {
Value::Str(UTCDate::from_timestamp(seconds as i64).to_string().into())
}
fn text(s: &str) -> HValue {
Value::Str(Cow::Owned(s.to_string()))
}
/// The text a reviewer reads: the subject, each body as text, and the
/// attachments' names; at most [`PREVIEW_LIMIT`].
async fn preview(server: &Server, queue_id: u64) -> trc::Result<Option<String>> {
let Some(message) = server.read_message(queue_id, QueueName::default()).await else {
return Ok(None);
};
let Some(raw) = server
.blob_store()
.get_blob(message.message.blob_hash.as_slice(), 0..usize::MAX)
.await?
else {
return Ok(None);
};
let Some(parsed) = MessageParser::new().parse(&raw) else {
return Ok(Some(
String::from_utf8_lossy(&raw[..raw.len().min(PREVIEW_LIMIT)]).into_owned(),
));
};
let mut out = String::new();
for part in parsed.text_bodies() {
match &part.body {
PartType::Text(text) => out.push_str(text),
PartType::Html(html) => out.push_str(&mail_parser::decoders::html::html_to_text(html)),
_ => {}
}
out.push_str("\n\n");
}
let attachments: Vec<&str> = parsed
.attachments()
.filter_map(|a| a.attachment_name())
.collect();
if !attachments.is_empty() {
out.push_str(&format!("Attachments: {}\n", attachments.join(", ")));
}
if out.len() > PREVIEW_LIMIT {
let mut cut = PREVIEW_LIMIT;
while !out.is_char_boundary(cut) {
cut -= 1;
}
out.truncate(cut);
}
Ok(Some(out))
}
fn to_value(record: &Held, properties: &[P], preview: Option<&str>) -> HValue {
let mut out = Map::with_capacity(properties.len());
for property in properties {
let value = match property {
P::Id => Value::Element(HeldMessageValue::Id(Id::from(record.queue_id))),
P::Sender => text(&record.sender),
P::Recipients => Value::Array(record.recipients.iter().map(|r| text(r)).collect()),
P::Subject => text(&record.subject),
P::Size => Value::Number(record.size.into()),
P::Rules => Value::Array(
record
.rules
.iter()
.map(|rule| {
let mut map = Map::with_capacity(2);
map.insert_unchecked(Key::Borrowed("name"), text(&rule.name));
map.insert_unchecked(Key::Borrowed("notice"), text(&rule.notice));
Value::Object(map)
})
.collect(),
),
P::Counts => Value::Array(
record
.counts
.iter()
.map(|(detector, count)| {
let mut map = Map::with_capacity(2);
map.insert_unchecked(Key::Borrowed("detector"), text(detector));
map.insert_unchecked(
Key::Borrowed("count"),
Value::Number((*count as u64).into()),
);
Value::Object(map)
})
.collect(),
),
P::HeldAt => date(record.held_at),
P::ExpiresAt => date(record.expires_at),
P::Preview => preview.map_or(Value::Null, text),
P::Decision | P::Note => Value::Null,
};
out.insert_unchecked(Key::Property(property.clone()), value);
}
Value::Object(out)
}
/// `inbuxa:HeldMessage/get`: held mail, oldest first.
pub async fn get(
server: &Server,
access_token: &AccessToken,
mut request: GetRequest<HeldMessage>,
) -> trc::Result<GetResponse<HeldMessage>> {
server_level(access_token)?;
let properties = request.unwrap_properties(ALL);
let (ids, not_found) = request.unwrap_ids(server.core.jmap.get_max_objects)?;
let mut response = GetResponse {
account_id: request.account_id.into(),
state: None,
list: Vec::new(),
not_found,
};
let all = held::all(server.store()).await?;
let wanted: Vec<&Held> = match &ids {
None => all.iter().collect(),
Some(ids) => {
let mut found = Vec::new();
for id in ids {
match all.iter().find(|h| h.queue_id == id.id()) {
Some(record) => found.push(record),
None => response.push_not_found(*id),
}
}
found
}
};
let with_preview = properties.contains(&P::Preview);
for record in wanted {
let text = if with_preview {
let text = preview(server, record.queue_id).await?;
// Reading someone's mail is recorded, as any access is
server
.audit_note(Record {
at: store::write::now() * 1000,
actor: server.audit_actor(access_token).await,
via: access_token.origin().cloned(),
remote_ip: None,
action: Action::BlobAccess,
target: Target {
kind: "inbuxa:HeldMessage".into(),
id: Some(Id::from(record.queue_id).to_string()),
name: Some(record.subject.clone()),
account_id: record.account_id,
tenant_id: record.tenant_id,
},
changes: vec![],
details: Some(format!(
"Read a message held for review, from {}",
record.sender
)),
reason: None,
outcome: Outcome::success(),
})
.await;
text
} else {
None
};
response
.list
.push(to_value(record, &properties, text.as_deref()));
}
Ok(response)
}
fn invalid(property: P, why: &str) -> SetError<P> {
SetError::invalid_properties()
.with_property(property)
.with_description(why.to_string())
}
/// `inbuxa:HeldMessage/set`: update with `decision` release or reject (and
/// an optional `note` for the sender). There is no create or destroy.
pub async fn set(
server: &Server,
access_token: &AccessToken,
mut request: SetRequest<'_, HeldMessage>,
) -> trc::Result<SetResponse<HeldMessage>> {
server_level(access_token)?;
let mut response = SetResponse::from_request(&request, server.core.jmap.set_max_objects)?;
let arguments: HeldMessageSetArguments = std::mem::take(&mut request.arguments);
let has_reason = arguments
.reason
.as_deref()
.is_some_and(|r| !r.trim().is_empty());
for (client_id, _) in request.unwrap_create() {
response.not_created.append(
client_id,
SetError::forbidden().with_description("Mail is held by DLP rules, not created."),
);
}
'update: for (id, value) in request.unwrap_update().into_valid() {
let Some(record) = held::get(server.store(), id.id()).await? else {
response.not_updated.append(id, SetError::not_found());
continue;
};
if !has_reason {
response.not_updated.append(
id,
SetError::invalid_properties().with_description(
"Say why: a reason is required and is kept in the audit log.",
),
);
continue;
}
let mut decision = None;
let mut note = None;
for (key, value) in value.into_expanded_object() {
match (&key, value) {
(Key::Property(P::Decision), Value::Str(s)) if s == "release" || s == "reject" => {
decision = Some(s.to_string());
}
(Key::Property(P::Note), Value::Str(s)) => {
let s = s.trim();
if !s.is_empty() {
note = Some(s.chars().take(1000).collect::<String>());
}
}
(Key::Property(P::Note), Value::Null) => {}
_ => {
response.not_updated.append(
id,
invalid(
P::Decision,
"Send decision: \"release\" or \"reject\", and an optional note.",
),
);
continue 'update;
}
}
}
let done = match decision.as_deref() {
Some("release") => smtp::queue::held::release(server, record.queue_id).await?,
Some("reject") => smtp::queue::held::reject(server, &record, note.as_deref()).await?,
_ => {
response
.not_updated
.append(id, invalid(P::Decision, "Say release or reject."));
continue;
}
};
if done {
response.updated.append(id, None);
} else {
response.not_updated.append(
id,
SetError::not_found().with_description("The message is no longer in the queue."),
);
}
}
for id in request.unwrap_destroy().into_valid() {
response.not_destroyed.append(
id,
SetError::forbidden().with_description("Release or reject it instead."),
);
}
Ok(response)
}
+1
View File
@@ -11,6 +11,7 @@ pub mod access;
pub mod account_lock;
pub mod legal_hold;
pub mod mail_rule;
pub mod held_message;
pub mod hold_export;
pub mod hold_export_api;
pub mod audit;
@@ -49,6 +49,12 @@ use trc::AddContext;
use types::{blob::BlobId, blob_hash::BlobHash, id::Id};
use utils::map::vec_map::VecMap;
/// inbuxa: held mail is the review queue's to decide.
fn held_refusal() -> SetError<Property> {
SetError::forbidden()
.with_description("This message is held for review: release or reject it under Compliance, Held mail.")
}
pub(crate) async fn queued_message_set(
mut set: RegistrySetResponse<'_>,
) -> trc::Result<RegistrySetResponse<'_>> {
@@ -66,6 +72,12 @@ pub(crate) async fn queued_message_set(
let mut refresh_queue = false;
'outer: for (id, value) in set.update.drain(..) {
let queue_id = id.id();
// inbuxa: held mail is released or rejected by review, not here
// (dlp-and-mail-flow-rules spec, §2.6)
if inbuxa_features::mailflow::held::is_held(set.server.store(), queue_id).await? {
set.response.not_updated.append(id, held_refusal());
continue;
}
let Some(archive) = set.server.read_message_archive(queue_id).await? else {
set.response.not_updated.append(id, SetError::not_found());
continue;
@@ -238,6 +250,11 @@ pub(crate) async fn queued_message_set(
// Process destroy operations
for id in set.destroy.drain(..) {
// inbuxa: §2.6, as above
if inbuxa_features::mailflow::held::is_held(set.server.store(), id.id()).await? {
set.response.not_destroyed.append(id, held_refusal());
continue;
}
let Some(message) = set.server.read_message(id.id(), QueueName::default()).await else {
set.response.not_destroyed.append(id, SetError::not_found());
continue;
+11
View File
@@ -214,6 +214,17 @@ impl EmailSubmissionSet for Server {
}
match undo_status {
// inbuxa: held for review: the review decides, not an unsend
// (dlp-and-mail-flow-rules spec, §2.6)
Some(email_submission::UndoStatus::Canceled)
if inbuxa_features::mailflow::held::is_held(self.store(), queue_id).await? =>
{
response.not_updated.append(
id,
SetError::new(SetErrorType::CannotUnsend)
.with_description("The message is held for review and can't be unsent."),
);
}
Some(email_submission::UndoStatus::Canceled) => {
if let Some(queue_message) =
self.read_message(queue_id, QueueName::default()).await