Compare commits

..
Author SHA1 Message Date
jcoffey-dev db817dd507 Merge pull request 'Release 2026.9.28.2' (#77) from release-2026.9.28.2 into main
publish / version (push) Successful in 31s
ci / fork-checks (push) Successful in 52s
ci / build (push) Canceled after 9m20s
publish / publish-amd64 (push) Successful in 34m25s
publish / release (push) Successful in 45s
publish / publish-arm64 (push) Successful in 36m5s
publish / binaries (push) Successful in 34s
publish / announce (push) Successful in 22s
2026-09-28 05:59:59 +00:00
jcoffey-dev b7e3a765ca Release 2026.9.28.2
ci / fork-checks (pull_request) Successful in 56s
ci / build (pull_request) Successful in 5m22s
2026-09-27 22:54:18 -07:00
jcoffey-dev 7ba9ec9fa0 Merge pull request 'Send "none" instead of "pass" as the DMARC report disposition' (#76) from fix/dmarc-disposition-compat into main
ci / build (push) Canceled after 11m35s
ci / fork-checks (push) Successful in 15s
2026-09-28 05:48:22 +00:00
jcoffey-dev 4c07779c16 Merge pull request 'Recheck DNSSEC lookups that hickory wrongly calls bogus' (#72) from fix/dnssec-insecure-fallback into main
ci / fork-checks (push) Successful in 2m2s
ci / build (push) Canceled after 14m59s
2026-09-28 05:33:21 +00:00
jcoffey-dev e1e8a9aeb0 Send "none" instead of "pass" as the DMARC report disposition
ci / build (pull_request) Successful in 16m34s
ci / fork-checks (pull_request) Successful in 52s
Cloudflare's DMARC report intake rejects every aggregate report we
send with "555 5.7.1 invalid_report_schema". Bisected against the live
endpoint: the only element it objects to is <disposition>pass</disposition>,
the value RFC 9990 added for mail that passed DMARC under an enforcing
policy. The RFC 9990 namespace, <np>, <discovery_method>, <testing> and
a missing <pct> are all accepted, and a report that differs only in
using "none" there goes through.

"none" (no action taken) is valid under both RFC 9990 and RFC 7489 and
says the same thing to the reader, so reports now go out with it. The
stored report keeps "pass"; only the serialized copy changes.
2026-09-27 22:31:13 -07:00
jcoffey-dev 3978cf5785 Merge pull request 'Release 2026.9.28.1' (#74) from release-2026.9.28.1 into main
ci / build (push) Canceled after 29m44s
ci / fork-checks (push) Successful in 14s
publish / version (push) Successful in 32s
publish / publish-amd64 (push) Successful in 28m55s
publish / release (push) Successful in 15s
publish / publish-arm64 (push) Successful in 1h2m48s
publish / binaries (push) Successful in 51s
publish / announce (push) Successful in 23s
2026-09-28 05:03:38 +00:00
jcoffey-dev 815a642cc4 Release 2026.9.28.1
ci / fork-checks (pull_request) Successful in 16s
ci / build (pull_request) Successful in 7m29s
Legal hold exports (LH-12, #73). The prepared Explain answers are
relabeled for this release; 706 carry over unchanged.
2026-09-27 21:55:55 -07:00
jcoffey-dev 1d5f4a2cd3 Merge pull request 'Export what a legal hold keeps as a ZIP (LH-12)' (#73) from feature/hold-export into main
ci / fork-checks (push) Successful in 50s
ci / build (push) Canceled after 12m21s
2026-09-28 04:51:16 +00:00
jcoffey-dev 68dd749291 Export what a legal hold keeps as a ZIP (LH-12)
ci / build (pull_request) Successful in 4m47s
ci / fork-checks (pull_request) Successful in 14s
inbuxa:HoldExport/set takes a hold, optionally some of the accounts it
covers, and a reason; the collection runs in the background and get
says when it's ready. The ZIP has, per account, mail as .eml under its
folders, calendars as .ics, contacts as .vcf, files as stored, and the
archived items the hold keeps under archived/; a manifest.csv gives each
entry's account, kind, folder, date, whether it was archived, size and
SHA-256, and manifest.sha256 hashes the manifest. Accounts the hold
doesn't cover are left out, and items outside its date range are too:
live mail by arrival, events by start, and archived items the same way,
so an export doesn't carry deleted items that only another hold keeps.

The finished file is a blob of whoever started the export, so only they
download it, and it lasts as long as any upload (uploadTtl). Exports
are records under the hold (SUBSPACE_INBUXA H/e): never changed or
destroyed, each with its status, counts, size and checksum. Starting
one needs sysLegalHoldExport, an active hold and a reason, and is
recorded in the audit log like the audit log's own export.

The build is in memory and capped at 2 GB; bigger holds fail with a
message saying so, and are split by picking accounts.

Tested: unit tests for safe ZIP names and the manifest and its hash;
the legal_hold system test, on RocksDB, PostgreSQL and MySQL, exports a
hold end to end (live and archived mail, the manifest's hash, an asked-
for account the hold doesn't cover left out) and checks the refusals
(no reason, a user without the permission, a released hold) and the
audit record; and by hand from the console on a local server. Not
covered by a test: the archived-item date range with two holds of
different ranges over one account.
2026-09-27 21:45:52 -07:00
jcoffey-dev b1bc5ed6e0 Recheck DNSSEC lookups that hickory wrongly calls bogus
ci / fork-checks (pull_request) Successful in 14s
ci / build (pull_request) Successful in 7m34s
hickory 0.26.3 rejects two kinds of valid answers, and outbound
delivery then retries those hosts until the message expires:

- A zone delegated beneath an unsigned zone (l.google.com under
  google.com). Proving the delegation insecure needs an SOA record in
  the DS reply, and public resolvers often leave it out. Every Google
  MX host behind a signed MX record was unreachable.
- A signed CNAME to a signed name that lacks the queried type. The
  NSEC denial is checked against the original name, not the target's.

On a bogus verdict, follow a signed CNAME and repeat the lookup at its
target; otherwise look up the name's zone and its parents, nearest
first. A zone that validates as unsigned means nothing below it can be
signed, so the plain resolver answers and the result is insecure. A
zone that validates as signed first leaves the verdict standing.
2026-09-27 21:45:13 -07:00
24 changed files with 1586 additions and 36 deletions
Generated
+2
View File
@@ -4258,6 +4258,7 @@ dependencies = [
"tungstenite 0.30.0",
"types",
"utils",
"zip",
]
[[package]]
@@ -8624,6 +8625,7 @@ dependencies = [
"types",
"utils",
"x509-parser",
"zip",
]
[[package]]
+103
View File
@@ -108,6 +108,45 @@ impl Keeping {
const FEATURE: u8 = b'H';
const KIND_HOLD: u8 = b'h';
const KIND_ORIGINAL: u8 = b'o';
const KIND_EXPORT: u8 = b'e';
/// How far a hold export has got (LH-12).
#[derive(Debug, Clone, Copy, PartialEq, Eq, SerdeSerialize, SerdeDeserialize)]
#[serde(rename_all = "camelCase")]
pub enum ExportStatus {
Running,
Ready,
Failed,
}
/// A collection of what a hold keeps, as a ZIP (LH-12).
#[derive(Debug, Clone, PartialEq, Eq, SerdeSerialize, SerdeDeserialize)]
#[serde(rename_all = "camelCase")]
pub struct Export {
pub id: u32,
pub hold_id: u32,
/// The accounts asked for; empty for every account the hold covers.
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub accounts: Vec<u32>,
pub reason: String,
pub created_at: u64,
pub created_by: String,
/// Whose blob the ZIP is, so only they download it.
pub created_by_id: u32,
pub status: ExportStatus,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub finished_at: Option<u64>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub blob_id: Option<String>,
#[serde(default)]
pub size: u64,
#[serde(default)]
pub items: u64,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub sha256: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub error: Option<String>,
}
/// How many times creating a hold retries when another node took its id.
const CREATE_ATTEMPTS: usize = 5;
@@ -402,6 +441,70 @@ pub async fn set_original_deadline(data: &Store, item_id: u64, until: Option<u64
.map(|_| ())
}
fn export_class(id: u32) -> ValueClass {
let mut key = Vec::with_capacity(6);
key.push(FEATURE);
key.push(KIND_EXPORT);
key.extend_from_slice(&id.to_be_bytes());
ValueClass::Any(AnyClass {
subspace: SUBSPACE_INBUXA,
key,
})
}
/// Every hold export, oldest first.
pub async fn exports(data: &Store) -> trc::Result<Vec<Export>> {
let mut exports = Vec::new();
data.iterate(
IterateParams::new(ValueKey::from(export_class(0)), ValueKey::from(export_class(u32::MAX))),
|_, value| {
if let Ok(Json(export)) = Json::<Export>::deserialize(value) {
exports.push(export);
}
Ok(true)
},
)
.await
.caused_by(trc::location!())?;
Ok(exports)
}
/// Writes a new export under the next free id, which it returns.
pub async fn create_export(data: &Store, export: &Export) -> trc::Result<u32> {
let mut attempt = 0;
loop {
attempt += 1;
let id = exports(data).await?.iter().map(|e| e.id).max().unwrap_or(0) + 1;
let stored = Export {
id,
..export.clone()
};
let mut batch = BatchBuilder::new();
batch.assert_value(export_class(id), AssertValue::None);
batch.set(export_class(id), Json(&stored).serialize()?);
match data.write(batch.build_all()).await {
Ok(_) => return Ok(id),
Err(err)
if attempt < CREATE_ATTEMPTS
&& matches!(
err.as_ref(),
trc::EventType::Store(trc::StoreEvent::AssertValueFailed)
) => {}
Err(err) => return Err(err.caused_by(trc::location!())),
}
}
}
/// Saves an export's progress.
pub async fn update_export(data: &Store, export: &Export) -> trc::Result<()> {
let mut batch = BatchBuilder::new();
batch.set(export_class(export.id), Json(export).serialize()?);
data.write(batch.build_all())
.await
.caused_by(trc::location!())
.map(|_| ())
}
/// One hold, released or not.
pub async fn get(data: &Store, id: u32) -> trc::Result<Option<Hold>> {
Ok(data
@@ -0,0 +1,211 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! `inbuxa:HoldExport/get` and `/set` under `urn:inbuxa:jmap`: collecting
//! what a legal hold keeps as a ZIP (audit-hold-lock spec, LH-12). Creating
//! one starts it; it runs in the background, and `get` says when it's ready
//! and which blob to download. The set call's `reason` says why (AU-12).
use crate::{
object::{AnyId, JmapObject, JmapObjectId},
request::deserialize::DeserializeArguments,
};
use jmap_tools::{Element, Key, Property};
use std::{borrow::Cow, str::FromStr};
use types::id::Id;
#[derive(Debug, Clone, Default)]
pub struct HoldExport;
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum HoldExportProperty {
Id,
HoldId,
AccountIds,
Reason,
Status,
CreatedAt,
CreatedBy,
FinishedAt,
BlobId,
Size,
Items,
Sha256,
Error,
}
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum HoldExportValue {
Id(Id),
}
impl Property for HoldExportProperty {
fn try_parse(parent: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {
match parent {
None => HoldExportProperty::parse(value),
Some(_) => None,
}
}
fn to_cow(&self) -> Cow<'static, str> {
match self {
HoldExportProperty::Id => "id",
HoldExportProperty::HoldId => "holdId",
HoldExportProperty::AccountIds => "accountIds",
HoldExportProperty::Reason => "reason",
HoldExportProperty::Status => "status",
HoldExportProperty::CreatedAt => "createdAt",
HoldExportProperty::CreatedBy => "createdBy",
HoldExportProperty::FinishedAt => "finishedAt",
HoldExportProperty::BlobId => "blobId",
HoldExportProperty::Size => "size",
HoldExportProperty::Items => "items",
HoldExportProperty::Sha256 => "sha256",
HoldExportProperty::Error => "error",
}
.into()
}
}
impl HoldExportProperty {
fn parse(value: &str) -> Option<Self> {
hashify::tiny_map!(value.as_bytes(),
b"id" => HoldExportProperty::Id,
b"holdId" => HoldExportProperty::HoldId,
b"accountIds" => HoldExportProperty::AccountIds,
b"reason" => HoldExportProperty::Reason,
b"status" => HoldExportProperty::Status,
b"createdAt" => HoldExportProperty::CreatedAt,
b"createdBy" => HoldExportProperty::CreatedBy,
b"finishedAt" => HoldExportProperty::FinishedAt,
b"blobId" => HoldExportProperty::BlobId,
b"size" => HoldExportProperty::Size,
b"items" => HoldExportProperty::Items,
b"sha256" => HoldExportProperty::Sha256,
b"error" => HoldExportProperty::Error,
)
}
}
impl FromStr for HoldExportProperty {
type Err = ();
fn from_str(s: &str) -> Result<Self, Self::Err> {
HoldExportProperty::parse(s).ok_or(())
}
}
impl Element for HoldExportValue {
type Property = HoldExportProperty;
fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {
match key {
Key::Property(HoldExportProperty::Id) => Id::from_str(value).ok().map(HoldExportValue::Id),
_ => None,
}
}
fn to_cow(&self) -> Cow<'static, str> {
match self {
HoldExportValue::Id(id) => id.to_string().into(),
}
}
}
/// The set call's own arguments: why (AU-12).
#[derive(Debug, Clone, Default)]
pub struct HoldExportSetArguments {
pub reason: Option<String>,
}
impl<'de> DeserializeArguments<'de> for HoldExportSetArguments {
fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>
where
A: serde::de::MapAccess<'de>,
{
if key == "reason" {
self.reason = map.next_value()?;
} else {
let _ = map.next_value::<serde::de::IgnoredAny>()?;
}
Ok(())
}
}
impl JmapObject for HoldExport {
type Property = HoldExportProperty;
type Element = HoldExportValue;
type Id = Id;
type Filter = ();
type Comparator = ();
type GetArguments = ();
type SetArguments<'de> = HoldExportSetArguments;
type QueryArguments = ();
type CopyArguments = ();
type ParseArguments = ();
const ID_PROPERTY: Self::Property = HoldExportProperty::Id;
}
impl From<Id> for HoldExportValue {
fn from(id: Id) -> Self {
HoldExportValue::Id(id)
}
}
impl JmapObjectId for HoldExportValue {
fn as_id(&self) -> Option<Id> {
match self {
HoldExportValue::Id(id) => Some(*id),
}
}
fn as_any_id(&self) -> Option<AnyId> {
match self {
HoldExportValue::Id(id) => Some(AnyId::Id(*id)),
}
}
fn as_id_ref(&self) -> Option<&str> {
None
}
fn try_set_id(&mut self, new_id: AnyId) -> bool {
if let AnyId::Id(id) = new_id {
*self = HoldExportValue::Id(id);
true
} else {
false
}
}
}
impl JmapObjectId for HoldExportProperty {
fn as_id(&self) -> Option<Id> {
None
}
fn as_any_id(&self) -> Option<AnyId> {
None
}
fn as_id_ref(&self) -> Option<&str> {
None
}
fn try_set_id(&mut self, _: AnyId) -> bool {
false
}
}
+1
View File
@@ -25,6 +25,7 @@ pub mod inbuxa_account_lock; // inbuxa: account lock with delegation
pub mod inbuxa_ai_limits; // inbuxa: AI spam classification
pub mod inbuxa_audit; // inbuxa: the audit log
pub mod inbuxa_legal_hold; // inbuxa: legal hold
pub mod inbuxa_hold_export; // inbuxa: legal hold exports
pub mod inbuxa_explanation; // inbuxa: "Explain this" with the local model
pub mod inbuxa_protocol_policy; // inbuxa: legacy protocols off
pub mod inbuxa_tenant_protocol_policy; // inbuxa: legacy protocols off, per tenant
+3
View File
@@ -73,6 +73,9 @@ impl Response<'_> {
GetResponseMethod::LegalHold(response) => {
response.eval_jptr(path, &mut results)
}
GetResponseMethod::HoldExport(response) => {
response.eval_jptr(path, &mut results)
}
GetResponseMethod::ProtocolPolicy(response) => {
response.eval_jptr(path, &mut results)
}
@@ -50,6 +50,7 @@ impl Response<'_> {
GetRequestMethod::AuditSettings(request) => request.resolve_references(self)?,
GetRequestMethod::AccountLock(request) => request.resolve_references(self)?,
GetRequestMethod::LegalHold(request) => request.resolve_references(self)?,
GetRequestMethod::HoldExport(request) => request.resolve_references(self)?,
GetRequestMethod::ProtocolPolicy(request) => request.resolve_references(self)?,
GetRequestMethod::TenantProtocolPolicy(request) => {
request.resolve_references(self)?
@@ -115,6 +116,9 @@ impl Response<'_> {
SetRequestMethod::LegalHold(request) => {
request.resolve_references(self, 1, false)?
}
SetRequestMethod::HoldExport(request) => {
request.resolve_references(self, 1, false)?
}
SetRequestMethod::ProtocolPolicy(request) => {
request.resolve_references(self, 1, false)?
}
+8 -1
View File
@@ -60,6 +60,7 @@ pub enum MethodObject {
AccountLock,
// inbuxa: legal hold
LegalHold,
HoldExport,
ProtocolPolicy,
TenantProtocolPolicy,
}
@@ -94,7 +95,8 @@ impl MethodObject {
| MethodObject::AuditExport
| MethodObject::AuditVerification
| MethodObject::AccountLock
| MethodObject::LegalHold => Capability::Inbuxa,
| MethodObject::LegalHold
| MethodObject::HoldExport => Capability::Inbuxa,
MethodObject::ProtocolPolicy => Capability::Inbuxa,
MethodObject::TenantProtocolPolicy => Capability::Inbuxa,
}
@@ -284,6 +286,8 @@ impl MethodName {
(MethodFunction::Set, MethodObject::AccountLock) => "inbuxa:AccountLock/set",
(MethodFunction::Get, MethodObject::LegalHold) => "inbuxa:LegalHold/get",
(MethodFunction::Set, MethodObject::LegalHold) => "inbuxa:LegalHold/set",
(MethodFunction::Get, MethodObject::HoldExport) => "inbuxa:HoldExport/get",
(MethodFunction::Set, MethodObject::HoldExport) => "inbuxa:HoldExport/set",
(MethodFunction::Set, MethodObject::AuditVerification) => {
"inbuxa:AuditVerification/set"
}
@@ -430,6 +434,8 @@ impl MethodName {
"inbuxa:AccountLock/set" => (MethodObject::AccountLock, MethodFunction::Set),
"inbuxa:LegalHold/get" => (MethodObject::LegalHold, MethodFunction::Get),
"inbuxa:LegalHold/set" => (MethodObject::LegalHold, MethodFunction::Set),
"inbuxa:HoldExport/get" => (MethodObject::HoldExport, MethodFunction::Get),
"inbuxa:HoldExport/set" => (MethodObject::HoldExport, MethodFunction::Set),
"inbuxa:AuditVerification/set" => (MethodObject::AuditVerification, MethodFunction::Set),
"inbuxa:ProtocolPolicy/get" => (MethodObject::ProtocolPolicy, MethodFunction::Get),
"inbuxa:ProtocolPolicy/set" => (MethodObject::ProtocolPolicy, MethodFunction::Set),
@@ -495,6 +501,7 @@ impl Display for MethodObject {
MethodObject::AuditVerification => "inbuxa:AuditVerification",
MethodObject::AccountLock => "inbuxa:AccountLock",
MethodObject::LegalHold => "inbuxa:LegalHold",
MethodObject::HoldExport => "inbuxa:HoldExport",
MethodObject::ProtocolPolicy => "inbuxa:ProtocolPolicy",
MethodObject::TenantProtocolPolicy => "inbuxa:TenantProtocolPolicy",
MethodObject::Registry(obj) => {
+2
View File
@@ -120,6 +120,7 @@ pub enum GetRequestMethod {
AuditSettings(Box<GetRequest<crate::object::inbuxa_audit::AuditSettings>>),
AccountLock(Box<GetRequest<crate::object::inbuxa_account_lock::AccountLock>>),
LegalHold(Box<GetRequest<crate::object::inbuxa_legal_hold::LegalHold>>),
HoldExport(Box<GetRequest<crate::object::inbuxa_hold_export::HoldExport>>),
ProtocolPolicy(Box<GetRequest<crate::object::inbuxa_protocol_policy::ProtocolPolicy>>),
TenantProtocolPolicy(
Box<GetRequest<crate::object::inbuxa_tenant_protocol_policy::TenantProtocolPolicy>>,
@@ -153,6 +154,7 @@ pub enum SetRequestMethod<'x> {
AuditVerification(Box<SetRequest<'x, crate::object::inbuxa_audit::AuditVerification>>),
AccountLock(Box<SetRequest<'x, crate::object::inbuxa_account_lock::AccountLock>>),
LegalHold(Box<SetRequest<'x, crate::object::inbuxa_legal_hold::LegalHold>>),
HoldExport(Box<SetRequest<'x, crate::object::inbuxa_hold_export::HoldExport>>),
ProtocolPolicy(Box<SetRequest<'x, crate::object::inbuxa_protocol_policy::ProtocolPolicy>>),
TenantProtocolPolicy(
Box<SetRequest<'x, crate::object::inbuxa_tenant_protocol_policy::TenantProtocolPolicy>>,
+15
View File
@@ -566,6 +566,21 @@ impl<'de> Visitor<'de> for CallVisitor {
return Err(de::Error::invalid_length(1, &self));
}
},
// inbuxa: legal hold exports
(MethodFunction::Get, MethodObject::HoldExport) => match seq.next_element() {
Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::HoldExport(value)),
Err(err) => RequestMethod::invalid(err),
Ok(None) => {
return Err(de::Error::invalid_length(1, &self));
}
},
(MethodFunction::Set, MethodObject::HoldExport) => match seq.next_element() {
Ok(Some(value)) => RequestMethod::Set(SetRequestMethod::HoldExport(value)),
Err(err) => RequestMethod::invalid(err),
Ok(None) => {
return Err(de::Error::invalid_length(1, &self));
}
},
// inbuxa: legal hold
(MethodFunction::Get, MethodObject::LegalHold) => match seq.next_element() {
Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::LegalHold(value)),
+15
View File
@@ -107,6 +107,7 @@ pub enum GetResponseMethod {
AuditSettings(GetResponse<crate::object::inbuxa_audit::AuditSettings>),
AccountLock(GetResponse<crate::object::inbuxa_account_lock::AccountLock>),
LegalHold(GetResponse<crate::object::inbuxa_legal_hold::LegalHold>),
HoldExport(GetResponse<crate::object::inbuxa_hold_export::HoldExport>),
ProtocolPolicy(GetResponse<crate::object::inbuxa_protocol_policy::ProtocolPolicy>),
TenantProtocolPolicy(
GetResponse<crate::object::inbuxa_tenant_protocol_policy::TenantProtocolPolicy>,
@@ -140,6 +141,7 @@ pub enum SetResponseMethod {
AuditVerification(Box<SetResponse<crate::object::inbuxa_audit::AuditVerification>>),
AccountLock(Box<SetResponse<crate::object::inbuxa_account_lock::AccountLock>>),
LegalHold(Box<SetResponse<crate::object::inbuxa_legal_hold::LegalHold>>),
HoldExport(Box<SetResponse<crate::object::inbuxa_hold_export::HoldExport>>),
Explanation(Box<SetResponse<crate::object::inbuxa_explanation::Explanation>>),
ProtocolPolicy(Box<SetResponse<crate::object::inbuxa_protocol_policy::ProtocolPolicy>>),
TenantProtocolPolicy(
@@ -780,3 +782,16 @@ impl<'x> From<SetResponse<crate::object::inbuxa_legal_hold::LegalHold>> for Resp
ResponseMethod::Set(SetResponseMethod::LegalHold(Box::new(value)))
}
}
// inbuxa: legal hold exports
impl<'x> From<GetResponse<crate::object::inbuxa_hold_export::HoldExport>> for ResponseMethod<'x> {
fn from(value: GetResponse<crate::object::inbuxa_hold_export::HoldExport>) -> Self {
ResponseMethod::Get(GetResponseMethod::HoldExport(value))
}
}
impl<'x> From<SetResponse<crate::object::inbuxa_hold_export::HoldExport>> for ResponseMethod<'x> {
fn from(value: SetResponse<crate::object::inbuxa_hold_export::HoldExport>) -> Self {
ResponseMethod::Set(SetResponseMethod::HoldExport(Box::new(value)))
}
}
+1
View File
@@ -39,6 +39,7 @@ base64 = "0.23"
p256 = { version = "0.13", features = ["ecdh"] }
sha1 = "0.11"
sha2 = "0.11"
zip = "8.6" # inbuxa: legal hold exports (LH-12)
reqwest = { version = "0.13", default-features = false, features = ["rustls", "http2"]}
tokio-tungstenite = "0.30"
tungstenite = "0.30"
+10
View File
@@ -97,6 +97,7 @@ impl JmapAuthorization for AccessToken {
// inbuxa: account lock (AL-12)
GetRequestMethod::AccountLock(_) => Permission::SysAccountLockGet,
GetRequestMethod::LegalHold(_) => Permission::SysLegalHoldGet,
GetRequestMethod::HoldExport(_) => Permission::SysLegalHoldExport,
// inbuxa: legacy protocols off. It takes listeners away and
// puts them back, so it takes the listener's permissions
GetRequestMethod::ProtocolPolicy(_) => Permission::SysNetworkListenerGet,
@@ -232,6 +233,14 @@ impl JmapAuthorization for AccessToken {
Permission::SysLegalHoldUpdate,
Permission::SysLegalHoldUpdate,
),
// inbuxa: LH-12, exporting held data
SetRequestMethod::HoldExport(s) => validate_set(
s,
self,
Permission::SysLegalHoldExport,
Permission::SysLegalHoldExport,
Permission::SysLegalHoldExport,
),
SetRequestMethod::AuditVerification(s) => validate_set(
s,
self,
@@ -380,6 +389,7 @@ impl JmapAuthorization for AccessToken {
| MethodObject::AuditVerification
| MethodObject::AccountLock
| MethodObject::LegalHold
| MethodObject::HoldExport
| MethodObject::ProtocolPolicy
| MethodObject::TenantProtocolPolicy => Permission::JmapEmailChanges,
// inbuxa: x:MaskedEmail/changes reads what /get reads
+36
View File
@@ -276,6 +276,9 @@ impl RequestHandler for Server {
SetResponseMethod::LegalHold(set_response) => {
set_response.update_created_ids(&mut response);
}
SetResponseMethod::HoldExport(set_response) => {
set_response.update_created_ids(&mut response);
}
SetResponseMethod::Explanation(set_response) => {
set_response.update_created_ids(&mut response);
}
@@ -450,6 +453,11 @@ impl RequestHandler for Server {
.into()
}
// inbuxa: legal hold (LH-1)
// inbuxa: legal hold exports (LH-12)
GetRequestMethod::HoldExport(mut req) => {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
crate::inbuxa::hold_export_api::get(self, *req).await?.into()
}
GetRequestMethod::LegalHold(mut req) => {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
crate::inbuxa::legal_hold::get(self, *req).await?.into()
@@ -807,6 +815,34 @@ impl RequestHandler for Server {
}
// inbuxa: legal hold (LH-1), each change recorded with its
// reason (AU-12)
// inbuxa: legal hold exports, recorded with their reason
// (AU-1.9, AU-12)
SetRequestMethod::HoldExport(mut req) => {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
let reason = req.arguments.reason.clone().or_else(|| {
req.create.as_ref().and_then(|create| {
create.values().find_map(|value| {
serde_json::to_value(value)
.ok()?
.get("reason")?
.as_str()
.map(str::to_string)
})
})
});
crate::inbuxa::audit::recorded(
self,
access_token,
session,
&method_name.obj.to_string(),
None,
reason,
*req,
|req| Box::pin(crate::inbuxa::hold_export_api::set(self, access_token, req)),
)
.await?
.into()
}
SetRequestMethod::LegalHold(mut req) => {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
let reason = req.arguments.reason.clone().or_else(|| {
+1
View File
@@ -425,6 +425,7 @@ impl IntermediateChangesResponse {
| MethodObject::AuditVerification
| MethodObject::AccountLock
| MethodObject::LegalHold
| MethodObject::HoldExport
| MethodObject::ProtocolPolicy
| MethodObject::TenantProtocolPolicy
| MethodObject::Registry(_) => unreachable!(),
+429
View File
@@ -0,0 +1,429 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! Collecting what a legal hold keeps, as a ZIP (audit-hold-lock spec,
//! LH-12). For each account the hold covers (or those asked for): its mail,
//! calendars, contacts and files, and the deleted items the hold keeps, each
//! with its SHA-256 in `manifest.csv`, and the manifest's own hash beside
//! it. The hold's date range applies as it does to what's kept (LH-3).
use common::{Server, hold::kept_member};
use email::{
cache::MessageCacheFetch,
message::metadata::{MESSAGE_RECEIVED_MASK, MessageMetadata},
};
use groupware::{cache::GroupwareCache, calendar::CalendarEvent, contact::ContactCard, file::FileNode};
use inbuxa_features::{
hold::{Hold, Keeping, is_held_until},
undelete::records,
};
use registry::schema::{prelude::ObjectType, structs::ArchivedItem};
use sha2::{Digest, Sha256};
use std::io::{Cursor, Write};
use store::{
ValueKey,
registry::RegistryQuery,
write::{AlignedBytes, Archive},
};
use trc::AddContext;
use types::{
collection::{Collection, SyncCollection},
field::EmailField,
id::Id,
};
use zip::{CompressionMethod, ZipWriter, write::SimpleFileOptions};
/// The largest ZIP built in memory. A bigger collection is refused with a
/// clear error rather than taking the node down; export fewer accounts.
pub const MAX_EXPORT: u64 = 2 * 1024 * 1024 * 1024;
/// One line of `manifest.csv`.
struct Entry {
path: String,
account: String,
kind: &'static str,
folder: String,
date: Option<i64>,
archived: bool,
size: usize,
sha256: String,
}
fn hex(bytes: &[u8]) -> String {
bytes.iter().map(|b| format!("{b:02x}")).collect()
}
fn csv(field: &str) -> String {
if field.contains([',', '"', '\n', '\r']) {
format!("\"{}\"", field.replace('"', "\"\""))
} else {
field.to_string()
}
}
/// A path segment that's safe in a ZIP: no separators, no leading dots.
fn segment(name: &str) -> String {
let cleaned: String = name
.chars()
.map(|c| if c == '/' || c == '\\' || c.is_control() { '_' } else { c })
.collect();
let trimmed = cleaned.trim_start_matches('.').trim();
if trimmed.is_empty() { "_".into() } else { trimmed.chars().take(120).collect() }
}
fn date_text(at: Option<i64>) -> String {
at.map(|at| jmap_proto::types::date::UTCDate::from_timestamp(at).to_string())
.unwrap_or_default()
}
struct Builder {
zip: ZipWriter<Cursor<Vec<u8>>>,
entries: Vec<Entry>,
written: u64,
}
impl Builder {
fn new() -> Self {
Builder {
zip: ZipWriter::new(Cursor::new(Vec::new())),
entries: Vec::new(),
written: 0,
}
}
#[allow(clippy::too_many_arguments)]
fn add(
&mut self,
path: String,
bytes: &[u8],
account: &str,
kind: &'static str,
folder: &str,
date: Option<i64>,
archived: bool,
) -> trc::Result<()> {
self.written += bytes.len() as u64;
if self.written > MAX_EXPORT {
return Err(trc::StoreEvent::UnexpectedError
.into_err()
.details("The collection is larger than one export can hold (2 GB). Export fewer accounts at a time."));
}
// Unique within the ZIP, however names collide
let mut name = path.clone();
let mut n = 1;
while self.entries.iter().any(|e| e.path == name) {
n += 1;
name = match path.rsplit_once('.') {
Some((stem, ext)) if !stem.ends_with('/') => format!("{stem} ({n}).{ext}"),
_ => format!("{path} ({n})"),
};
}
let options = SimpleFileOptions::default().compression_method(CompressionMethod::Deflated);
self.zip
.start_file(name.as_str(), options)
.and_then(|_| self.zip.write_all(bytes).map_err(Into::into))
.map_err(|err| {
trc::StoreEvent::UnexpectedError
.into_err()
.details("Failed to write the export")
.reason(err)
})?;
self.entries.push(Entry {
path: name,
account: account.to_string(),
kind,
folder: folder.to_string(),
date,
archived,
size: bytes.len(),
sha256: hex(&Sha256::digest(bytes)),
});
Ok(())
}
/// Closes the ZIP with its manifest and the manifest's hash. Returns the
/// bytes and how many items went in.
fn finish(mut self) -> trc::Result<(Vec<u8>, usize)> {
let mut manifest = String::from("path,account,kind,folder,date,archived,size,sha256\n");
for e in &self.entries {
manifest.push_str(&format!(
"{},{},{},{},{},{},{},{}\n",
csv(&e.path),
csv(&e.account),
e.kind,
csv(&e.folder),
date_text(e.date),
e.archived,
e.size,
e.sha256
));
}
let manifest_hash = hex(&Sha256::digest(manifest.as_bytes()));
let options = SimpleFileOptions::default().compression_method(CompressionMethod::Deflated);
let fail = |err: zip::result::ZipError| {
trc::StoreEvent::UnexpectedError
.into_err()
.details("Failed to write the export")
.reason(err)
};
self.zip.start_file("manifest.csv", options).map_err(fail)?;
self.zip.write_all(manifest.as_bytes()).map_err(|e| fail(e.into()))?;
self.zip.start_file("manifest.sha256", options).map_err(fail)?;
self.zip
.write_all(format!("{manifest_hash} manifest.csv\n").as_bytes())
.map_err(|e| fail(e.into()))?;
let items = self.entries.len();
let bytes = self.zip.finish().map_err(fail)?.into_inner();
Ok((bytes, items))
}
}
/// The accounts to collect: those asked for that the hold covers, or every
/// account it covers, deleted ones it keeps included.
async fn accounts(server: &Server, hold: &Hold, asked: &[u32]) -> trc::Result<Vec<u32>> {
let mut covered = Vec::new();
for id in server
.registry()
.query::<Vec<Id>>(RegistryQuery::new(ObjectType::Account))
.await
.caused_by(trc::location!())?
{
let account_id = id.document_id();
if let Some(member) = server.member_of(account_id).await
&& hold.scope.covers(&member)
{
covered.push(account_id);
}
}
for (account_id, kept) in inbuxa_features::undelete::data::kept_accounts(server.store()).await? {
if hold.scope.covers(&kept_member(account_id, &kept)) {
covered.push(account_id);
}
}
covered.sort_unstable();
covered.dedup();
if !asked.is_empty() {
covered.retain(|id| asked.contains(id));
}
Ok(covered)
}
async fn blob(server: &Server, hash: &[u8]) -> trc::Result<Option<Vec<u8>>> {
server.blob_store().get_blob(hash, 0..usize::MAX).await
}
/// Collects `accounts` under `hold` into a ZIP. Returns its bytes and item
/// count.
pub async fn build(server: &Server, hold: &Hold, asked: &[u32]) -> trc::Result<(Vec<u8>, usize)> {
let keeping = Keeping::new(None, std::slice::from_ref(hold));
let data = server.store();
let mut out = Builder::new();
for account_id in accounts(server, hold, asked).await? {
let address = server.audit_account_name(account_id).await;
let base = format!("{}/", segment(&address));
let live = server.account(account_id).await.is_ok();
if live {
// Mail, by the folder it's in
let cache = server
.get_cached_messages(account_id)
.await
.caused_by(trc::location!())?;
for message in cache.emails.items.iter() {
let Some(metadata_) = data
.get_value::<Archive<AlignedBytes>>(ValueKey::property(
account_id,
Collection::Email,
message.document_id,
EmailField::Metadata,
))
.await?
else {
continue;
};
let metadata = metadata_
.unarchive::<MessageMetadata>()
.caused_by(trc::location!())?;
let received = metadata.rcvd_attach.to_native() & MESSAGE_RECEIVED_MASK;
if !keeping.covers(Some(received)) {
continue;
}
let folder = message
.mailboxes
.first()
.and_then(|m| cache.mailboxes.items.iter().find(|b| b.document_id == m.mailbox_id))
.map(|b| b.path.clone())
.unwrap_or_default();
let hash = types::blob_hash::BlobHash::from(&metadata.blob_hash);
if let Some(bytes) = blob(server, hash.as_slice()).await? {
let path = format!(
"{base}mail/{}/{}.eml",
folder.split('/').map(segment).collect::<Vec<_>>().join("/"),
Id::from(message.document_id)
);
out.add(path, &bytes, &address, "email", &folder, Some(received as i64), false)?;
}
}
// Calendars, contacts and files, by their DAV paths
for (sync, kind) in [
(SyncCollection::Calendar, "event"),
(SyncCollection::AddressBook, "contact"),
(SyncCollection::FileNode, "file"),
] {
let resources = server
.fetch_dav_resources(account_id, account_id, sync)
.await
.caused_by(trc::location!())?;
for path in resources.paths.iter() {
let Some(resource) = resources.resources.get(path.resource_idx) else {
continue;
};
let folder = path.path.rsplit_once('/').map(|(f, _)| f).unwrap_or_default();
let zip_path = |ext: Option<&str>| {
let mut p = format!(
"{base}{}/{}",
match kind {
"event" => "calendar",
"contact" => "contacts",
_ => "files",
},
path.path.split('/').map(segment).collect::<Vec<_>>().join("/")
);
if let Some(ext) = ext
&& !p.ends_with(ext)
{
p.push_str(ext);
}
p
};
use common::DavResourceMetadata as M;
match &resource.data {
M::CalendarEvent { start, .. } => {
if !keeping.covers_event(Some((*start).max(0) as u64)) {
continue;
}
let Some(event_) = data
.get_value::<Archive<AlignedBytes>>(ValueKey::archive(
account_id,
Collection::CalendarEvent,
resource.document_id,
))
.await?
else {
continue;
};
let event = event_.unarchive::<CalendarEvent>().caused_by(trc::location!())?;
let text = event.data.event.to_string();
out.add(zip_path(Some(".ics")), text.as_bytes(), &address, kind, folder, Some(*start), false)?;
}
M::ContactCard { .. } => {
let Some(card_) = data
.get_value::<Archive<AlignedBytes>>(ValueKey::archive(
account_id,
Collection::ContactCard,
resource.document_id,
))
.await?
else {
continue;
};
let card = card_.unarchive::<ContactCard>().caused_by(trc::location!())?;
let mut text = String::with_capacity(256);
let _ = card.card.write_to(&mut text, server.core.groupware.vcard_version);
out.add(zip_path(Some(".vcf")), text.as_bytes(), &address, kind, folder, None, false)?;
}
M::File { size: Some(_), .. } => {
let Some(file_) = data
.get_value::<Archive<AlignedBytes>>(ValueKey::archive(
account_id,
Collection::FileNode,
resource.document_id,
))
.await?
else {
continue;
};
let file = file_.unarchive::<FileNode>().caused_by(trc::location!())?;
let Some(props) = file.file.as_ref() else {
continue;
};
let hash = types::blob_hash::BlobHash::from(&props.blob_hash);
if let Some(bytes) = blob(server, hash.as_slice()).await? {
out.add(zip_path(None), &bytes, &address, kind, folder, None, false)?;
}
}
_ => {}
}
}
}
}
// What the hold keeps of what was deleted
for (id, item) in records::of_account(data, server.registry(), account_id).await? {
if !is_held_until(item.archived_until().timestamp().max(0) as u64) {
continue;
}
let (kind, ext, date) = match &item {
ArchivedItem::Email(e) => ("email", ".eml", Some(e.received_at.timestamp())),
ArchivedItem::CalendarEvent(e) => ("event", ".ics", e.start_time.map(|t| t.timestamp())),
ArchivedItem::ContactCard(_) => ("contact", ".vcf", None),
ArchivedItem::FileNode(_) => ("file", "", None),
ArchivedItem::SieveScript(_) => ("sieve", ".sieve", None),
};
// Kept by this hold, not only by another one over the same account
let in_range = match kind {
"event" => keeping.covers_event(date.map(|d| d.max(0) as u64)),
_ => keeping.covers(date.map(|d| d.max(0) as u64)),
};
if !in_range {
continue;
}
let name = match &item {
ArchivedItem::FileNode(f) => segment(&f.name),
ArchivedItem::SieveScript(s) => format!("{}{ext}", segment(&s.name)),
_ => format!("{id}{ext}"),
};
if let Some(bytes) = blob(server, item.blob_id().hash.as_slice()).await? {
out.add(format!("{base}archived/{kind}/{name}"), &bytes, &address, kind, "", date, true)?;
}
}
}
out.finish()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn names_are_safe_in_a_zip() {
assert_eq!(segment("../etc/passwd"), "_etc_passwd");
assert_eq!(segment(" "), "_");
assert_eq!(segment("Q3 report.pdf"), "Q3 report.pdf");
assert_eq!(csv("a,b"), "\"a,b\"");
assert_eq!(csv("say \"hi\""), "\"say \"\"hi\"\"\"");
}
#[test]
fn a_zip_carries_its_manifest_and_its_hash() {
let mut b = Builder::new();
b.add("[email protected]/mail/INBOX/b.eml".into(), b"Subject: x\r\n\r\ny", "[email protected]", "email", "INBOX", Some(0), false)
.unwrap();
b.add("[email protected]/mail/INBOX/b.eml".into(), b"other", "[email protected]", "email", "INBOX", None, true)
.unwrap();
let (bytes, items) = b.finish().unwrap();
assert_eq!(items, 2);
let mut zip = zip::ZipArchive::new(Cursor::new(bytes)).unwrap();
let mut manifest = String::new();
std::io::Read::read_to_string(&mut zip.by_name("manifest.csv").unwrap(), &mut manifest).unwrap();
assert!(manifest.contains("[email protected]/mail/INBOX/b (2).eml"), "{manifest}");
let mut hash = String::new();
std::io::Read::read_to_string(&mut zip.by_name("manifest.sha256").unwrap(), &mut hash).unwrap();
assert!(hash.starts_with(&hex(&Sha256::digest(manifest.as_bytes()))));
}
}
+294
View File
@@ -0,0 +1,294 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! `inbuxa:HoldExport` (audit-hold-lock spec, LH-12): starting a collection
//! of what a hold keeps, and seeing how it went. The ZIP is the creator's
//! blob, to download once it's ready. Creating one is audited, with its
//! reason (AU-1.9, AU-12).
use common::{Server, auth::AccessToken};
use inbuxa_features::hold::{self, Export, ExportStatus};
use jmap_proto::{
error::set::SetError,
method::{
get::{GetRequest, GetResponse},
set::{SetRequest, SetResponse},
},
object::inbuxa_hold_export::{
HoldExport, HoldExportProperty as P, HoldExportSetArguments, HoldExportValue,
},
types::date::UTCDate,
};
use jmap_tools::{Key, Map, Value};
use sha2::{Digest, Sha256};
use std::str::FromStr;
use store::write::now;
use types::id::Id;
type LValue = Value<'static, P, HoldExportValue>;
const ALL: &[P] = &[
P::Id,
P::HoldId,
P::AccountIds,
P::Reason,
P::Status,
P::CreatedAt,
P::CreatedBy,
P::FinishedAt,
P::BlobId,
P::Size,
P::Items,
P::Sha256,
P::Error,
];
fn date(seconds: u64) -> LValue {
Value::Str(UTCDate::from_timestamp(seconds as i64).to_string().into())
}
fn opt_text(value: &Option<String>) -> LValue {
value.as_ref().map_or(Value::Null, |v| Value::Str(v.clone().into()))
}
fn to_value(export: &Export, properties: &[P]) -> LValue {
let mut out = Map::with_capacity(properties.len());
for property in properties {
let value = match property {
P::Id => Value::Element(HoldExportValue::Id(Id::from(export.id))),
P::HoldId => Value::Str(Id::from(export.hold_id).to_string().into()),
P::AccountIds => Value::Array(
export
.accounts
.iter()
.map(|id| Value::Str(Id::from(*id).to_string().into()))
.collect(),
),
P::Reason => Value::Str(export.reason.clone().into()),
P::Status => Value::Str(
match export.status {
ExportStatus::Running => "running",
ExportStatus::Ready => "ready",
ExportStatus::Failed => "failed",
}
.into(),
),
P::CreatedAt => date(export.created_at),
P::CreatedBy => Value::Str(export.created_by.clone().into()),
P::FinishedAt => export.finished_at.map_or(Value::Null, date),
P::BlobId => opt_text(&export.blob_id),
P::Size => Value::Number(export.size.into()),
P::Items => Value::Number(export.items.into()),
P::Sha256 => opt_text(&export.sha256),
P::Error => opt_text(&export.error),
};
out.insert_unchecked(Key::Property(property.clone()), value);
}
Value::Object(out)
}
/// `inbuxa:HoldExport/get`: every export, newest first.
pub async fn get(
server: &Server,
mut request: GetRequest<HoldExport>,
) -> trc::Result<GetResponse<HoldExport>> {
let properties = request.unwrap_properties(ALL);
let (ids, not_found) = request.unwrap_ids(server.core.jmap.get_max_objects)?;
let mut response = GetResponse {
account_id: request.account_id.into(),
state: None,
list: Vec::new(),
not_found,
};
let mut exports = hold::exports(server.store()).await?;
exports.reverse();
match ids {
None => response
.list
.extend(exports.iter().map(|e| to_value(e, &properties))),
Some(ids) => {
for id in ids {
match exports.iter().find(|e| u64::from(e.id) == id.id()) {
Some(export) => response.list.push(to_value(export, &properties)),
None => response.push_not_found(id),
}
}
}
}
Ok(response)
}
fn invalid(property: P, why: &str) -> SetError<P> {
SetError::invalid_properties()
.with_property(property)
.with_description(why.to_string())
}
/// `inbuxa:HoldExport/set`: create starts an export; nothing else is
/// allowed. The request layer records it with its reason.
pub async fn set(
server: &Server,
access_token: &AccessToken,
mut request: SetRequest<'_, HoldExport>,
) -> trc::Result<SetResponse<HoldExport>> {
let mut response = SetResponse::from_request(&request, server.core.jmap.set_max_objects)?;
let arguments: HoldExportSetArguments = std::mem::take(&mut request.arguments);
let data = server.store();
let actor = server.audit_actor(access_token).await;
'create: for (client_id, value) in request.unwrap_create() {
let mut hold_id = None;
let mut accounts = Vec::new();
let mut reason = arguments.reason.clone();
for (key, value) in value.into_expanded_object() {
match (&key, &value) {
(Key::Property(P::HoldId), Value::Str(id)) => {
hold_id = Id::from_str(id).ok().and_then(|id| u32::try_from(id.id()).ok())
}
(Key::Property(P::AccountIds), Value::Array(items)) => {
for item in items {
match item {
Value::Str(id) => match Id::from_str(id) {
Ok(id) => accounts.push(id.document_id()),
Err(_) => {
response.not_created.append(
client_id,
invalid(P::AccountIds, "accountIds must be account ids."),
);
continue 'create;
}
},
_ => {
response.not_created.append(
client_id,
invalid(P::AccountIds, "accountIds must be account ids."),
);
continue 'create;
}
}
}
}
(Key::Property(P::Reason), Value::Str(r)) => reason = Some(r.to_string()),
_ => {
response.not_created.append(
client_id,
SetError::invalid_properties().with_property(key.clone().into_owned()),
);
continue 'create;
}
}
}
let Some(reason) = reason
.map(|r| r.trim().chars().take(500).collect::<String>())
.filter(|r| !r.is_empty())
else {
response.not_created.append(
client_id,
invalid(P::Reason, "Say why: a reason is required and is kept in the audit log."),
);
continue;
};
let hold = match hold_id {
Some(id) => hold::get(data, id).await?,
None => None,
};
let Some(hold) = hold else {
response
.not_created
.append(client_id, invalid(P::HoldId, "No such legal hold."));
continue;
};
if !hold.is_active() {
response.not_created.append(
client_id,
invalid(P::HoldId, "That hold was released; export while a hold is in place."),
);
continue;
}
let Some(created_by_id) = actor.account_id else {
response
.not_created
.append(client_id, SetError::forbidden().with_description("Sign in as a person to export."));
continue;
};
accounts.sort_unstable();
accounts.dedup();
let export = Export {
id: 0,
hold_id: hold.id,
accounts,
reason,
created_at: now(),
created_by: actor.name.clone(),
created_by_id,
status: ExportStatus::Running,
finished_at: None,
blob_id: None,
size: 0,
items: 0,
sha256: None,
error: None,
};
let id = hold::create_export(data, &export).await?;
let export = Export { id, ..export };
// The collection runs on its own; get says when it's ready
let server = server.clone();
tokio::spawn(async move {
let mut done = export.clone();
match crate::inbuxa::hold_export::build(&server, &hold, &export.accounts).await {
Ok((bytes, items)) => match server.put_jmap_blob(export.created_by_id, &bytes).await {
Ok(blob) => {
done.status = ExportStatus::Ready;
done.blob_id = Some(blob.to_string());
done.size = bytes.len() as u64;
done.items = items as u64;
done.sha256 = Some(
Sha256::digest(&bytes).iter().map(|b| format!("{b:02x}")).collect(),
);
}
Err(err) => {
done.status = ExportStatus::Failed;
done.error = Some(err.to_string());
}
},
Err(err) => {
done.status = ExportStatus::Failed;
done.error = Some(
err.value_as_str(trc::Key::Details)
.map(str::to_string)
.unwrap_or_else(|| err.to_string()),
);
}
}
done.finished_at = Some(now());
if let Err(err) = hold::update_export(server.store(), &done).await {
trc::error!(err.details("Failed to save a legal hold export's result"));
}
});
let mut out = Map::with_capacity(1);
out.insert_unchecked(
Key::Property(P::Id),
Value::Element(HoldExportValue::Id(Id::from(id))),
);
response.created.insert(client_id, Value::Object(out));
}
for (id, _) in request.unwrap_update() {
response.not_updated.append(
id,
SetError::forbidden().with_description("An export can't be changed; start a new one."),
);
}
for id in request.unwrap_destroy() {
response.not_destroyed.append(
id,
SetError::forbidden().with_description("Exports stay listed; the file expires on its own."),
);
}
Ok(response)
}
+2
View File
@@ -10,6 +10,8 @@
pub mod access;
pub mod account_lock;
pub mod legal_hold;
pub mod hold_export;
pub mod hold_export_api;
pub mod audit;
pub mod audit_log;
pub mod ai_limits;
+273 -28
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/
use common::{
@@ -13,6 +15,8 @@ use mail_auth::{
MX, RecordSet,
common::resolver::ToFqdn,
hickory_resolver::{
TokioResolver,
lookup::Lookup,
net::{DnsError, NetError},
proto::{
dnssec::Proof,
@@ -83,16 +87,15 @@ impl TlsaLookup for Server {
return mail_auth::common::resolver::mock_resolve(key.as_ref());
}
let mx_lookup = match self
.core
.smtp
.resolvers
.dnssec
.resolver
.mx_lookup(Name::from_str_relaxed::<&str>(key.as_ref())?)
let (mx_lookup, forced_insecure) = match validated_lookup(
&self.core.smtp.resolvers.dnssec.resolver,
self.core.smtp.resolvers.dns.resolver(),
Name::from_str_relaxed::<&str>(key.as_ref())?,
RecordType::MX,
)
.await
{
Ok(mx_lookup) => mx_lookup,
Ok(validated) => (validated.lookup, validated.insecure),
Err(err) => {
if let Some(denial) = NegativeAnswer::from_error(&err)
&& denial.response_code == ResponseCode::NoError
@@ -144,7 +147,11 @@ impl TlsaLookup for Server {
.collect::<Arc<[MX]>>();
let records = RecordSet {
rrset,
dnssec_status: dnssec_status.unwrap_or(DnssecStatus::Indeterminate),
dnssec_status: if forced_insecure {
DnssecStatus::Insecure
} else {
dnssec_status.unwrap_or(DnssecStatus::Indeterminate)
},
};
self.inner
@@ -285,16 +292,15 @@ impl TlsaLookup for Server {
}
let name = Name::from_str_relaxed::<&str>(key.as_ref())?;
let lookup = match self
.core
.smtp
.resolvers
.dnssec
.resolver
.ipv4_lookup(name.clone())
let (lookup, forced_insecure) = match validated_lookup(
&self.core.smtp.resolvers.dnssec.resolver,
self.core.smtp.resolvers.dns.resolver(),
name.clone(),
RecordType::A,
)
.await
{
Ok(lookup) => lookup,
Ok(validated) => (validated.lookup, validated.insecure),
Err(err) => {
if let Some(denial) = NegativeAnswer::from_error(&err)
&& denial.response_code == ResponseCode::NoError
@@ -325,7 +331,11 @@ impl TlsaLookup for Server {
_ => None,
})
.collect::<Arc<[Ipv4Addr]>>(),
dnssec_status: tlsa_base_status(&name, answers, RecordType::A),
dnssec_status: if forced_insecure {
DnssecStatus::Insecure
} else {
tlsa_base_status(&name, answers, RecordType::A)
},
};
self.inner
@@ -363,16 +373,15 @@ impl TlsaLookup for Server {
}
let name = Name::from_str_relaxed::<&str>(key.as_ref())?;
let lookup = match self
.core
.smtp
.resolvers
.dnssec
.resolver
.ipv6_lookup(name.clone())
let (lookup, forced_insecure) = match validated_lookup(
&self.core.smtp.resolvers.dnssec.resolver,
self.core.smtp.resolvers.dns.resolver(),
name.clone(),
RecordType::AAAA,
)
.await
{
Ok(lookup) => lookup,
Ok(validated) => (validated.lookup, validated.insecure),
Err(err) => {
if let Some(denial) = NegativeAnswer::from_error(&err)
&& denial.response_code == ResponseCode::NoError
@@ -403,7 +412,11 @@ impl TlsaLookup for Server {
_ => None,
})
.collect::<Arc<[Ipv6Addr]>>(),
dnssec_status: tlsa_base_status(&name, answers, RecordType::AAAA),
dnssec_status: if forced_insecure {
DnssecStatus::Insecure
} else {
tlsa_base_status(&name, answers, RecordType::AAAA)
},
};
self.inner
@@ -415,6 +428,115 @@ impl TlsaLookup for Server {
}
}
// inbuxa: hickory 0.26.3 calls some valid answers bogus, and the queue then
// retries those hosts until the message expires. Two cases seen in production:
//
// - A zone delegated beneath an unsigned zone, such as `l.google.com` under
// `google.com`. To prove the delegation insecure, hickory wants an SOA
// record in the DS reply, and public resolvers often send none.
// - A signed CNAME to a signed name without the record type queried. Hickory
// checks the denial of existence against the name first asked for, not the
// target's, and rejects it.
//
// When hickory says bogus, check the answer again with lookups it gets right.
// A signed CNAME is followed and the lookup repeated at its target. Otherwise
// the name's zone and its parents are looked up, nearest first. If one
// validates as unsigned, nothing below it can be signed, so the plain resolver
// answers and the result is insecure. If one validates as signed first, the
// verdict stands.
const MAX_BOGUS_ALIASES: usize = 8;
struct ValidatedLookup {
lookup: Lookup,
insecure: bool,
}
enum BogusRecheck {
Alias(Name),
Insecure,
Bogus,
}
async fn validated_lookup(
dnssec: &TokioResolver,
plain: &TokioResolver,
name: Name,
record_type: RecordType,
) -> Result<ValidatedLookup, NetError> {
let mut query = name;
let mut aliases = 0;
loop {
let err = match dnssec.lookup(query.clone(), record_type).await {
Ok(lookup) => {
return Ok(ValidatedLookup {
lookup,
insecure: false,
});
}
Err(err @ NetError::Dns(DnsError::DnssecBogus)) => err,
Err(err) => return Err(err),
};
match recheck_bogus(dnssec, &query).await {
BogusRecheck::Alias(target) if aliases < MAX_BOGUS_ALIASES => {
aliases += 1;
query = target;
}
BogusRecheck::Insecure => {
return plain
.lookup(query, record_type)
.await
.map(|lookup| ValidatedLookup {
lookup,
insecure: true,
});
}
BogusRecheck::Alias(_) | BogusRecheck::Bogus => return Err(err),
}
}
}
async fn recheck_bogus(dnssec: &TokioResolver, name: &Name) -> BogusRecheck {
if let Ok(lookup) = dnssec.lookup(name.clone(), RecordType::CNAME).await
&& let Some(target) = secure_alias(name, lookup.answers())
{
return BogusRecheck::Alias(target);
}
let mut zone = name.clone();
while !zone.is_root() {
if let Ok(lookup) = dnssec.lookup(zone.clone(), RecordType::SOA).await {
match apex_status(&zone, lookup.answers()) {
Some(DnssecStatus::Insecure) => return BogusRecheck::Insecure,
Some(DnssecStatus::Secure) => return BogusRecheck::Bogus,
_ => {}
}
}
zone = zone.base_name();
}
BogusRecheck::Bogus
}
fn secure_alias(query: &Name, answers: &[Record]) -> Option<Name> {
answers.iter().find_map(|record| match &record.data {
RData::CNAME(target) if &record.name == query && record.proof.is_secure() => {
Some(target.0.clone())
}
_ => None,
})
}
fn apex_status(zone: &Name, answers: &[Record]) -> Option<DnssecStatus> {
answers
.iter()
.filter(|record| record.record_type() == RecordType::SOA && &record.name == zone)
.map(|record| proof_to_dnssec_status(record.proof))
.reduce(least_secure)
}
struct NegativeAnswer {
response_code: ResponseCode,
dnssec_status: DnssecStatus,
@@ -511,7 +633,7 @@ pub(crate) fn least_secure(a: DnssecStatus, b: DnssecStatus) -> DnssecStatus {
#[cfg(test)]
mod tests {
use super::*;
use mail_auth::hickory_resolver::proto::rr::rdata::{A, CNAME};
use mail_auth::hickory_resolver::proto::rr::rdata::{A, CNAME, SOA};
use std::net::Ipv4Addr;
fn name(value: &str) -> Name {
@@ -624,4 +746,127 @@ mod tests {
DnssecStatus::Insecure
);
}
fn soa(owner: &str, proof: Proof) -> Record {
let mut record = Record::from_rdata(
name(owner),
3600,
RData::SOA(SOA::new(
name("ns1.example.org."),
name("hostmaster.example.org."),
1,
900,
900,
1800,
60,
)),
);
record.proof = proof;
record
}
#[test]
fn secure_alias_follows_signed_cname() {
assert_eq!(
secure_alias(
&name("mail.example.org."),
&[alias("mail.example.org.", "mx.example.net.", Proof::Secure)]
),
Some(name("mx.example.net."))
);
}
#[test]
fn secure_alias_ignores_unsigned_or_other_cname() {
let query = name("mail.example.org.");
assert_eq!(
secure_alias(
&query,
&[alias(
"mail.example.org.",
"mx.example.net.",
Proof::Insecure
)]
),
None
);
assert_eq!(
secure_alias(
&query,
&[alias(
"other.example.org.",
"mx.example.net.",
Proof::Secure
)]
),
None
);
}
#[test]
fn apex_status_reads_the_zone_soa() {
let zone = name("example.com.");
for (proof, expected) in [
(Proof::Secure, Some(DnssecStatus::Secure)),
(Proof::Insecure, Some(DnssecStatus::Insecure)),
(Proof::Bogus, Some(DnssecStatus::Bogus)),
] {
assert_eq!(
apex_status(&zone, &[soa("example.com.", proof)]),
expected,
"proof {proof}"
);
}
}
#[test]
fn apex_status_ignores_other_records() {
assert_eq!(
apex_status(
&name("example.com."),
&[
soa("sub.example.com.", Proof::Insecure),
address("example.com.", Proof::Insecure),
]
),
None
);
}
// Needs the network: a signed MX pointing into a zone delegated beneath an
// unsigned one. Run with `--ignored` to check a hickory upgrade.
#[tokio::test]
#[ignore]
async fn validated_lookup_proves_delegation_below_unsigned_zone() {
use mail_auth::hickory_resolver::{
config::{CLOUDFLARE, ResolverConfig, ResolverOpts},
net::runtime::TokioRuntimeProvider,
};
let build = |validate: bool| {
// Same options as the server's DNSSEC resolver; hickory fails
// validation with concurrent requests.
let mut opts = ResolverOpts::default();
opts.validate = validate;
opts.num_concurrent_reqs = 1;
opts.cache_size = 0;
TokioResolver::builder_with_config(
ResolverConfig::udp_and_tcp(&CLOUDFLARE),
TokioRuntimeProvider::default(),
)
.with_options(opts)
.build()
.unwrap()
};
let (dnssec, plain) = (build(true), build(false));
let validated =
validated_lookup(&dnssec, &plain, name("aspmx.l.google.com."), RecordType::A)
.await
.unwrap();
assert!(validated.insecure);
assert!(!validated.lookup.answers().is_empty());
}
}
+57 -2
View File
@@ -26,7 +26,10 @@ use mail_auth::{
common::verify::VerifySignature,
dkim2::Dkim2Output,
dmarc::{self},
report::{AuthFailureType, IdentityAlignment, PolicyPublished, Record, SPFDomainScope},
report::{
ActionDisposition, AuthFailureType, IdentityAlignment, PolicyPublished, Record, Report,
SPFDomainScope,
},
};
use registry::{
schema::{
@@ -459,7 +462,7 @@ impl DmarcReporting for Server {
.await
.unwrap_or_else(|| "MAILER-DAEMON@localhost".to_compact_string());
let mut message = Vec::with_capacity(2048);
let _ = mail_auth::report::Report::from(report.report).write_rfc5322(
let _ = with_compatible_dispositions(Report::from(report.report)).write_rfc5322(
&self
.eval_if(
&self.core.smtp.report.submitter,
@@ -710,3 +713,55 @@ impl DmarcReporting for Server {
}
}
}
// inbuxa: RFC 9990 added "pass" to the evaluated disposition for mail that
// passed DMARC under an enforcing policy. Cloudflare's report intake rejects
// the whole report with "555 5.7.1 invalid_report_schema" when it sees that
// value, and older parsers built on the RFC 7489 schema do the same. "none"
// (no action taken) is valid under both and says the same thing, so reports
// go out with that instead.
fn with_compatible_dispositions(mut report: Report) -> Report {
for record in &mut report.record {
let disposition = &mut record.row.policy_evaluated.disposition;
if *disposition == ActionDisposition::Pass {
*disposition = ActionDisposition::None;
}
}
report
}
#[cfg(test)]
mod tests {
use super::*;
use mail_auth::report::DmarcResult;
fn record(disposition: ActionDisposition) -> Record {
Record::new()
.with_source_ip("192.0.2.1".parse().unwrap())
.with_count(1)
.with_action_disposition(disposition)
.with_dmarc_dkim_result(DmarcResult::Pass)
.with_dmarc_spf_result(DmarcResult::Fail)
.with_header_from("example.org")
}
#[test]
fn pass_disposition_is_reported_as_none() {
let xml = with_compatible_dispositions(
Report::new()
.with_domain("example.org")
.with_record(record(ActionDisposition::Pass))
.with_record(record(ActionDisposition::Quarantine))
.with_record(record(ActionDisposition::Reject)),
)
.to_xml();
assert!(!xml.contains("<disposition>pass</disposition>"), "{xml}");
assert!(xml.contains("<disposition>none</disposition>"), "{xml}");
assert!(
xml.contains("<disposition>quarantine</disposition>"),
"{xml}"
);
assert!(xml.contains("<disposition>reject</disposition>"), "{xml}");
}
}
+1 -1
View File
@@ -81,7 +81,7 @@ fn legacy_setting(name: &str, is_set: impl Fn(&str) -> bool) -> Option<String> {
#[macro_export]
macro_rules! brand_version {
() => {
"2026.9.28"
"2026.9.28.2"
};
}
Binary file not shown.
+1
View File
@@ -86,6 +86,7 @@ dns-update = { version = "0.5", features = ["test_provider"] }
x509-parser = "0.18"
rcgen = "0.14"
sha2 = "0.11"
zip = "8.6" # inbuxa: reading legal hold exports
time = "0.3"
testcontainers = { version = "0.28", features = ["reusable-containers"] }
rust-s3 = { version = "0.37", default-features = false, features = ["tokio-rustls-tls"] }
+4 -1
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/
use crate::{
@@ -174,7 +176,8 @@ async fn report_dmarc() {
let source_ip = record.source_ip().unwrap();
if source_ip == "192.168.1.2".parse::<IpAddr>().unwrap() {
assert_eq!(record.count(), 2);
assert_eq!(record.action_disposition(), ActionDisposition::Pass);
// inbuxa: "pass" goes out as "none" for RFC 7489 parsers
assert_eq!(record.action_disposition(), ActionDisposition::None);
assert_eq!(record.envelope_from(), "[email protected]");
assert_eq!(record.header_from(), "[email protected]");
assert_eq!(record.envelope_to().unwrap(), "[email protected]");
+110
View File
@@ -436,6 +436,90 @@ pub async fn test(test: &mut TestServer) {
names.sort_unstable();
assert_eq!(names, vec!["Matter 7001", "Matter 7002"], "LH-14: {response}");
// LH-12: collect what the hold keeps, as a ZIP with its manifest
import(&frozen_client, "Still in the inbox", None).await;
let (_, response) = admin
.hold_call(
"inbuxa:HoldExport/set",
json!({"create": {"x": {"holdId": first, "accountIds": [frozen.id_string()]}}}),
)
.await;
assert_eq!(
response["notCreated"]["x"]["type"], "invalidProperties",
"AU-12: an export without a reason: {response}"
);
let (_, response) = admin
.hold_call(
"inbuxa:HoldExport/set",
json!({"reason": "Production to opposing counsel",
"create": {"x": {"holdId": first,
"accountIds": [frozen.id_string(), admin.id_string()]}}}),
)
.await;
let export_id = response["created"]["x"]["id"]
.as_str()
.unwrap_or_else(|| panic!("LH-12: not started: {response}"))
.to_string();
let mut export = Value::Null;
for _ in 0..60 {
let (_, got) = admin
.hold_call("inbuxa:HoldExport/get", json!({"ids": [export_id]}))
.await;
export = got["list"][0].clone();
if export["status"] != "running" {
break;
}
tokio::time::sleep(std::time::Duration::from_millis(250)).await;
}
assert_eq!(export["status"], "ready", "LH-12: {export}");
let bytes = admin
.jmap_client()
.await
.download(export["blobId"].as_str().unwrap())
.await
.unwrap();
assert_eq!(export["size"].as_u64(), Some(bytes.len() as u64), "{export}");
let mut zip = zip::ZipArchive::new(std::io::Cursor::new(bytes)).unwrap();
let names = (0..zip.len())
.map(|i| zip.by_index(i).unwrap().name().to_string())
.collect::<Vec<_>>();
assert!(
names.iter().any(|n| n.starts_with("[email protected]/mail/") && n.ends_with(".eml")),
"LH-12: live mail missing: {names:?}"
);
assert!(
names.iter().any(|n| n.starts_with("[email protected]/archived/email/")),
"LH-12: the kept deleted mail is missing: {names:?}"
);
assert!(
names
.iter()
.all(|n| n.starts_with("[email protected]/") || n.starts_with("manifest.")),
"LH-12: an account the hold doesn't cover was exported: {names:?}"
);
let mut manifest = String::new();
std::io::Read::read_to_string(&mut zip.by_name("manifest.csv").unwrap(), &mut manifest).unwrap();
let mut hash = String::new();
std::io::Read::read_to_string(&mut zip.by_name("manifest.sha256").unwrap(), &mut hash).unwrap();
use sha2::Digest;
let expected: String = sha2::Sha256::digest(manifest.as_bytes())
.iter()
.map(|b| format!("{b:02x}"))
.collect();
assert!(hash.starts_with(&expected), "LH-12: the manifest's hash doesn't match");
assert!(manifest.contains(",true,"), "LH-12: nothing marked archived: {manifest}");
// LH-13: only sysLegalHoldExport starts one
let (_, response) = frozen
.hold_call(
"inbuxa:HoldExport/set",
json!({"reason": "Mine", "create": {"x": {"holdId": first}}}),
)
.await;
assert!(
response.to_string().contains("forbidden") && response["created"].is_null(),
"LH-13: a user exported a hold: {response}"
);
let (_, response) = frozen
.hold_call("x:ArchivedItem/set", json!({"destroy": [item_id]}))
.await;
@@ -470,6 +554,16 @@ pub async fn test(test: &mut TestServer) {
.await;
let item = archived(frozen.archived_items().await);
assert!(!is_held(&item), "LH-10: the last release left it held: {item}");
let (_, response) = admin
.hold_call(
"inbuxa:HoldExport/set",
json!({"reason": "Too late", "create": {"x": {"holdId": first}}}),
)
.await;
assert_eq!(
response["notCreated"]["x"]["type"], "invalidProperties",
"LH-12: a released hold was exported: {response}"
);
let until = item["archivedUntil"].as_str().unwrap_or_default().to_string();
let grace = chrono::Utc::now() + chrono::Duration::days(29);
assert!(
@@ -574,6 +668,22 @@ pub async fn test(test: &mut TestServer) {
for reason in ["Counsel's letter", "Counsel widened the matter", "Matter settled"] {
assert!(reasons.contains(&reason), "AU-12: {reason:?} not recorded: {reasons:?}");
}
// AU-1.9: an export is recorded with its reason
let (_, query) = admin
.hold_call(
"inbuxa:AuditEvent/query",
json!({"filter": {"targetKind": "inbuxa:HoldExport"}}),
)
.await;
let (_, exports) = admin
.hold_call("inbuxa:AuditEvent/get", json!({"ids": query["ids"].clone()}))
.await;
assert!(
exports["list"]
.as_array()
.is_some_and(|l| l.iter().any(|r| r["reason"] == "Production to opposing counsel")),
"AU-1.9: the export isn't recorded: {exports}"
);
// A release is recorded under the hold's name, from before to after
let list = records["list"].as_array().cloned().unwrap_or_default();
let release = list