Journaling: capture at the queue, the built-in journal, retention
ci / fork-checks (pull_request) Successful in 41s
ci / build (pull_request) Successful in 8m2s

Phase 2 of the journaling spec.

- A copy of each message is taken in MessageWrapper::queue, after DLP and
  transport rules, for every enabled journal that takes it (direction and
  scope: everyone, or accounts, groups, domains, tenants). If the copy
  can't be taken the message isn't queued (temporary failure).
- The journal report: the envelope one field a line (sender, To, Cc, Bcc
  from the envelope, list members from their ORCPT, direction, held for
  review), then the queued message byte for byte as message/rfc822.
- The built-in journal under J in the inbuxa subspace: one chain per node
  whose links name each entry by SHA-256, so entries can expire out of
  chain order; purge leaves a marker, and verify catches an entry changed
  or removed early and a report that doesn't match.
- Retention per journal (30 to 3650 days); an entry keeps what it was
  written with. The daily maintenance purges what's due, keeping entries
  whose people a legal hold covers (deleted accounts a hold keeps too),
  and records the counts in the audit log.
- inbuxa:Journal get/set, audited by the request layer. Permissions
  680-683: administrators see and change journals; the Compliance Officer
  sees, searches and exports. Whoever changes journals may grant search and
  export without holding them, so officers can still be appointed.
- Catalog entries (inbuxa:Journal, source "journal"); spec as-built notes.

tests/src/system/journal.rs: validation, internal mail with a Bcc,
outgoing into two journals, incoming over LMTP, the report and its
original, tamper and early removal caught, hold-aware purge, retention
changes leave entries alone, disabled and removed journals take nothing.
This commit is contained in:
2026-09-28 20:46:04 -07:00
parent 792ff9d1ee
commit 441ad0b18e
33 changed files with 2853 additions and 4 deletions
@@ -0,0 +1,205 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! `inbuxa:Journal/get` and `/set` under `urn:inbuxa:jmap`: journals
//! (journaling spec, JR-9, JR-12). What a journal has taken stays when the
//! journal changes or goes; each entry keeps its own retention.
use crate::{
object::{AnyId, JmapObject, JmapObjectId},
request::deserialize::DeserializeArguments,
};
use jmap_tools::{Element, Key, Property};
use std::{borrow::Cow, str::FromStr};
use types::id::Id;
#[derive(Debug, Clone, Default)]
pub struct Journal;
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum JournalProperty {
Id,
Name,
Description,
Enabled,
/// `outgoing`, `incoming`, `internal` or `any`.
Direction,
/// Everyone, or chosen accounts, groups, domains and tenants.
Scope,
/// How long an entry is kept; each keeps what it was written with.
RetentionDays,
CreatedBy,
CreatedAt,
UpdatedAt,
}
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum JournalValue {
Id(Id),
}
impl Property for JournalProperty {
fn try_parse(parent: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {
// Keys inside the scope stay plain keys
match parent {
None => JournalProperty::parse(value),
Some(_) => None,
}
}
fn to_cow(&self) -> Cow<'static, str> {
match self {
JournalProperty::Id => "id",
JournalProperty::Name => "name",
JournalProperty::Description => "description",
JournalProperty::Enabled => "enabled",
JournalProperty::Direction => "direction",
JournalProperty::Scope => "scope",
JournalProperty::RetentionDays => "retentionDays",
JournalProperty::CreatedBy => "createdBy",
JournalProperty::CreatedAt => "createdAt",
JournalProperty::UpdatedAt => "updatedAt",
}
.into()
}
}
impl JournalProperty {
fn parse(value: &str) -> Option<Self> {
hashify::tiny_map!(value.as_bytes(),
b"id" => JournalProperty::Id,
b"name" => JournalProperty::Name,
b"description" => JournalProperty::Description,
b"enabled" => JournalProperty::Enabled,
b"direction" => JournalProperty::Direction,
b"scope" => JournalProperty::Scope,
b"retentionDays" => JournalProperty::RetentionDays,
b"createdBy" => JournalProperty::CreatedBy,
b"createdAt" => JournalProperty::CreatedAt,
b"updatedAt" => JournalProperty::UpdatedAt,
)
}
}
impl FromStr for JournalProperty {
type Err = ();
fn from_str(s: &str) -> Result<Self, Self::Err> {
JournalProperty::parse(s).ok_or(())
}
}
impl Element for JournalValue {
type Property = JournalProperty;
fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {
match key {
Key::Property(JournalProperty::Id) => Id::from_str(value).ok().map(JournalValue::Id),
_ => None,
}
}
fn to_cow(&self) -> Cow<'static, str> {
match self {
JournalValue::Id(id) => id.to_string().into(),
}
}
}
/// The set call's own argument: why, for the audit log.
#[derive(Debug, Clone, Default)]
pub struct JournalSetArguments {
pub reason: Option<String>,
}
impl<'de> DeserializeArguments<'de> for JournalSetArguments {
fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>
where
A: serde::de::MapAccess<'de>,
{
if key == "reason" {
self.reason = map.next_value()?;
} else {
let _ = map.next_value::<serde::de::IgnoredAny>()?;
}
Ok(())
}
}
impl JmapObject for Journal {
type Property = JournalProperty;
type Element = JournalValue;
type Id = Id;
type Filter = ();
type Comparator = ();
type GetArguments = ();
type SetArguments<'de> = JournalSetArguments;
type QueryArguments = ();
type CopyArguments = ();
type ParseArguments = ();
const ID_PROPERTY: Self::Property = JournalProperty::Id;
}
impl From<Id> for JournalValue {
fn from(id: Id) -> Self {
JournalValue::Id(id)
}
}
impl JmapObjectId for JournalValue {
fn as_id(&self) -> Option<Id> {
match self {
JournalValue::Id(id) => Some(*id),
}
}
fn as_any_id(&self) -> Option<AnyId> {
match self {
JournalValue::Id(id) => Some(AnyId::Id(*id)),
}
}
fn as_id_ref(&self) -> Option<&str> {
None
}
fn try_set_id(&mut self, new_id: AnyId) -> bool {
if let AnyId::Id(id) = new_id {
*self = JournalValue::Id(id);
true
} else {
false
}
}
}
impl JmapObjectId for JournalProperty {
fn as_id(&self) -> Option<Id> {
None
}
fn as_any_id(&self) -> Option<AnyId> {
None
}
fn as_id_ref(&self) -> Option<&str> {
None
}
fn try_set_id(&mut self, _: AnyId) -> bool {
false
}
}
+1
View File
@@ -30,6 +30,7 @@ pub mod inbuxa_inventory_snapshot; // inbuxa: personal-data catalog
pub mod inbuxa_audit; // inbuxa: the audit log
pub mod inbuxa_legal_hold; // inbuxa: legal hold
pub mod inbuxa_mail_rule; // inbuxa: DLP and mail flow rules
pub mod inbuxa_journal; // inbuxa: journaling
pub mod inbuxa_held_message; // inbuxa: mail held for review
pub mod inbuxa_hold_export; // inbuxa: legal hold exports
pub mod inbuxa_explanation; // inbuxa: "Explain this" with the local model
+3
View File
@@ -88,6 +88,9 @@ impl Response<'_> {
GetResponseMethod::MailRule(response) => {
response.eval_jptr(path, &mut results)
}
GetResponseMethod::Journal(response) => {
response.eval_jptr(path, &mut results)
}
GetResponseMethod::HeldMessage(response) => {
response.eval_jptr(path, &mut results)
}
@@ -55,6 +55,7 @@ impl Response<'_> {
GetRequestMethod::AccountLock(request) => request.resolve_references(self)?,
GetRequestMethod::LegalHold(request) => request.resolve_references(self)?,
GetRequestMethod::MailRule(request) => request.resolve_references(self)?,
GetRequestMethod::Journal(request) => request.resolve_references(self)?,
GetRequestMethod::HeldMessage(request) => request.resolve_references(self)?,
GetRequestMethod::HoldExport(request) => request.resolve_references(self)?,
GetRequestMethod::ProtocolPolicy(request) => request.resolve_references(self)?,
@@ -131,6 +132,9 @@ impl Response<'_> {
SetRequestMethod::MailRule(request) => {
request.resolve_references(self, 1, false)?
}
SetRequestMethod::Journal(request) => {
request.resolve_references(self, 1, false)?
}
SetRequestMethod::HeldMessage(request) => {
request.resolve_references(self, 1, false)?
}
+9 -1
View File
@@ -69,6 +69,8 @@ pub enum MethodObject {
// inbuxa: DLP and mail flow rules
MailRule,
HeldMessage,
// inbuxa: journaling
Journal,
TenantProtocolPolicy,
}
@@ -109,7 +111,8 @@ impl MethodObject {
| MethodObject::LegalHold
| MethodObject::HoldExport
| MethodObject::MailRule
| MethodObject::HeldMessage => Capability::Inbuxa,
| MethodObject::HeldMessage
| MethodObject::Journal => Capability::Inbuxa,
MethodObject::ProtocolPolicy => Capability::Inbuxa,
MethodObject::TenantProtocolPolicy => Capability::Inbuxa,
}
@@ -307,6 +310,8 @@ impl MethodName {
(MethodFunction::Set, MethodObject::LegalHold) => "inbuxa:LegalHold/set",
(MethodFunction::Get, MethodObject::MailRule) => "inbuxa:MailRule/get",
(MethodFunction::Set, MethodObject::MailRule) => "inbuxa:MailRule/set",
(MethodFunction::Get, MethodObject::Journal) => "inbuxa:Journal/get",
(MethodFunction::Set, MethodObject::Journal) => "inbuxa:Journal/set",
(MethodFunction::Get, MethodObject::HeldMessage) => "inbuxa:HeldMessage/get",
(MethodFunction::Set, MethodObject::HeldMessage) => "inbuxa:HeldMessage/set",
(MethodFunction::Get, MethodObject::HoldExport) => "inbuxa:HoldExport/get",
@@ -465,6 +470,8 @@ impl MethodName {
"inbuxa:LegalHold/set" => (MethodObject::LegalHold, MethodFunction::Set),
"inbuxa:MailRule/get" => (MethodObject::MailRule, MethodFunction::Get),
"inbuxa:MailRule/set" => (MethodObject::MailRule, MethodFunction::Set),
"inbuxa:Journal/get" => (MethodObject::Journal, MethodFunction::Get),
"inbuxa:Journal/set" => (MethodObject::Journal, MethodFunction::Set),
"inbuxa:HeldMessage/get" => (MethodObject::HeldMessage, MethodFunction::Get),
"inbuxa:HeldMessage/set" => (MethodObject::HeldMessage, MethodFunction::Set),
"inbuxa:HoldExport/get" => (MethodObject::HoldExport, MethodFunction::Get),
@@ -539,6 +546,7 @@ impl Display for MethodObject {
MethodObject::AccountLock => "inbuxa:AccountLock",
MethodObject::LegalHold => "inbuxa:LegalHold",
MethodObject::MailRule => "inbuxa:MailRule",
MethodObject::Journal => "inbuxa:Journal",
MethodObject::HeldMessage => "inbuxa:HeldMessage",
MethodObject::HoldExport => "inbuxa:HoldExport",
MethodObject::ProtocolPolicy => "inbuxa:ProtocolPolicy",
+2
View File
@@ -125,6 +125,7 @@ pub enum GetRequestMethod {
AccountLock(Box<GetRequest<crate::object::inbuxa_account_lock::AccountLock>>),
LegalHold(Box<GetRequest<crate::object::inbuxa_legal_hold::LegalHold>>),
MailRule(Box<GetRequest<crate::object::inbuxa_mail_rule::MailRule>>),
Journal(Box<GetRequest<crate::object::inbuxa_journal::Journal>>),
HeldMessage(Box<GetRequest<crate::object::inbuxa_held_message::HeldMessage>>),
HoldExport(Box<GetRequest<crate::object::inbuxa_hold_export::HoldExport>>),
ProtocolPolicy(Box<GetRequest<crate::object::inbuxa_protocol_policy::ProtocolPolicy>>),
@@ -163,6 +164,7 @@ pub enum SetRequestMethod<'x> {
AccountLock(Box<SetRequest<'x, crate::object::inbuxa_account_lock::AccountLock>>),
LegalHold(Box<SetRequest<'x, crate::object::inbuxa_legal_hold::LegalHold>>),
MailRule(Box<SetRequest<'x, crate::object::inbuxa_mail_rule::MailRule>>),
Journal(Box<SetRequest<'x, crate::object::inbuxa_journal::Journal>>),
HeldMessage(Box<SetRequest<'x, crate::object::inbuxa_held_message::HeldMessage>>),
HoldExport(Box<SetRequest<'x, crate::object::inbuxa_hold_export::HoldExport>>),
ProtocolPolicy(Box<SetRequest<'x, crate::object::inbuxa_protocol_policy::ProtocolPolicy>>),
+15
View File
@@ -653,6 +653,21 @@ impl<'de> Visitor<'de> for CallVisitor {
return Err(de::Error::invalid_length(1, &self));
}
},
// inbuxa: journaling
(MethodFunction::Get, MethodObject::Journal) => match seq.next_element() {
Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::Journal(value)),
Err(err) => RequestMethod::invalid(err),
Ok(None) => {
return Err(de::Error::invalid_length(1, &self));
}
},
(MethodFunction::Set, MethodObject::Journal) => match seq.next_element() {
Ok(Some(value)) => RequestMethod::Set(SetRequestMethod::Journal(value)),
Err(err) => RequestMethod::invalid(err),
Ok(None) => {
return Err(de::Error::invalid_length(1, &self));
}
},
// inbuxa: legal hold
(MethodFunction::Get, MethodObject::LegalHold) => match seq.next_element() {
Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::LegalHold(value)),
+15
View File
@@ -112,6 +112,7 @@ pub enum GetResponseMethod {
AccountLock(GetResponse<crate::object::inbuxa_account_lock::AccountLock>),
LegalHold(GetResponse<crate::object::inbuxa_legal_hold::LegalHold>),
MailRule(GetResponse<crate::object::inbuxa_mail_rule::MailRule>),
Journal(GetResponse<crate::object::inbuxa_journal::Journal>),
HeldMessage(GetResponse<crate::object::inbuxa_held_message::HeldMessage>),
HoldExport(GetResponse<crate::object::inbuxa_hold_export::HoldExport>),
ProtocolPolicy(GetResponse<crate::object::inbuxa_protocol_policy::ProtocolPolicy>),
@@ -150,6 +151,7 @@ pub enum SetResponseMethod {
AccountLock(Box<SetResponse<crate::object::inbuxa_account_lock::AccountLock>>),
LegalHold(Box<SetResponse<crate::object::inbuxa_legal_hold::LegalHold>>),
MailRule(Box<SetResponse<crate::object::inbuxa_mail_rule::MailRule>>),
Journal(Box<SetResponse<crate::object::inbuxa_journal::Journal>>),
HeldMessage(Box<SetResponse<crate::object::inbuxa_held_message::HeldMessage>>),
HoldExport(Box<SetResponse<crate::object::inbuxa_hold_export::HoldExport>>),
Explanation(Box<SetResponse<crate::object::inbuxa_explanation::Explanation>>),
@@ -841,6 +843,19 @@ impl<'x> From<SetResponse<crate::object::inbuxa_mail_rule::MailRule>> for Respon
}
}
// inbuxa: journaling
impl<'x> From<GetResponse<crate::object::inbuxa_journal::Journal>> for ResponseMethod<'x> {
fn from(value: GetResponse<crate::object::inbuxa_journal::Journal>) -> Self {
ResponseMethod::Get(GetResponseMethod::Journal(value))
}
}
impl<'x> From<SetResponse<crate::object::inbuxa_journal::Journal>> for ResponseMethod<'x> {
fn from(value: SetResponse<crate::object::inbuxa_journal::Journal>) -> Self {
ResponseMethod::Set(SetResponseMethod::Journal(Box::new(value)))
}
}
impl<'x> From<GetResponse<crate::object::inbuxa_legal_hold::LegalHold>> for ResponseMethod<'x> {
fn from(value: GetResponse<crate::object::inbuxa_legal_hold::LegalHold>) -> Self {
ResponseMethod::Get(GetResponseMethod::LegalHold(value))