Compare commits

...
22 Commits
Author SHA1 Message Date
jcoffey-dev 7e06a3b1f6 Merge pull request 'Release 2026.9.29.1' (#123) from release/2026.9.29.1-pr into main
ci / fork-checks (push) Successful in 48s
publish / version (push) Successful in 11s
ci / build (push) Successful in 30m5s
publish / publish-amd64 (push) Successful in 32m52s
publish / release (push) Successful in 10s
publish / publish-arm64 (push) Successful in 43m6s
publish / binaries (push) Successful in 36s
publish / announce (push) Successful in 10s
2026-09-29 15:40:42 +00:00
jcoffey-dev e147206e82 Release 2026.9.29.1
ci / fork-checks (pull_request) Successful in 50s
ci / build (pull_request) Successful in 13m48s
2026-09-29 08:25:13 -07:00
jcoffey-dev ca6484c356 Honor the client registration override only in setup and recovery
The recovery administrator signs in before any OAuth client is
registered, so it needs to skip the registration check. Outside
bootstrap and recovery mode, every account now signs in through a
registered client and one of its redirect URIs.
2026-09-29 08:25:13 -07:00
jcoffey-dev ffcfde0b5a Merge pull request 'Release 2026.9.29' (#121) from release/2026.9.29-pr into main
publish / version (push) Successful in 11s
ci / fork-checks (push) Successful in 1m6s
ci / build (push) Successful in 32m20s
publish / publish-amd64 (push) Successful in 39m12s
publish / release (push) Successful in 5s
publish / publish-arm64 (push) Successful in 45m10s
publish / binaries (push) Successful in 41s
publish / announce (push) Successful in 22s
2026-09-29 05:43:53 +00:00
jcoffey-dev f1f05db790 Release 2026.9.29
ci / fork-checks (pull_request) Successful in 46s
ci / build (pull_request) Successful in 4m19s
2026-09-28 22:38:59 -07:00
jcoffey-dev e1076a04b2 Merge pull request 'Journaling spec: built, and the console as built' (#120) from spec/journaling-built into main
ci / fork-checks (push) Successful in 32s
ci / build (push) Canceled after 19m51s
2026-09-29 05:23:58 +00:00
jcoffey-dev 8fc8d94bbc Merge pull request 'Journaling: a Journal link in Management › Compliance' (#119) from feature/journal-menu into main
ci / fork-checks (push) Canceled after 22s
ci / build (push) Canceled after 22s
2026-09-29 05:23:36 +00:00
jcoffey-dev 0c600a63fa Journaling spec: built, and the console as built
ci / fork-checks (pull_request) Successful in 19s
ci / build (pull_request) Successful in 8m0s
2026-09-28 22:09:07 -07:00
jcoffey-dev f78925b316 Journaling: a Journal link in Management › Compliance
ci / fork-checks (pull_request) Successful in 56s
ci / build (pull_request) Successful in 17m1s
The console's journal page (CustomComponent/Journal), after Data Loss
Prevention; the console shows it to those who may see journals.
2026-09-28 22:06:12 -07:00
jcoffey-dev 6ee7ba1b7e Merge pull request 'Logs: a total only when it's known, not the query cap' (#117) from fix/log-query-total into main
ci / fork-checks (push) Successful in 17s
ci / build (push) Canceled after 23m14s
2026-09-29 05:00:18 +00:00
jcoffey-dev a992caf810 Merge pull request 'Journaling: search, read and export over JMAP, and the chain check' (#118) from feature/journal-search into main
ci / fork-checks (push) Successful in 17s
ci / build (push) Canceled after 6m46s
2026-09-29 04:53:28 +00:00
jcoffey-dev daa486f7e7 Journaling: search, read and export over JMAP, and the chain check
ci / fork-checks (pull_request) Successful in 16s
ci / build (pull_request) Successful in 8m0s
Phase 4 of the journaling spec.

- inbuxa:JournalEntry/query and /get (sysJournalSearch): filter by time,
  sender, recipient, either, direction, subject words, Message-ID and
  journal, newest first; the whole report only when asked for.
- inbuxa:JournalExport/set (sysJournalExport): a reason is required; a
  ZIP of the matching reports with manifest.csv, exceptions.csv and
  manifest.sha256, up to 10,000 reports and 1 GB.
- inbuxa:JournalVerification/set (sysJournalGet): chains and reports
  rechecked.
- Every search, listing, read, export and check is written to the audit
  log before anything is returned, with existing actions only.
- Catalog entries for the three objects; spec as-built notes.

journal_tests: administrators can't search; a Compliance Officer searches,
lists, reads a report, exports (reason required) and checks the chain;
the officer can't change journals; each of those is in the audit log.
2026-09-28 21:45:09 -07:00
jcoffey-dev 9c29fb2bea Logs: a total only when it's known, not the query cap
ci / fork-checks (pull_request) Successful in 55s
ci / build (pull_request) Successful in 17m3s
2026-09-28 21:42:53 -07:00
jcoffey-dev abd5811420 Merge pull request 'Journaling: outside archives, and Journal it in mail flow rules' (#116) from feature/journal-archive into main
ci / fork-checks (push) Successful in 50s
ci / build (push) Canceled after 19m30s
2026-09-29 04:33:59 +00:00
jcoffey-dev 80051539d5 Merge pull request 'Security to-do list: accepted items, kept on the server' (#114) from feature/security-acceptances into main
ci / fork-checks (push) Canceled after 11s
ci / build (push) Canceled after 10s
2026-09-29 04:33:46 +00:00
jcoffey-dev 64550ebbd0 Journaling: outside archives, and Journal it in mail flow rules
ci / fork-checks (pull_request) Successful in 1m19s
ci / build (pull_request) Successful in 5m44s
Phase 3 of the journaling spec.

- A journal's destination: builtIn (true for journals stored before) and
  archiveAddress, at least one. Reports to an archive are queued from the
  empty sender, one per address, flagged so they're never journaled.
- A pending record per report. When the queue lets go of one without
  delivering it (refused, expired, deleted), it becomes its own entry in
  the built-in journal under the sending journals' retention, the
  journal's archiveFailures (count, last time, reason) goes up, and the
  audit log records it; if that can't be written it stays queued.
- Journal it: a rule action naming a journal, on mail flow rules and
  beside a DLP rule's block, warn or hold. A journal whose scope chooses
  nobody takes only what rules send it.
- The report lists recipients a rule added or redirected to under
  "Added by rule", by rule name.
- A rule's route is cleared between messages in one SMTP session, with the
  new journal marks; a second message used to keep the first one's route.

tests/src/system/journal.rs: destination validation, a rule-only journal
fed by a rule that also adds a recipient, an unreachable archive's report
kept in the built-in journal with the failure counted, a report delivered
to an archive here and not journaled itself.
2026-09-28 21:27:44 -07:00
jcoffey-dev eea96e8674 Security to-do list: accepted items, kept on the server
ci / fork-checks (pull_request) Successful in 19s
ci / build (pull_request) Successful in 8m5s
2026-09-28 21:22:43 -07:00
jcoffey-dev 4c5583e725 Merge pull request 'Journaling: capture at the queue, the built-in journal, retention' (#115) from feature/journal-capture into main
ci / fork-checks (push) Successful in 46s
ci / build (push) Canceled after 22m4s
2026-09-29 04:11:40 +00:00
jcoffey-dev 441ad0b18e Journaling: capture at the queue, the built-in journal, retention
ci / fork-checks (pull_request) Successful in 41s
ci / build (pull_request) Successful in 8m2s
Phase 2 of the journaling spec.

- A copy of each message is taken in MessageWrapper::queue, after DLP and
  transport rules, for every enabled journal that takes it (direction and
  scope: everyone, or accounts, groups, domains, tenants). If the copy
  can't be taken the message isn't queued (temporary failure).
- The journal report: the envelope one field a line (sender, To, Cc, Bcc
  from the envelope, list members from their ORCPT, direction, held for
  review), then the queued message byte for byte as message/rfc822.
- The built-in journal under J in the inbuxa subspace: one chain per node
  whose links name each entry by SHA-256, so entries can expire out of
  chain order; purge leaves a marker, and verify catches an entry changed
  or removed early and a report that doesn't match.
- Retention per journal (30 to 3650 days); an entry keeps what it was
  written with. The daily maintenance purges what's due, keeping entries
  whose people a legal hold covers (deleted accounts a hold keeps too),
  and records the counts in the audit log.
- inbuxa:Journal get/set, audited by the request layer. Permissions
  680-683: administrators see and change journals; the Compliance Officer
  sees, searches and exports. Whoever changes journals may grant search and
  export without holding them, so officers can still be appointed.
- Catalog entries (inbuxa:Journal, source "journal"); spec as-built notes.

tests/src/system/journal.rs: validation, internal mail with a Bcc,
outgoing into two journals, incoming over LMTP, the report and its
original, tamper and early removal caught, hold-aware purge, retention
changes leave entries alone, disabled and removed journals take nothing.
2026-09-28 20:46:04 -07:00
jcoffey-dev 792ff9d1ee Merge pull request 'Spec: journaling' (#113) from spec/journaling into main
ci / fork-checks (push) Successful in 48s
ci / build (push) Canceled after 54m15s
2026-09-29 03:17:22 +00:00
jcoffey-dev af49e94d97 Journaling spec: approved, with the answers
ci / fork-checks (pull_request) Successful in 32s
ci / build (pull_request) Successful in 3m56s
2026-09-28 20:13:15 -07:00
jcoffey-dev 37c00b609c Spec: journaling
ci / fork-checks (pull_request) Successful in 47s
ci / build (pull_request) Successful in 23m6s
2026-09-28 19:45:37 -07:00
51 changed files with 6837 additions and 25 deletions
+19 -1
View File
@@ -165,6 +165,14 @@ impl AccessToken {
mut requested_permissions: Permissions,
) -> Result<(), Vec<Permission>> {
requested_permissions.difference(self.permissions_bits());
// inbuxa: journaling, JR-18: whoever sets up journals may give
// others (or, through a role, themselves) the reading of them,
// which administrators don't hold by default; the role change is
// in the audit log
if self.has_permission(Permission::SysJournalUpdate) {
requested_permissions.clear(Permission::SysJournalSearch as usize);
requested_permissions.clear(Permission::SysJournalExport as usize);
}
if requested_permissions.is_empty() {
Ok(())
} else {
@@ -304,9 +312,19 @@ impl Default for DefaultPermissions {
| Permission::SysDlpPolicyGet
| Permission::SysDlpPolicyUpdate
| Permission::SysDlpReviewGet
| Permission::SysDlpReviewUpdate => {
| Permission::SysDlpReviewUpdate
// inbuxa: every security check is server-wide (security
// to-do list spec)
| Permission::SysSecurityAccept => {
default.superuser.push(permission);
}
// inbuxa: journals are the server's; administrators set them
// up but read what's journaled only if granted it
// (journaling spec, JR-18, settled answer 5)
Permission::SysJournalGet | Permission::SysJournalUpdate => {
default.superuser.push(permission);
}
Permission::SysJournalSearch | Permission::SysJournalExport => {}
// inbuxa: AL-12: tenant administrators lock and delegate
// within their tenant
Permission::SysAccountLockGet
@@ -69,6 +69,10 @@ const OFFICER: &[Permission] = &[
Permission::SysDlpPolicyGet,
Permission::SysDlpReviewGet,
Permission::SysDlpReviewUpdate,
// journaling spec, JR-18: see journals, search and export them
Permission::SysJournalGet,
Permission::SysJournalSearch,
Permission::SysJournalExport,
];
/// What a tenant's officer holds besides [`READS`].
@@ -31,7 +31,8 @@ use types::id::Id;
/// Granted to the default administrator roles: "Explain this"
/// (ai-explain spec, EX-4: superuser by default), the audit log, account
/// locks and legal holds (audit-hold-lock spec, AU-9, AL-12, LH-13), and
/// the data inventory (personal-data catalog spec).
/// the data inventory (personal-data catalog spec), and accepting security
/// to-do items (security to-do list spec).
const ADMIN_GRANTS: &[Permission] = &[
Permission::SysAiExplain,
Permission::SysAuditGet,
@@ -52,6 +53,9 @@ const ADMIN_GRANTS: &[Permission] = &[
Permission::SysDlpPolicyUpdate,
Permission::SysDlpReviewGet,
Permission::SysDlpReviewUpdate,
Permission::SysJournalGet,
Permission::SysJournalUpdate,
Permission::SysSecurityAccept,
];
/// Granted to the server-level Compliance Officer role once it exists:
@@ -61,6 +65,10 @@ const OFFICER_GRANTS: &[Permission] = &[
Permission::SysDlpPolicyGet,
Permission::SysDlpReviewGet,
Permission::SysDlpReviewUpdate,
// journaling spec, JR-18: see journals, search and export them
Permission::SysJournalGet,
Permission::SysJournalSearch,
Permission::SysJournalExport,
];
/// Granted to the default tenant administrator roles: reading and exporting
+120
View File
@@ -0,0 +1,120 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! Reports on their way to an outside archive (JR-7). Keys, after `J`:
//!
//! - `o` + the report's queue id: what goes into the built-in journal if
//! the archive never takes the report, as JSON. Cleared once it's
//! delivered or kept.
//! - `w` + journal id (u32): how often that journal's archive didn't take a
//! report, and the last time and reason, for the console's warning.
use super::{FEATURE, Json, entries::Entry};
use serde::{Deserialize as SerdeDeserialize, Serialize as SerdeSerialize};
use store::{
SUBSPACE_INBUXA, Serialize, Store, ValueKey,
write::{AnyClass, BatchBuilder, ValueClass},
};
use trc::AddContext;
const KIND_PENDING: u8 = b'o';
const KIND_FAILURES: u8 = b'w';
/// A report queued to an archive.
#[derive(Debug, Clone, PartialEq, Eq, SerdeSerialize, SerdeDeserialize)]
#[serde(rename_all = "camelCase")]
pub struct Pending {
pub address: String,
/// The entry, should the archive not take it: its own, with the
/// sending journals' retention, whatever else the built-in journal has.
pub entry: Entry,
}
/// How a journal's archive has been taking its reports.
#[derive(Debug, Clone, Default, PartialEq, Eq, SerdeSerialize, SerdeDeserialize)]
#[serde(rename_all = "camelCase")]
pub struct Failures {
pub count: u64,
/// Seconds.
pub last_at: u64,
pub last_reason: String,
}
fn class(kind: u8, id: &[u8]) -> ValueClass {
let mut key = Vec::with_capacity(2 + id.len());
key.push(FEATURE);
key.push(kind);
key.extend_from_slice(id);
ValueClass::Any(AnyClass {
subspace: SUBSPACE_INBUXA,
key,
})
}
pub async fn set_pending(data: &Store, queue_id: u64, pending: &Pending) -> trc::Result<()> {
let mut batch = BatchBuilder::new();
batch.set(
class(KIND_PENDING, &queue_id.to_be_bytes()),
Json(pending).serialize()?,
);
data.write(batch.build_all())
.await
.caused_by(trc::location!())?;
Ok(())
}
pub async fn pending(data: &Store, queue_id: u64) -> trc::Result<Option<Pending>> {
Ok(data
.get_value::<Json<Pending>>(ValueKey::from(class(KIND_PENDING, &queue_id.to_be_bytes())))
.await
.caused_by(trc::location!())?
.map(|Json(pending)| pending))
}
pub async fn clear_pending(data: &Store, queue_id: u64) -> trc::Result<()> {
let mut batch = BatchBuilder::new();
batch.clear(class(KIND_PENDING, &queue_id.to_be_bytes()));
data.write(batch.build_all())
.await
.caused_by(trc::location!())?;
Ok(())
}
pub async fn failures(data: &Store, journal_id: u32) -> trc::Result<Failures> {
Ok(data
.get_value::<Json<Failures>>(ValueKey::from(class(
KIND_FAILURES,
&journal_id.to_be_bytes(),
)))
.await
.caused_by(trc::location!())?
.map(|Json(failures)| failures)
.unwrap_or_default())
}
/// Counts one report an archive didn't take, for each of `journals`.
pub async fn record_failure(
data: &Store,
journals: &[u32],
at: u64,
reason: &str,
) -> trc::Result<()> {
for journal_id in journals {
let mut failures = failures(data, *journal_id).await?;
failures.count += 1;
failures.last_at = at;
failures.last_reason = reason.chars().take(500).collect();
let mut batch = BatchBuilder::new();
batch.set(
class(KIND_FAILURES, &journal_id.to_be_bytes()),
Json(&failures).serialize()?,
);
data.write(batch.build_all())
.await
.caused_by(trc::location!())?;
}
Ok(())
}
+869
View File
@@ -0,0 +1,869 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! The built-in journal (JR-5, JR-6, JR-13). Keys, after `J`:
//!
//! - `e` + node + seq: a chain link: its seq, the hash of the link before
//! it, and the SHA-256 of its entry. One chain per node, as the audit log
//! keeps (AU-6), but a link names its entry by hash instead of holding it,
//! so an entry can go at the end of its own retention without breaking
//! the chain: entries don't expire in chain order.
//! - `c` + node + seq: the entry, as JSON; its bytes are what the link's
//! hash names.
//! - `p` + node + seq: when an entry past its retention was purged. A link
//! whose entry is gone without this marker is a broken chain.
//! - `t` + time + node + seq: the time index, for search.
//! - `x` + expiry + node + seq: the expiry index, for purge.
//! - `h` + node: the chain's head: its hash, then its seq as the last eight
//! bytes, which each append asserts.
//! - `f` + node: where the chain starts after purged links at its start
//! were cleared, and the hash the first kept link names.
//!
//! The report itself is a blob, kept by a temporary link that lasts until
//! its entry is purged. Nothing here changes or removes an entry before
//! its time; nothing in JMAP can.
use super::{Direction, FEATURE, Json};
use crate::hold::HELD_UNTIL;
use serde::{Deserialize as SerdeDeserialize, Serialize as SerdeSerialize};
use sha2::{Digest, Sha256};
use std::fmt;
use store::{
BlobStore, Deserialize, IterateParams, SUBSPACE_INBUXA, Serialize, Store, ValueKey,
write::{AnyClass, BatchBuilder, BlobLink, BlobOp, ValueClass, assert::AssertValue},
};
use tokio::sync::Mutex;
use trc::AddContext;
use types::blob_hash::BlobHash;
const KIND_LINK: u8 = b'e';
const KIND_CONTENT: u8 = b'c';
const KIND_PURGED: u8 = b'p';
const KIND_TIME: u8 = b't';
const KIND_EXPIRY: u8 = b'x';
const KIND_HEAD: u8 = b'h';
const KIND_FLOOR: u8 = b'f';
const APPEND_ATTEMPTS: usize = 5;
/// Entries purged per batch.
const PURGE_BATCH: usize = 100;
/// Where one entry sits: its node's chain and its place in it.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub struct EntryId {
pub node: u64,
pub seq: u64,
}
impl EntryId {
/// As one number, for JMAP ids: the node in the top 16 bits.
pub fn to_u64(&self) -> u64 {
(self.node << 48) | (self.seq & ((1 << 48) - 1))
}
pub fn from_u64(id: u64) -> Self {
EntryId {
node: id >> 48,
seq: id & ((1 << 48) - 1),
}
}
}
impl fmt::Display for EntryId {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}-{}", self.node, self.seq)
}
}
/// One journaled message (JR-5).
#[derive(Debug, Clone, PartialEq, Eq, SerdeSerialize, SerdeDeserialize)]
#[serde(rename_all = "camelCase")]
pub struct Entry {
pub queue_id: u64,
/// Seconds.
pub at: u64,
pub direction: Direction,
pub sender: String,
pub authenticated: bool,
pub recipients: Vec<String>,
pub subject: String,
pub message_id: String,
/// The people here on either side, whose holds keep the entry.
pub accounts: Vec<u32>,
pub tenants: Vec<u32>,
/// The journals that took it.
pub journals: Vec<u32>,
pub held: bool,
/// The report's blob, hex.
pub blob: String,
pub size: u64,
/// SHA-256 of the report, hex.
pub sha256: String,
/// Seconds.
pub expires_at: u64,
}
impl Entry {
pub fn blob_hash(&self) -> Option<BlobHash> {
let bytes = unhex(&self.blob)?;
BlobHash::try_from_hash_slice(&bytes).ok()
}
}
#[derive(Debug, Clone, SerdeSerialize, SerdeDeserialize)]
#[serde(rename_all = "camelCase")]
struct Link {
seq: u64,
prev: String,
content: String,
}
#[derive(Debug, Clone, Default, PartialEq, SerdeSerialize, SerdeDeserialize)]
struct Floor {
seq: u64,
prev: String,
}
#[derive(Debug, Clone, Default, PartialEq)]
struct Head {
seq: u64,
hash: String,
}
impl Head {
fn to_bytes(&self) -> Vec<u8> {
let mut bytes = self.hash.as_bytes().to_vec();
bytes.extend_from_slice(&self.seq.to_be_bytes());
bytes
}
}
impl Deserialize for Head {
fn deserialize(bytes: &[u8]) -> trc::Result<Self> {
let split = bytes.len().checked_sub(8).ok_or_else(|| {
trc::StoreEvent::DataCorruption
.into_err()
.details("Invalid journal chain head")
})?;
Ok(Head {
seq: u64::from_be_bytes(bytes[split..].try_into().unwrap()),
hash: String::from_utf8_lossy(&bytes[..split]).into_owned(),
})
}
}
struct Raw(Vec<u8>);
impl Deserialize for Raw {
fn deserialize(bytes: &[u8]) -> trc::Result<Self> {
Ok(Raw(bytes.to_vec()))
}
}
fn class(kind: u8, parts: &[u64]) -> ValueClass {
let mut key = Vec::with_capacity(2 + parts.len() * 8);
key.push(FEATURE);
key.push(kind);
for part in parts {
key.extend_from_slice(&part.to_be_bytes());
}
ValueClass::Any(AnyClass {
subspace: SUBSPACE_INBUXA,
key,
})
}
fn key(kind: u8, parts: &[u64]) -> ValueKey<ValueClass> {
ValueKey::from(class(kind, parts))
}
/// Where an entry's content is kept, for tests that check tampering shows.
pub fn content_key(id: EntryId) -> ValueKey<ValueClass> {
key(KIND_CONTENT, &[id.node, id.seq])
}
/// The numbers after the kind byte, from the key's tail.
fn parse_key(key: &[u8], kind: u8, parts: usize) -> Option<Vec<u64>> {
let len = 2 + parts * 8;
let tail = key.get(key.len().checked_sub(len)?..)?;
(tail[0] == FEATURE && tail[1] == kind).then_some(())?;
Some(
tail[2..]
.chunks_exact(8)
.map(|chunk| u64::from_be_bytes(chunk.try_into().unwrap()))
.collect(),
)
}
pub fn hex(bytes: &[u8]) -> String {
bytes.iter().map(|b| format!("{b:02x}")).collect()
}
fn unhex(value: &str) -> Option<Vec<u8>> {
(value.len() % 2 == 0).then_some(())?;
(0..value.len())
.step_by(2)
.map(|i| u8::from_str_radix(value.get(i..i + 2)?, 16).ok())
.collect()
}
pub fn sha256(bytes: &[u8]) -> String {
hex(&Sha256::digest(bytes))
}
async fn head(data: &Store, node: u64) -> trc::Result<Option<Head>> {
data.get_value::<Head>(key(KIND_HEAD, &[node]))
.await
.caused_by(trc::location!())
}
async fn floor(data: &Store, node: u64) -> trc::Result<Floor> {
Ok(data
.get_value::<Json<Floor>>(key(KIND_FLOOR, &[node]))
.await
.caused_by(trc::location!())?
.map(|Json(floor)| floor)
.unwrap_or(Floor {
seq: 1,
prev: String::new(),
}))
}
async fn nodes(data: &Store) -> trc::Result<Vec<u64>> {
let mut nodes = Vec::new();
data.iterate(
IterateParams::new(key(KIND_HEAD, &[0]), key(KIND_HEAD, &[u64::MAX])).no_values(),
|key, _| {
if let Some(parts) = parse_key(key, KIND_HEAD, 1) {
nodes.push(parts[0]);
}
Ok(true)
},
)
.await
.caused_by(trc::location!())?;
Ok(nodes)
}
/// Lines up this process's appends; the store's assert settles the rest.
static APPENDING: Mutex<()> = Mutex::const_new(());
/// Adds an entry to this node's chain, and links its report's blob (already
/// written) until the entry is purged. An error means nothing was written.
pub async fn append(data: &Store, node: u64, entry: &Entry) -> trc::Result<EntryId> {
let blob = entry.blob_hash().ok_or_else(|| {
trc::StoreEvent::UnexpectedError
.into_err()
.details("Journal entry without a blob")
})?;
let content = Json(entry).serialize()?;
let content_hash = sha256(&content);
let _appending = APPENDING.lock().await;
let mut attempt = 0;
loop {
attempt += 1;
let current = head(data, node).await?;
let (seq, prev) = current
.as_ref()
.map_or((1, String::new()), |head| (head.seq + 1, head.hash.clone()));
let link = Json(&Link {
seq,
prev,
content: content_hash.clone(),
})
.serialize()?;
let new_head = Head {
seq,
hash: sha256(&link),
};
let mut batch = BatchBuilder::new();
batch.assert_value(
class(KIND_HEAD, &[node]),
current.map_or(AssertValue::None, |head| AssertValue::U64(head.seq)),
);
batch
.set(class(KIND_LINK, &[node, seq]), link)
.set(class(KIND_CONTENT, &[node, seq]), content.clone())
.set(class(KIND_TIME, &[entry.at, node, seq]), vec![])
.set(class(KIND_EXPIRY, &[entry.expires_at, node, seq]), vec![])
.set(class(KIND_HEAD, &[node]), new_head.to_bytes())
.set(
BlobOp::Link {
hash: blob.clone(),
to: BlobLink::Temporary { until: HELD_UNTIL },
},
vec![],
)
.set(BlobOp::Commit { hash: blob.clone() }, vec![]);
match data.write(batch.build_all()).await {
Ok(_) => return Ok(EntryId { node, seq }),
Err(err)
if attempt < APPEND_ATTEMPTS
&& matches!(
err.as_ref(),
trc::EventType::Store(trc::StoreEvent::AssertValueFailed)
) => {}
Err(err) => return Err(err.caused_by(trc::location!())),
}
}
}
/// One entry, unless it was purged.
pub async fn get(data: &Store, id: EntryId) -> trc::Result<Option<Entry>> {
Ok(data
.get_value::<Json<Entry>>(key(KIND_CONTENT, &[id.node, id.seq]))
.await
.caused_by(trc::location!())?
.map(|Json(entry)| entry))
}
/// Entries written in `[after, before)` (seconds), newest first, up to
/// `limit`.
pub async fn list(
data: &Store,
after: u64,
before: u64,
limit: usize,
) -> trc::Result<Vec<(EntryId, Entry)>> {
let mut ids = Vec::new();
data.iterate(
IterateParams::new(
key(KIND_TIME, &[after, 0, 0]),
key(KIND_TIME, &[before.saturating_sub(1), u64::MAX, u64::MAX]),
)
.descending()
.no_values(),
|key, _| {
if let Some(parts) = parse_key(key, KIND_TIME, 3) {
ids.push(EntryId {
node: parts[1],
seq: parts[2],
});
}
Ok(ids.len() < limit)
},
)
.await
.caused_by(trc::location!())?;
let mut out = Vec::with_capacity(ids.len());
for id in ids {
if let Some(entry) = get(data, id).await? {
out.push((id, entry));
}
}
Ok(out)
}
/// Most results one search page returns.
pub const MAX_QUERY_LIMIT: usize = 500;
/// A search of the journal (JR-15): conditions that must all hold.
#[derive(Debug, Clone, Default, PartialEq, Eq, SerdeSerialize)]
#[serde(rename_all = "camelCase")]
pub struct Filter {
/// From this time on, in seconds.
#[serde(skip_serializing_if = "Option::is_none")]
pub after: Option<u64>,
/// Before this time, in seconds.
#[serde(skip_serializing_if = "Option::is_none")]
pub before: Option<u64>,
/// Part of the sender's address, ignoring case.
#[serde(skip_serializing_if = "Option::is_none")]
pub sender: Option<String>,
/// Part of any recipient's address, ignoring case.
#[serde(skip_serializing_if = "Option::is_none")]
pub recipient: Option<String>,
/// Part of the sender's or any recipient's address.
#[serde(skip_serializing_if = "Option::is_none")]
pub address: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub direction: Option<Direction>,
/// Words that must all appear in the subject, ignoring case.
#[serde(skip_serializing_if = "Option::is_none")]
pub text: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub message_id: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub journal_id: Option<u32>,
}
impl Filter {
pub fn matches(&self, entry: &Entry) -> bool {
let has = |value: &str, part: &str| value.to_lowercase().contains(&part.to_lowercase());
self.after.is_none_or(|after| entry.at >= after)
&& self.before.is_none_or(|before| entry.at < before)
&& self.sender.as_deref().is_none_or(|s| has(&entry.sender, s))
&& self
.recipient
.as_deref()
.is_none_or(|r| entry.recipients.iter().any(|a| has(a, r)))
&& self
.address
.as_deref()
.is_none_or(|a| has(&entry.sender, a) || entry.recipients.iter().any(|r| has(r, a)))
&& self
.direction
.is_none_or(|d| d == Direction::Any || d == entry.direction)
&& self.text.as_deref().is_none_or(|text| {
let subject = entry.subject.to_lowercase();
text.to_lowercase()
.split_whitespace()
.all(|word| subject.contains(word))
})
&& self.message_id.as_deref().is_none_or(|id| {
entry.message_id.trim_matches(['<', '>']) == id.trim_matches(['<', '>'])
})
&& self.journal_id.is_none_or(|j| entry.journals.contains(&j))
}
}
/// Entries matching `filter`, newest first: a page from `position`, up to
/// `limit`, and, when asked, how many match in all.
pub async fn query(
data: &Store,
filter: &Filter,
position: usize,
limit: usize,
count_all: bool,
) -> trc::Result<(Vec<EntryId>, usize)> {
let after = filter.after.unwrap_or(0);
let before = filter.before.unwrap_or(u64::MAX);
let mut ids = Vec::new();
data.iterate(
IterateParams::new(
key(KIND_TIME, &[after, 0, 0]),
key(KIND_TIME, &[before.saturating_sub(1), u64::MAX, u64::MAX]),
)
.descending()
.no_values(),
|key, _| {
if let Some(parts) = parse_key(key, KIND_TIME, 3) {
ids.push(EntryId {
node: parts[1],
seq: parts[2],
});
}
Ok(true)
},
)
.await
.caused_by(trc::location!())?;
let mut page = Vec::new();
let mut total = 0;
for id in ids {
let Some(entry) = get(data, id).await? else {
continue;
};
if !filter.matches(&entry) {
continue;
}
if total >= position && page.len() < limit {
page.push(id);
}
total += 1;
if !count_all && page.len() >= limit {
break;
}
}
Ok((page, total))
}
/// What a purge did.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct Purged {
pub removed: usize,
/// Past their time, kept for a legal hold.
pub kept_for_hold: usize,
}
/// Removes entries past their retention (JR-13), except those `held` keeps:
/// the entry, its indexes and its blob's link go; the chain link stays,
/// with a purge marker. Then each chain's start moves past purged links.
pub async fn purge(
data: &Store,
now: u64,
held: impl Fn(&Entry) -> bool + Sync + Send,
) -> trc::Result<Purged> {
let mut due = Vec::new();
data.iterate(
IterateParams::new(
key(KIND_EXPIRY, &[0, 0, 0]),
key(KIND_EXPIRY, &[now, u64::MAX, u64::MAX]),
)
.ascending()
.no_values(),
|key, _| {
if let Some(parts) = parse_key(key, KIND_EXPIRY, 3) {
due.push((
parts[0],
EntryId {
node: parts[1],
seq: parts[2],
},
));
}
Ok(due.len() < 100_000)
},
)
.await
.caused_by(trc::location!())?;
let mut purged = Purged::default();
for chunk in due.chunks(PURGE_BATCH) {
let mut batch = BatchBuilder::new();
for (expires_at, id) in chunk {
let parts = [id.node, id.seq];
let Some(entry) = get(data, *id).await? else {
// Its entry is already gone: only the index is left
batch.clear(class(KIND_EXPIRY, &[*expires_at, id.node, id.seq]));
continue;
};
if held(&entry) {
purged.kept_for_hold += 1;
continue;
}
batch
.clear(class(KIND_CONTENT, &parts))
.clear(class(KIND_TIME, &[entry.at, id.node, id.seq]))
.clear(class(KIND_EXPIRY, &[*expires_at, id.node, id.seq]))
.set(class(KIND_PURGED, &parts), now.to_be_bytes().to_vec());
if let Some(blob) = entry.blob_hash() {
batch.clear(BlobOp::Link {
hash: blob,
to: BlobLink::Temporary { until: HELD_UNTIL },
});
}
purged.removed += 1;
}
if !batch.is_empty() {
data.write(batch.build_all())
.await
.caused_by(trc::location!())?;
}
}
for node in nodes(data).await? {
advance_floor(data, node).await?;
}
Ok(purged)
}
/// Clears the purged links at the start of a node's chain, recording where
/// it now starts and the hash that start names.
async fn advance_floor(data: &Store, node: u64) -> trc::Result<()> {
let start = floor(data, node).await?;
let mut cleared: Vec<u64> = Vec::new();
let mut next = start.clone();
let mut purged_seqs = Vec::new();
data.iterate(
IterateParams::new(
key(KIND_PURGED, &[node, start.seq]),
key(KIND_PURGED, &[node, u64::MAX]),
)
.ascending()
.no_values(),
|key, _| {
if let Some(parts) = parse_key(key, KIND_PURGED, 2) {
purged_seqs.push(parts[1]);
}
Ok(purged_seqs.len() < 100_000)
},
)
.await
.caused_by(trc::location!())?;
for seq in purged_seqs {
if seq != next.seq {
break;
}
let Some(Raw(link)) = data
.get_value::<Raw>(key(KIND_LINK, &[node, seq]))
.await
.caused_by(trc::location!())?
else {
break;
};
next = Floor {
seq: seq + 1,
prev: sha256(&link),
};
cleared.push(seq);
}
if cleared.is_empty() {
return Ok(());
}
// The floor moves first: a run cut short leaves links before it, which
// the next run clears, never a chain that looks broken
let mut batch = BatchBuilder::new();
batch.set(class(KIND_FLOOR, &[node]), Json(&next).serialize()?);
data.write(batch.build_all())
.await
.caused_by(trc::location!())?;
for chunk in cleared.chunks(PURGE_BATCH) {
let mut batch = BatchBuilder::new();
for seq in chunk {
batch
.clear(class(KIND_LINK, &[node, *seq]))
.clear(class(KIND_PURGED, &[node, *seq]));
}
data.write(batch.build_all())
.await
.caused_by(trc::location!())?;
}
Ok(())
}
/// One node's chain, as [`verify`] found it.
#[derive(Debug, Clone, PartialEq, Eq, SerdeSerialize)]
#[serde(rename_all = "camelCase")]
pub struct ChainReport {
pub node: u64,
pub entries: u64,
pub purged: u64,
pub first_seq: u64,
pub last_seq: u64,
#[serde(skip_serializing_if = "Option::is_none")]
pub broken_at: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub reason: Option<String>,
}
/// Rechecks every node's chain (JR-6): each link names the hash of the one
/// before it, seqs run without gaps, the head matches the last link, each
/// entry hashes to what its link names or was purged, and, with `blobs`,
/// each report is there and hashes to what its entry names.
pub async fn verify(data: &Store, blobs: Option<&BlobStore>) -> trc::Result<Vec<ChainReport>> {
let mut reports = Vec::new();
for node in nodes(data).await? {
let start = floor(data, node).await?;
let head = head(data, node).await?.unwrap_or_default();
let mut report = ChainReport {
node,
entries: 0,
purged: 0,
first_seq: start.seq,
last_seq: start.seq.saturating_sub(1),
broken_at: None,
reason: None,
};
let mut links = Vec::new();
data.iterate(
IterateParams::new(
key(KIND_LINK, &[node, start.seq]),
key(KIND_LINK, &[node, u64::MAX]),
)
.ascending(),
|key, value| {
if let Some(parts) = parse_key(key, KIND_LINK, 2) {
links.push((parts[1], value.to_vec()));
}
Ok(true)
},
)
.await
.caused_by(trc::location!())?;
let mut expected_seq = start.seq;
let mut expected_prev = start.prev.clone();
for (seq, bytes) in links {
let broken = |report: &mut ChainReport, reason: &str| {
report.broken_at = Some(EntryId { node, seq }.to_string());
report.reason = Some(reason.to_string());
};
let Ok(Json(link)) = Json::<Link>::deserialize(&bytes) else {
broken(&mut report, "The link can't be read.");
break;
};
if seq != expected_seq || link.seq != seq {
report.broken_at = Some(EntryId { node, seq }.to_string());
report.reason = Some(format!(
"Entry {expected_seq} is missing; the next one found is {seq}."
));
break;
}
if link.prev != expected_prev {
broken(
&mut report,
"The link doesn't follow from the one before it: one of them was changed.",
);
break;
}
match data
.get_value::<Raw>(key(KIND_CONTENT, &[node, seq]))
.await
.caused_by(trc::location!())?
{
Some(Raw(content)) => {
if sha256(&content) != link.content {
broken(&mut report, "The entry was changed after it was written.");
break;
}
if let Some(blobs) = blobs {
let Ok(Json(entry)) = Json::<Entry>::deserialize(&content) else {
broken(&mut report, "The entry can't be read.");
break;
};
let report_bytes = match entry.blob_hash() {
Some(hash) => blobs
.get_blob(hash.as_slice(), 0..usize::MAX)
.await
.caused_by(trc::location!())?,
None => None,
};
match report_bytes {
Some(bytes) if sha256(&bytes) == entry.sha256 => {}
Some(_) => {
broken(&mut report, "The report doesn't match its entry.");
break;
}
None => {
broken(&mut report, "The report is missing.");
break;
}
}
}
report.entries += 1;
}
None => {
if data
.get_value::<Raw>(key(KIND_PURGED, &[node, seq]))
.await
.caused_by(trc::location!())?
.is_none()
{
broken(&mut report, "The entry was removed before its time.");
break;
}
report.purged += 1;
}
}
expected_prev = sha256(&bytes);
expected_seq = seq + 1;
report.last_seq = seq;
}
if report.broken_at.is_none()
&& (head.seq != report.last_seq
|| (report.last_seq >= report.first_seq && head.hash != expected_prev))
{
report.broken_at = Some(
EntryId {
node,
seq: report.last_seq,
}
.to_string(),
);
report.reason = Some(
"The chain's recorded end doesn't match its last link: entries were removed \
or changed at the end."
.into(),
);
}
reports.push(report);
}
Ok(reports)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn keys_read_back() {
let ValueClass::Any(any) = class(KIND_EXPIRY, &[5, 3, 9]) else {
panic!()
};
assert_eq!(parse_key(&any.key, KIND_EXPIRY, 3), Some(vec![5, 3, 9]));
let mut with_subspace = vec![SUBSPACE_INBUXA];
with_subspace.extend_from_slice(&any.key);
assert_eq!(
parse_key(&with_subspace, KIND_EXPIRY, 3),
Some(vec![5, 3, 9])
);
assert_eq!(parse_key(&any.key, KIND_TIME, 3), None);
}
#[test]
fn filters_match() {
let entry = Entry {
queue_id: 1,
at: 100,
direction: Direction::Outgoing,
sender: "[email protected]".into(),
authenticated: true,
recipients: vec!["[email protected]".into()],
subject: "Q3 figures, final".into(),
message_id: "<[email protected]>".into(),
accounts: vec![3],
tenants: vec![],
journals: vec![2],
held: false,
blob: String::new(),
size: 0,
sha256: String::new(),
expires_at: 0,
};
let yes = |f: Filter| assert!(f.matches(&entry), "{f:?}");
let no = |f: Filter| assert!(!f.matches(&entry), "{f:?}");
yes(Filter::default());
yes(Filter {
sender: Some("alice@".into()),
..Default::default()
});
yes(Filter {
address: Some("BANK".into()),
..Default::default()
});
yes(Filter {
text: Some("final q3".into()),
..Default::default()
});
yes(Filter {
message_id: Some("[email protected]".into()),
..Default::default()
});
yes(Filter {
direction: Some(Direction::Any),
..Default::default()
});
no(Filter {
direction: Some(Direction::Incoming),
..Default::default()
});
no(Filter {
recipient: Some("alice".into()),
..Default::default()
});
no(Filter {
before: Some(100),
..Default::default()
});
yes(Filter {
after: Some(100),
journal_id: Some(2),
..Default::default()
});
no(Filter {
journal_id: Some(5),
..Default::default()
});
}
#[test]
fn hex_round_trips() {
let bytes = [0u8, 1, 0xab, 0xff];
assert_eq!(unhex(&hex(&bytes)), Some(bytes.to_vec()));
assert_eq!(unhex("abc"), None);
assert_eq!(unhex("zz"), None);
}
#[test]
fn ids_read_back() {
let id = EntryId { node: 3, seq: 77 };
assert_eq!(EntryId::from_u64(id.to_u64()), id);
assert_eq!(id.to_string(), "3-77");
}
}
+512
View File
@@ -0,0 +1,512 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! Journaling (journaling spec, JR-1 to JR-18): a copy of each message the
//! server queues, with its envelope, kept where nothing in the product
//! changes or removes it before its retention ends.
//!
//! - this module: journals, what makes one valid, and where they're kept;
//! - [`report`]: the journal report around the untouched message (JR-3);
//! - [`entries`]: the built-in journal and its chain (JR-5, JR-6, JR-13).
//!
//! Kept in the fork's subspace (`store::SUBSPACE_INBUXA`). Every key starts
//! with `J`; journals are `j` + id (u32), as JSON. There are few, so they're
//! read whole.
pub mod archive;
pub mod entries;
pub mod report;
use crate::{hold::Member, mailflow::rules::jmap_ids};
use serde::{Deserialize as SerdeDeserialize, Serialize as SerdeSerialize, de::DeserializeOwned};
use std::{
sync::{Arc, RwLock},
time::{Duration, Instant},
};
use store::{
Deserialize, IterateParams, SUBSPACE_INBUXA, Serialize, Store, ValueKey,
write::{AnyClass, BatchBuilder, ValueClass, assert::AssertValue},
};
use trc::AddContext;
pub(crate) const FEATURE: u8 = b'J';
const KIND_JOURNAL: u8 = b'j';
const CREATE_ATTEMPTS: usize = 5;
/// Retention a journal may be given, in days (settled answer 3).
pub const MIN_RETENTION_DAYS: u32 = 30;
pub const MAX_RETENTION_DAYS: u32 = 3650;
/// Most entries in one scope list.
const MAX_LIST: usize = 5_000;
/// Which way a message goes, from this server's side (JR-9).
#[derive(Debug, Clone, Copy, PartialEq, Eq, SerdeSerialize, SerdeDeserialize)]
#[serde(rename_all = "camelCase")]
pub enum Direction {
/// From someone here to at least one recipient elsewhere.
Outgoing,
/// From elsewhere to someone here.
Incoming,
/// From someone here, to people here only.
Internal,
Any,
}
impl Direction {
pub fn as_str(&self) -> &'static str {
match self {
Direction::Outgoing => "outgoing",
Direction::Incoming => "incoming",
Direction::Internal => "internal",
Direction::Any => "any",
}
}
/// A message's direction: `Any` is never one.
pub fn of(sender_local: bool, any_remote: bool, any_local: bool) -> Direction {
match (sender_local, any_remote) {
(true, true) => Direction::Outgoing,
(true, false) => Direction::Internal,
(false, _) if any_local => Direction::Incoming,
// Nobody here on either side: relayed mail counts as outgoing
(false, _) => Direction::Outgoing,
}
}
fn includes(&self, direction: Direction) -> bool {
*self == Direction::Any || *self == direction
}
}
/// Whose mail a journal takes (JR-9): everyone, or people reached through
/// their account, domain, group or tenant. Ids are in the JMAP form.
#[derive(Debug, Clone, Default, PartialEq, Eq, SerdeSerialize, SerdeDeserialize)]
#[serde(rename_all = "camelCase")]
pub struct Scope {
#[serde(default)]
pub everyone: bool,
#[serde(default, with = "jmap_ids")]
pub accounts: Vec<u32>,
#[serde(default, with = "jmap_ids")]
pub groups: Vec<u32>,
#[serde(default, with = "jmap_ids")]
pub domains: Vec<u32>,
#[serde(default, with = "jmap_ids")]
pub tenants: Vec<u32>,
}
impl Scope {
fn lists(&self) -> [&Vec<u32>; 4] {
[&self.accounts, &self.groups, &self.domains, &self.tenants]
}
/// Whether this scope reaches one person here.
pub fn covers(&self, member: &Member) -> bool {
self.everyone
|| self.accounts.contains(&member.account)
|| member.domains.iter().any(|d| self.domains.contains(d))
|| member.groups.iter().any(|g| self.groups.contains(g))
|| member.tenant.is_some_and(|t| self.tenants.contains(&t))
}
}
/// A journal (JR-9): what it takes, and how long its entries are kept.
#[derive(Debug, Clone, PartialEq, Eq, SerdeSerialize, SerdeDeserialize)]
#[serde(rename_all = "camelCase")]
pub struct Journal {
#[serde(default)]
pub id: u32,
pub name: String,
#[serde(default)]
pub description: String,
#[serde(default)]
pub enabled: bool,
pub direction: Direction,
pub scope: Scope,
/// How long an entry this journal writes is kept. An entry keeps the
/// retention it was written with (JR-12).
pub retention_days: u32,
/// Whether entries go into the built-in journal (JR-5).
#[serde(default = "yes")]
pub built_in: bool,
/// An outside archive's journal address, sent each report (JR-7).
#[serde(default, skip_serializing_if = "Option::is_none")]
pub archive_address: Option<String>,
#[serde(default)]
pub created_by: String,
#[serde(default)]
pub created_at: u64,
#[serde(default)]
pub updated_at: u64,
}
/// Why a journal was refused: the property, and what to do.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Invalid {
pub property: &'static str,
pub reason: String,
}
fn invalid(property: &'static str, reason: impl Into<String>) -> Result<(), Invalid> {
Err(Invalid {
property,
reason: reason.into(),
})
}
impl Journal {
pub fn validate(&self) -> Result<(), Invalid> {
if self.name.trim().is_empty() {
return invalid("name", "Give the journal a name.");
}
if self.name.len() > 200 || self.description.len() > 2_000 {
return invalid("name", "The name or description is too long.");
}
if !(MIN_RETENTION_DAYS..=MAX_RETENTION_DAYS).contains(&self.retention_days) {
return invalid(
"retentionDays",
format!("Keep entries between {MIN_RETENTION_DAYS} and {MAX_RETENTION_DAYS} days."),
);
}
// Neither is a journal only rules send mail to (JR-10)
let chosen = self.scope.lists().iter().any(|list| !list.is_empty());
if self.scope.everyone && chosen {
return invalid(
"scope",
"Journal everyone, or choose accounts, groups, domains or tenants; not both.",
);
}
if !self.built_in && self.archive_address.is_none() {
return invalid(
"builtIn",
"Keep entries in the built-in journal, send them to an archive, or both.",
);
}
if let Some(address) = &self.archive_address
&& !is_address(address)
{
return invalid(
"archiveAddress",
format!("\"{address}\" isn't an email address."),
);
}
if self.scope.lists().iter().any(|list| list.len() > MAX_LIST) {
return invalid("scope", format!("Choose at most {MAX_LIST} of each."));
}
Ok(())
}
/// Whether this journal takes a message going `direction` with these
/// people here on either side.
/// Whether only rules send this journal mail (JR-10).
pub fn rules_only(&self) -> bool {
!self.scope.everyone && self.scope.lists().iter().all(|list| list.is_empty())
}
pub fn takes(&self, direction: Direction, members: &[Member]) -> bool {
self.enabled
&& self.direction.includes(direction)
&& (self.scope.everyone || members.iter().any(|m| self.scope.covers(m)))
}
}
fn yes() -> bool {
true
}
/// An address an archive can be sent to: one `@`, something either side,
/// nothing that would break an envelope.
fn is_address(address: &str) -> bool {
address.len() <= 320
&& address.split_once('@').is_some_and(|(local, domain)| {
!local.is_empty() && domain.contains('.') && !domain.contains('@')
})
&& !address
.chars()
.any(|c| c.is_whitespace() || c.is_control() || matches!(c, '<' | '>' | ',' | ';'))
}
/// A value stored as JSON.
pub(crate) struct Json<T>(pub T);
impl<T: SerdeSerialize> Serialize for Json<T> {
fn serialize(&self) -> trc::Result<Vec<u8>> {
serde_json::to_vec(&self.0).map_err(|err| {
trc::StoreEvent::UnexpectedError
.into_err()
.details("Failed to serialize a journal record")
.reason(err)
})
}
}
impl<T: DeserializeOwned + Sync + Send> Deserialize for Json<T> {
fn deserialize(bytes: &[u8]) -> trc::Result<Self> {
serde_json::from_slice(bytes).map(Json).map_err(|err| {
trc::StoreEvent::DataCorruption
.into_err()
.details("Invalid journal record")
.reason(err)
})
}
}
fn class(id: u32) -> ValueClass {
let mut key = Vec::with_capacity(6);
key.push(FEATURE);
key.push(KIND_JOURNAL);
key.extend_from_slice(&id.to_be_bytes());
ValueClass::Any(AnyClass {
subspace: SUBSPACE_INBUXA,
key,
})
}
fn key(id: u32) -> ValueKey<ValueClass> {
ValueKey::from(class(id))
}
pub async fn get(data: &Store, id: u32) -> trc::Result<Option<Journal>> {
Ok(data
.get_value::<Json<Journal>>(key(id))
.await
.caused_by(trc::location!())?
.map(|Json(journal)| journal))
}
/// Every journal, oldest first.
pub async fn all(data: &Store) -> trc::Result<Vec<Journal>> {
let mut journals = Vec::new();
data.iterate(IterateParams::new(key(0), key(u32::MAX)), |_, value| {
if let Ok(Json(journal)) = Json::<Journal>::deserialize(value) {
journals.push(journal);
}
Ok(true)
})
.await
.caused_by(trc::location!())?;
journals.sort_by_key(|journal| journal.id);
Ok(journals)
}
/// Writes a new journal under the next free id, which it returns.
pub async fn create(data: &Store, journal: &Journal) -> trc::Result<u32> {
let mut attempt = 0;
loop {
attempt += 1;
let id = all(data).await?.iter().map(|j| j.id).max().unwrap_or(0) + 1;
let stored = Journal {
id,
..journal.clone()
};
let mut batch = BatchBuilder::new();
batch.assert_value(class(id), AssertValue::None);
batch.set(class(id), Json(&stored).serialize()?);
match data.write(batch.build_all()).await {
Ok(_) => {
invalidate();
return Ok(id);
}
Err(err)
if attempt < CREATE_ATTEMPTS
&& matches!(
err.as_ref(),
trc::EventType::Store(trc::StoreEvent::AssertValueFailed)
) => {}
Err(err) => return Err(err.caused_by(trc::location!())),
}
}
}
/// Replaces a stored journal (same id).
pub async fn update(data: &Store, journal: &Journal) -> trc::Result<()> {
let mut batch = BatchBuilder::new();
batch.set(class(journal.id), Json(journal).serialize()?);
data.write(batch.build_all())
.await
.caused_by(trc::location!())?;
invalidate();
Ok(())
}
/// Removes a journal. Its entries stay, each until its own time.
pub async fn delete(data: &Store, id: u32) -> trc::Result<()> {
let mut batch = BatchBuilder::new();
batch.clear(class(id));
data.write(batch.build_all())
.await
.caused_by(trc::location!())?;
invalidate();
Ok(())
}
/// How long a node keeps its copy of the journals before reading them again.
pub const TTL: Duration = Duration::from_secs(30);
type Cached = Option<(Instant, Arc<Vec<Journal>>)>;
static CACHE: RwLock<Cached> = RwLock::new(None);
/// Forgets this node's copy, so the next message reads the journals again.
pub fn invalidate() {
if let Ok(mut cache) = CACHE.write() {
*cache = None;
}
}
/// The enabled journals, from this node's copy (refreshed every [`TTL`]).
pub async fn enabled(data: &Store) -> trc::Result<Arc<Vec<Journal>>> {
if let Ok(cache) = CACHE.read()
&& let Some((at, journals)) = cache.as_ref()
&& at.elapsed() < TTL
{
return Ok(journals.clone());
}
let journals = Arc::new(
all(data)
.await?
.into_iter()
.filter(|journal| journal.enabled)
.collect::<Vec<_>>(),
);
if let Ok(mut cache) = CACHE.write() {
*cache = Some((Instant::now(), journals.clone()));
}
Ok(journals)
}
#[cfg(test)]
mod tests {
use super::*;
fn journal(scope: Scope) -> Journal {
Journal {
id: 1,
name: "Finance".into(),
description: String::new(),
enabled: true,
direction: Direction::Any,
scope,
retention_days: 365,
built_in: true,
archive_address: None,
created_by: String::new(),
created_at: 0,
updated_at: 0,
}
}
fn member(account: u32, groups: Vec<u32>) -> Member {
Member {
account,
domains: vec![1],
groups,
tenant: None,
}
}
#[test]
fn scope_is_everyone_or_chosen() {
assert!(
journal(Scope {
everyone: true,
..Default::default()
})
.validate()
.is_ok()
);
// Nobody chosen: only rules send it mail
let rules_only = journal(Scope::default());
assert!(rules_only.validate().is_ok());
assert!(rules_only.rules_only());
assert!(!rules_only.takes(Direction::Any, &[member(3, vec![7])]));
let both = Scope {
everyone: true,
groups: vec![4],
..Default::default()
};
assert_eq!(journal(both).validate().unwrap_err().property, "scope");
}
#[test]
fn destinations() {
let mut j = journal(Scope {
everyone: true,
..Default::default()
});
j.built_in = false;
assert_eq!(j.validate().unwrap_err().property, "builtIn");
j.archive_address = Some("[email protected]".into());
assert!(j.validate().is_ok());
for bad in [
"archive",
"a@b",
"a [email protected]",
"<[email protected]>",
"a@[email protected]",
] {
j.archive_address = Some(bad.into());
assert_eq!(
j.validate().unwrap_err().property,
"archiveAddress",
"{bad}"
);
}
// Stored before destinations existed: the built-in journal
let old: Journal = serde_json::from_str(
r#"{"name":"Old","direction":"any","scope":{"everyone":true},"retentionDays":30}"#,
)
.unwrap();
assert!(old.built_in && old.archive_address.is_none());
}
#[test]
fn retention_has_bounds() {
let mut j = journal(Scope {
everyone: true,
..Default::default()
});
j.retention_days = 29;
assert_eq!(j.validate().unwrap_err().property, "retentionDays");
j.retention_days = 3651;
assert!(j.validate().is_err());
j.retention_days = 3650;
assert!(j.validate().is_ok());
}
#[test]
fn takes_by_direction_and_member() {
let mut j = journal(Scope {
groups: vec![7],
..Default::default()
});
assert!(j.takes(Direction::Outgoing, &[member(3, vec![7])]));
assert!(!j.takes(Direction::Outgoing, &[member(3, vec![8])]));
assert!(!j.takes(Direction::Outgoing, &[]));
j.direction = Direction::Incoming;
assert!(!j.takes(Direction::Outgoing, &[member(3, vec![7])]));
j.enabled = false;
assert!(!j.takes(Direction::Incoming, &[member(3, vec![7])]));
}
#[test]
fn directions() {
assert_eq!(Direction::of(true, true, true), Direction::Outgoing);
assert_eq!(Direction::of(true, false, true), Direction::Internal);
assert_eq!(Direction::of(false, false, true), Direction::Incoming);
assert_eq!(Direction::of(false, true, true), Direction::Incoming);
}
#[test]
fn scope_ids_are_jmap_ids() {
let scope: Scope = serde_json::from_str(r#"{"groups":["b"],"tenants":[7]}"#).unwrap();
assert_eq!(scope.groups, vec![1]);
assert_eq!(scope.tenants, vec![7]);
assert_eq!(
serde_json::to_value(&scope).unwrap()["tenants"],
serde_json::json!(["h"])
);
}
}
+385
View File
@@ -0,0 +1,385 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! The journal report (JR-3, JR-4): a message whose first part lists the
//! envelope, one field a line, and whose second part is the message as it
//! was queued, byte for byte, as `message/rfc822`. Field names are fixed
//! English: a report is a record, and scripts read it.
use super::Direction;
use mail_builder::headers::{Header, date::Date, text::Text};
use mail_parser::MessageParser;
use sha2::{Digest, Sha256};
/// One envelope recipient, with the address it was given as (a list's, for
/// the list's members).
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Recipient {
pub address: String,
pub orcpt: Option<String>,
/// The mail flow rule that added or redirected to it.
pub added_by: Option<String>,
}
/// What the queue knows about a message.
#[derive(Debug, Clone)]
pub struct Envelope<'x> {
pub sender: &'x str,
pub authenticated: bool,
pub recipients: &'x [Recipient],
pub queue_id: u64,
/// Seconds.
pub received: u64,
pub direction: Direction,
pub held: bool,
}
/// What a report says, besides the envelope's own fields.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct Fields {
pub subject: String,
pub message_id: String,
pub to: Vec<String>,
pub cc: Vec<String>,
/// Envelope recipients in neither To nor Cc, nor reached through a list.
pub bcc: Vec<String>,
/// A list's address, and its members among the recipients.
pub expanded: Vec<(String, Vec<String>)>,
/// A rule's name, and the recipients it added.
pub added: Vec<(String, Vec<String>)>,
}
/// One line's worth of a value: no line breaks, no control characters.
fn line(value: &str) -> String {
value
.chars()
.map(|c| if c.is_control() { ' ' } else { c })
.collect::<String>()
.trim()
.to_string()
}
/// The address an ORCPT names, without its `rfc822;` type.
fn orcpt_address(orcpt: &str) -> String {
let orcpt = orcpt.trim();
let bare = match orcpt.split_once(';') {
Some((kind, address)) if kind.eq_ignore_ascii_case("rfc822") => address,
_ => orcpt,
};
bare.trim().to_lowercase()
}
/// Sorts the envelope's recipients by how they were addressed.
pub fn fields(envelope: &Envelope<'_>, original: &[u8]) -> Fields {
let parsed = MessageParser::default().parse_headers(original);
let headed = |which: Option<&mail_parser::Address<'_>>| -> Vec<String> {
which
.map(|list| {
list.iter()
.filter_map(|addr| addr.address())
.map(|address| address.to_lowercase())
.collect()
})
.unwrap_or_default()
};
let (subject, message_id, header_to, header_cc) = match &parsed {
Some(message) => (
message.subject().map(line).unwrap_or_default(),
message
.message_id()
.map(|id| format!("<{}>", line(id)))
.unwrap_or_default(),
headed(message.to()),
headed(message.cc()),
),
None => Default::default(),
};
let mut fields = Fields {
subject,
message_id,
..Default::default()
};
for rcpt in envelope.recipients {
let address = rcpt.address.to_lowercase();
let via = rcpt
.orcpt
.as_deref()
.map(orcpt_address)
.filter(|via| !via.is_empty() && *via != address);
if let Some(rule) = &rcpt.added_by {
match fields.added.iter_mut().find(|(name, _)| name == rule) {
Some((_, added)) => added.push(line(&rcpt.address)),
None => fields.added.push((line(rule), vec![line(&rcpt.address)])),
}
} else if header_to.contains(&address) {
fields.to.push(line(&rcpt.address));
} else if header_cc.contains(&address) {
fields.cc.push(line(&rcpt.address));
} else if let Some(via) = via {
match fields.expanded.iter_mut().find(|(list, _)| *list == via) {
Some((_, members)) => members.push(line(&rcpt.address)),
None => fields
.expanded
.push((line(&via), vec![line(&rcpt.address)])),
}
} else {
fields.bcc.push(line(&rcpt.address));
}
}
fields
}
/// The report's first part.
pub fn text(envelope: &Envelope<'_>, fields: &Fields) -> String {
let mut out = String::new();
let mut field = |name: &str, value: &str| {
if !value.is_empty() {
out.push_str(name);
out.push_str(": ");
out.push_str(value);
out.push_str("\r\n");
}
};
let sender = if envelope.sender.is_empty() {
"<>".to_string()
} else {
line(envelope.sender)
};
field("Sender", &sender);
field(
"Authenticated",
if envelope.authenticated { "yes" } else { "no" },
);
field("Subject", &fields.subject);
field("Message-ID", &fields.message_id);
field("Queue ID", &format!("{:x}", envelope.queue_id));
field(
"Received",
&mail_parser::DateTime::from_timestamp(envelope.received as i64).to_rfc3339(),
);
field("Direction", envelope.direction.as_str());
field("To", &fields.to.join(", "));
field("Cc", &fields.cc.join(", "));
field("Bcc", &fields.bcc.join(", "));
for (list, members) in &fields.expanded {
field("Expanded", &format!("{list} -> {}", members.join(", ")));
}
for (rule, added) in &fields.added {
field("Added by rule", &format!("{rule} -> {}", added.join(", ")));
}
if envelope.held {
field("Held for review", "yes");
}
out
}
fn hex(bytes: &[u8]) -> String {
bytes.iter().map(|b| format!("{b:02x}")).collect()
}
/// Whether a message can travel as 8bit: no NULs, no line past 998 bytes.
fn fits_8bit(message: &[u8]) -> bool {
!message.contains(&0) && message.split(|b| *b == b'\n').all(|l| l.len() <= 998)
}
/// The whole report: headers, the fields, then the original untouched.
/// `from` is the address the report is from; `host` names the server in its
/// Message-ID.
pub fn build(
envelope: &Envelope<'_>,
original: &[u8],
from: &str,
host: &str,
) -> (Vec<u8>, Fields) {
let fields = fields(envelope, original);
let body = text(envelope, &fields);
// A boundary that can't occur in the original
let mut boundary = format!("journal-{}", &hex(&Sha256::digest(original))[..32]);
while original
.windows(boundary.len())
.any(|window| window == boundary.as_bytes())
{
boundary.push('x');
}
let mut out: Vec<u8> = Vec::with_capacity(original.len() + body.len() + 1024);
out.extend_from_slice(format!("From: Journal <{}>\r\n", line(from)).as_bytes());
out.extend_from_slice(b"Date: ");
out.extend_from_slice(Date::new(envelope.received as i64).to_rfc822().as_bytes());
out.extend_from_slice(b"\r\n");
out.extend_from_slice(b"Subject: ");
let subject = if fields.subject.is_empty() {
"Journal report".to_string()
} else {
format!("Journal report: {}", fields.subject)
};
Text::new(subject).write_header(&mut out, "Subject: ".len());
out.extend_from_slice(
format!(
"Message-ID: <journal.{:x}.{}@{}>\r\n",
envelope.queue_id,
envelope.received,
line(host)
)
.as_bytes(),
);
out.extend_from_slice(format!("X-Inbuxa-Journal: {:x}\r\n", envelope.queue_id).as_bytes());
out.extend_from_slice(b"MIME-Version: 1.0\r\n");
out.extend_from_slice(
format!("Content-Type: multipart/mixed; boundary=\"{boundary}\"\r\n\r\n").as_bytes(),
);
out.extend_from_slice(format!("--{boundary}\r\n").as_bytes());
out.extend_from_slice(
b"Content-Type: text/plain; charset=utf-8\r\nContent-Transfer-Encoding: 8bit\r\n\r\n",
);
out.extend_from_slice(body.as_bytes());
out.extend_from_slice(format!("\r\n--{boundary}\r\n").as_bytes());
out.extend_from_slice(b"Content-Type: message/rfc822\r\n");
out.extend_from_slice(b"Content-Disposition: attachment; filename=\"original.eml\"\r\n");
out.extend_from_slice(if fits_8bit(original) {
b"Content-Transfer-Encoding: 8bit\r\n\r\n".as_slice()
} else {
b"Content-Transfer-Encoding: binary\r\n\r\n".as_slice()
});
out.extend_from_slice(original);
// The line break before a boundary belongs to the boundary: the
// original keeps its own last one
out.extend_from_slice(format!("\r\n--{boundary}--\r\n").as_bytes());
(out, fields)
}
/// Where the original starts and ends inside a report [`build`] made.
pub fn original(report: &[u8]) -> Option<&[u8]> {
let parsed = MessageParser::default().parse(report)?;
let part = parsed.attachment(0)?;
let start = part.raw_body_offset() as usize;
let end = part.raw_end_offset() as usize;
report.get(start..end)
}
#[cfg(test)]
mod tests {
use super::*;
const ORIGINAL: &[u8] = b"From: [email protected]\r\n\
To: Bank <[email protected]>\r\n\
Cc: [email protected]\r\n\
Subject: Q3 figures\r\n\
Message-ID: <[email protected]>\r\n\
\r\n\
The figures.\r\n";
fn rcpt(address: &str, orcpt: Option<&str>) -> Recipient {
Recipient {
address: address.into(),
orcpt: orcpt.map(Into::into),
added_by: None,
}
}
fn envelope(recipients: &[Recipient]) -> Envelope<'_> {
Envelope {
sender: "[email protected]",
authenticated: true,
recipients,
queue_id: 0x1a2b,
received: 1_790_000_000,
direction: Direction::Outgoing,
held: false,
}
}
#[test]
fn recipients_sorted_by_how_they_were_addressed() {
let recipients = [
rcpt("[email protected]", None),
rcpt("[email protected]", Some("rfc822;[email protected]")),
rcpt("[email protected]", None),
rcpt("[email protected]", Some("[email protected]")),
rcpt("[email protected]", Some("rfc822;[email protected]")),
];
let fields = fields(&envelope(&recipients), ORIGINAL);
assert_eq!(fields.subject, "Q3 figures");
assert_eq!(fields.message_id, "<[email protected]>");
assert_eq!(fields.to, vec!["[email protected]"]);
assert_eq!(fields.cc, vec!["[email protected]"]);
assert_eq!(fields.bcc, vec!["[email protected]"]);
assert_eq!(
fields.expanded,
vec![(
"[email protected]".to_string(),
vec![
"[email protected]".to_string(),
"[email protected]".to_string()
]
)]
);
}
#[test]
fn report_carries_the_original_untouched() {
let recipients = [
rcpt("[email protected]", None),
rcpt("[email protected]", None),
];
let (report, _) = build(
&envelope(&recipients),
ORIGINAL,
"[email protected]",
"mx.example.com",
);
let text = String::from_utf8_lossy(&report);
assert!(text.contains("Sender: [email protected]\r\n"));
assert!(text.contains("Bcc: [email protected]\r\n"));
assert!(text.contains("Queue ID: 1a2b\r\n"));
assert!(text.contains("Direction: outgoing\r\n"));
assert!(text.contains("Subject: Journal report: Q3 figures\r\n"));
assert!(!text.contains("Held for review"));
assert_eq!(original(&report), Some(ORIGINAL));
let unterminated = &ORIGINAL[..ORIGINAL.len() - 2];
let (report, _) = build(
&envelope(&recipients),
unterminated,
"[email protected]",
"mx.example.com",
);
assert_eq!(original(&report), Some(unterminated));
}
#[test]
fn rule_added_recipients_say_so() {
let mut copied = rcpt("[email protected]", None);
copied.added_by = Some("Copy finance".into());
let recipients = [rcpt("[email protected]", None), copied];
let env = envelope(&recipients);
let fields = fields(&env, ORIGINAL);
assert!(fields.bcc.is_empty(), "{fields:?}");
assert!(
text(&env, &fields).contains("Added by rule: Copy finance -> [email protected]\r\n")
);
}
#[test]
fn values_stay_on_one_line() {
let recipients = [rcpt("[email protected]", None)];
let mut env = envelope(&recipients);
env.sender = "[email protected]\r\nBcc: [email protected]";
env.held = true;
let body = text(&env, &Fields::default());
assert_eq!(body.matches("\r\n").count(), body.lines().count());
assert!(body.contains("Sender: [email protected] Bcc: [email protected]\r\n"));
assert!(body.contains("Held for review: yes\r\n"));
}
#[test]
fn an_empty_sender_is_shown_as_such() {
let recipients = [rcpt("[email protected]", None)];
let mut env = envelope(&recipients);
env.sender = "";
assert!(text(&env, &Fields::default()).starts_with("Sender: <>\r\n"));
}
}
+1
View File
@@ -22,6 +22,7 @@ pub mod ai;
pub mod audit;
pub mod branding;
pub mod hold;
pub mod journal;
pub mod lock;
pub mod mailflow;
pub mod masked_email;
+74 -3
View File
@@ -62,7 +62,7 @@ fn one() -> u32 {
/// Group and tenant ids in the JMAP form clients use (`"b"`, `"c"`…), held
/// as numbers for matching. Plain numbers are read too.
mod jmap_ids {
pub(crate) mod jmap_ids {
use serde::{Deserialize, Deserializer, Serializer, de::Error, ser::SerializeSeq};
use std::str::FromStr;
use types::id::Id;
@@ -95,6 +95,33 @@ mod jmap_ids {
}
}
/// One id in the same form.
pub(crate) mod jmap_id {
use serde::{Deserialize, Deserializer, Serializer, de::Error};
use std::str::FromStr;
use types::id::Id;
pub fn serialize<S: Serializer>(id: &u32, serializer: S) -> Result<S::Ok, S::Error> {
serializer.serialize_str(&Id::from(*id).to_string())
}
#[derive(Deserialize)]
#[serde(untagged)]
enum Either {
Text(String),
Number(u32),
}
pub fn deserialize<'de, D: Deserializer<'de>>(deserializer: D) -> Result<u32, D::Error> {
match Either::deserialize(deserializer)? {
Either::Number(n) => Ok(n),
Either::Text(text) => Id::from_str(&text)
.map(|id| id.document_id())
.map_err(|_| D::Error::custom(format!("\"{text}\" isn't an id"))),
}
}
}
/// A detector and the least it must find.
#[derive(Debug, Clone, PartialEq, Eq, SerdeSerialize, SerdeDeserialize)]
#[serde(rename_all = "camelCase")]
@@ -222,6 +249,11 @@ pub enum Action {
Route {
queue: String,
},
/// Journaling spec, JR-10: a copy into this journal, whatever its scope.
Journal {
#[serde(with = "jmap_id")]
journal: u32,
},
// DLP actions
Block {
notice: String,
@@ -311,10 +343,16 @@ impl Rule {
if self.direction != Direction::Outgoing {
return Err(invalid("direction", "DLP rules check outgoing mail only."));
}
if dlp_actions != 1 || self.actions.len() != 1 {
// One of block, warn or hold; journaling may go with it
if dlp_actions != 1
|| self
.actions
.iter()
.any(|a| !a.is_dlp() && !matches!(a, Action::Journal { .. }))
{
return Err(invalid(
"actions",
"A DLP rule has exactly one action: block, warn or hold.",
"A DLP rule has exactly one action: block, warn or hold, and may also journal the message.",
));
}
}
@@ -478,6 +516,7 @@ fn validate_action(action: &Action) -> Result<(), String> {
}
Action::Refuse { text: t } => text(t, "refusal text"),
Action::Route { queue } => text(queue, "queue"),
Action::Journal { .. } => Ok(()),
Action::Block { notice } | Action::Warn { notice } | Action::Hold { notice, .. } => {
text(notice, "notice")
}
@@ -619,6 +658,38 @@ mod tests {
}
}
#[test]
fn journal_action_goes_with_either_kind() {
let hold = Action::Hold {
notice: "Held.".into(),
notify_sender: false,
};
let journal = Action::Journal { journal: 3 };
assert!(
rule(Kind::Dlp, vec![hold.clone(), journal.clone()])
.validate()
.is_ok()
);
assert!(rule(Kind::Dlp, vec![journal.clone()]).validate().is_err());
assert!(
rule(
Kind::Dlp,
vec![hold, Action::PrefixSubject { text: "x".into() }]
)
.validate()
.is_err()
);
assert!(
rule(Kind::Transport, vec![journal.clone()])
.validate()
.is_ok()
);
let json = serde_json::to_value(&journal).unwrap();
assert_eq!(json, serde_json::json!({"type": "journal", "journal": "d"}));
let back: Action = serde_json::from_value(json).unwrap();
assert_eq!(back, journal);
}
#[test]
fn wire_format() {
let json = r#"{"name":"Cards","kind":"dlp","direction":"outgoing",
+280
View File
@@ -0,0 +1,280 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! Accepted security to-do items (security to-do list spec, SS-23 to SS-26).
//!
//! The console runs the checks; the server only keeps what an administrator
//! accepted, so every administrator sees the same accepted risks. An
//! acceptance names the check, what within it (a domain, a certificate…),
//! the value the check saw, and why. It holds only while the check still
//! sees that value, which the console compares. Acceptances are created and
//! removed, never edited.
//!
//! Kept in the fork's subspace (`store::SUBSPACE_INBUXA`). Every key starts
//! with `Q`, then one byte for the kind:
//!
//! - `a` + acceptance id (u32): the acceptance, as JSON.
//!
//! Numbers are big-endian. There are at most [`MAX_ACCEPTANCES`], so
//! they're read whole.
use serde::{Deserialize as SerdeDeserialize, Serialize as SerdeSerialize};
use store::{
Deserialize, IterateParams, SUBSPACE_INBUXA, Serialize, Store, ValueKey,
write::{AnyClass, BatchBuilder, ValueClass, assert::AssertValue},
};
use trc::AddContext;
const FEATURE: u8 = b'Q';
const KIND_ACCEPTANCE: u8 = b'a';
const CREATE_ATTEMPTS: usize = 5;
pub const MAX_ACCEPTANCES: usize = 200;
/// The checks are SS-1 to SS-18; a few spare for checks added later.
const MAX_CHECK: u32 = 40;
const MAX_SUBJECT: usize = 255;
const MAX_VALUE_BYTES: usize = 4096;
const MAX_NOTE: usize = 500;
#[derive(Debug, Clone, PartialEq, SerdeSerialize, SerdeDeserialize)]
#[serde(rename_all = "camelCase")]
pub struct Acceptance {
#[serde(default)]
pub id: u32,
/// Which check: `SS-1`, `SS-2`…
pub check: String,
/// What within the check: empty for a server-wide setting, else the
/// domain, strategy or certificate it names.
#[serde(default)]
pub subject: String,
/// The value the check saw when it was accepted.
#[serde(default)]
pub accepted_value: serde_json::Value,
/// Why. Required.
pub note: String,
#[serde(default)]
pub accepted_by: String,
/// Seconds since the epoch.
#[serde(default)]
pub accepted_at: u64,
}
#[derive(Debug, PartialEq, Eq)]
pub struct Invalid {
pub property: &'static str,
pub reason: String,
}
fn invalid(property: &'static str, reason: impl Into<String>) -> Invalid {
Invalid {
property,
reason: reason.into(),
}
}
impl Acceptance {
/// What an administrator sends is checked whole before it's kept.
pub fn validate(&self) -> Result<(), Invalid> {
let check_ok = self
.check
.strip_prefix("SS-")
.and_then(|n| n.parse::<u32>().ok())
.is_some_and(|n| (1..=MAX_CHECK).contains(&n));
if !check_ok {
return Err(invalid("check", "A check is named SS-1, SS-2 and so on."));
}
if self.subject.chars().count() > MAX_SUBJECT {
return Err(invalid(
"subject",
format!("At most {MAX_SUBJECT} characters."),
));
}
let value_bytes = serde_json::to_vec(&self.accepted_value)
.map(|v| v.len())
.unwrap_or(usize::MAX);
if value_bytes > MAX_VALUE_BYTES {
return Err(invalid(
"acceptedValue",
format!("At most {MAX_VALUE_BYTES} bytes."),
));
}
let note = self.note.trim();
if note.is_empty() {
return Err(invalid("note", "Say why this is accepted."));
}
if note.chars().count() > MAX_NOTE {
return Err(invalid("note", format!("At most {MAX_NOTE} characters.")));
}
Ok(())
}
}
// --- Storage --------------------------------------------------------------
struct Json<T>(T);
impl<T: SerdeSerialize> Serialize for Json<T> {
fn serialize(&self) -> trc::Result<Vec<u8>> {
serde_json::to_vec(&self.0).map_err(|err| {
trc::StoreEvent::UnexpectedError
.into_err()
.details("Failed to serialize a security acceptance")
.reason(err)
})
}
}
impl Deserialize for Json<Acceptance> {
fn deserialize(bytes: &[u8]) -> trc::Result<Self> {
serde_json::from_slice(bytes).map(Json).map_err(|err| {
trc::StoreEvent::DataCorruption
.into_err()
.details("Invalid security acceptance")
.reason(err)
})
}
}
fn class(id: u32) -> ValueClass {
let mut key = Vec::with_capacity(6);
key.push(FEATURE);
key.push(KIND_ACCEPTANCE);
key.extend_from_slice(&id.to_be_bytes());
ValueClass::Any(AnyClass {
subspace: SUBSPACE_INBUXA,
key,
})
}
fn key(id: u32) -> ValueKey<ValueClass> {
ValueKey::from(class(id))
}
pub async fn get(data: &Store, id: u32) -> trc::Result<Option<Acceptance>> {
Ok(data
.get_value::<Json<Acceptance>>(key(id))
.await
.caused_by(trc::location!())?
.map(|Json(acceptance)| acceptance))
}
/// Every acceptance, oldest first.
pub async fn all(data: &Store) -> trc::Result<Vec<Acceptance>> {
let mut out = Vec::new();
data.iterate(IterateParams::new(key(0), key(u32::MAX)), |_, value| {
if let Ok(Json(acceptance)) = Json::<Acceptance>::deserialize(value) {
out.push(acceptance);
}
Ok(true)
})
.await
.caused_by(trc::location!())?;
out.sort_by_key(|a| a.id);
Ok(out)
}
pub enum Created {
Id(u32),
/// There are already [`MAX_ACCEPTANCES`].
Full,
}
/// Keeps a new acceptance under the next free id. Two nodes creating at
/// once can't take the same id: the key must be absent.
pub async fn create(data: &Store, acceptance: &Acceptance) -> trc::Result<Created> {
let mut attempt = 0;
loop {
attempt += 1;
let existing = all(data).await?;
if existing.len() >= MAX_ACCEPTANCES {
return Ok(Created::Full);
}
let id = existing.iter().map(|a| a.id).max().unwrap_or(0) + 1;
let stored = Acceptance {
id,
..acceptance.clone()
};
let mut batch = BatchBuilder::new();
batch.assert_value(class(id), AssertValue::None);
batch.set(class(id), Json(&stored).serialize()?);
match data.write(batch.build_all()).await {
Ok(_) => return Ok(Created::Id(id)),
Err(err)
if attempt < CREATE_ATTEMPTS
&& matches!(
err.as_ref(),
trc::EventType::Store(trc::StoreEvent::AssertValueFailed)
) => {}
Err(err) => return Err(err.caused_by(trc::location!())),
}
}
}
pub async fn delete(data: &Store, id: u32) -> trc::Result<()> {
let mut batch = BatchBuilder::new();
batch.clear(class(id));
data.write(batch.build_all())
.await
.caused_by(trc::location!())?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
fn acceptance() -> Acceptance {
Acceptance {
id: 0,
check: "SS-1".into(),
subject: String::new(),
accepted_value: serde_json::json!(true),
note: "Old clients on the LAN; closed by 2027.".into(),
accepted_by: String::new(),
accepted_at: 0,
}
}
#[test]
fn a_note_is_required() {
assert!(acceptance().validate().is_ok());
let blank = Acceptance {
note: " ".into(),
..acceptance()
};
assert_eq!(blank.validate().unwrap_err().property, "note");
let long = Acceptance {
note: "x".repeat(501),
..acceptance()
};
assert_eq!(long.validate().unwrap_err().property, "note");
}
#[test]
fn only_named_checks() {
for bad in ["", "SS-0", "SS-41", "ss-1", "SS-x", "1"] {
let a = Acceptance {
check: bad.into(),
..acceptance()
};
assert_eq!(a.validate().unwrap_err().property, "check", "{bad}");
}
}
#[test]
fn subject_and_value_are_bounded() {
let a = Acceptance {
subject: "d".repeat(256),
..acceptance()
};
assert_eq!(a.validate().unwrap_err().property, "subject");
let a = Acceptance {
accepted_value: serde_json::json!("v".repeat(4096)),
..acceptance()
};
assert_eq!(a.validate().unwrap_err().property, "acceptedValue");
}
}
+1
View File
@@ -10,6 +10,7 @@
//! ships. The legacy-protocols switch is INBUXA's own design, specified in
//! `legacy-protocols.md`.
pub mod acceptance;
pub mod legacy_use;
pub mod log_files;
pub mod listeners;
+11 -7
View File
@@ -270,13 +270,17 @@ impl ClientRegistrationHandler for Server {
false
};
// Check if the account is allowed to override client registration
if self
.access_token(account_id)
.await
.caused_by(trc::location!())?
.build()
.has_permission(Permission::OAuthClientOverride)
// Check if the account is allowed to override client registration.
// inbuxa: only while setting up or recovering, when the recovery
// administrator signs in before any client is registered (contract C-5)
let registry = self.registry();
if (registry.is_bootstrap_mode() || registry.is_recovery_mode())
&& self
.access_token(account_id)
.await
.caused_by(trc::location!())?
.build()
.has_permission(Permission::OAuthClientOverride)
{
return Ok(None);
}
@@ -0,0 +1,217 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! `inbuxa:Journal/get` and `/set` under `urn:inbuxa:jmap`: journals
//! (journaling spec, JR-9, JR-12). What a journal has taken stays when the
//! journal changes or goes; each entry keeps its own retention.
use crate::{
object::{AnyId, JmapObject, JmapObjectId},
request::deserialize::DeserializeArguments,
};
use jmap_tools::{Element, Key, Property};
use std::{borrow::Cow, str::FromStr};
use types::id::Id;
#[derive(Debug, Clone, Default)]
pub struct Journal;
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum JournalProperty {
Id,
Name,
Description,
Enabled,
/// `outgoing`, `incoming`, `internal` or `any`.
Direction,
/// Everyone, or chosen accounts, groups, domains and tenants.
Scope,
/// How long an entry is kept; each keeps what it was written with.
RetentionDays,
/// Whether entries go into the built-in journal.
BuiltIn,
/// An outside archive's journal address.
ArchiveAddress,
/// Reports the archive didn't take: how many, when and why last.
ArchiveFailures,
CreatedBy,
CreatedAt,
UpdatedAt,
}
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum JournalValue {
Id(Id),
}
impl Property for JournalProperty {
fn try_parse(parent: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {
// Keys inside the scope stay plain keys
match parent {
None => JournalProperty::parse(value),
Some(_) => None,
}
}
fn to_cow(&self) -> Cow<'static, str> {
match self {
JournalProperty::Id => "id",
JournalProperty::Name => "name",
JournalProperty::Description => "description",
JournalProperty::Enabled => "enabled",
JournalProperty::Direction => "direction",
JournalProperty::Scope => "scope",
JournalProperty::RetentionDays => "retentionDays",
JournalProperty::BuiltIn => "builtIn",
JournalProperty::ArchiveAddress => "archiveAddress",
JournalProperty::ArchiveFailures => "archiveFailures",
JournalProperty::CreatedBy => "createdBy",
JournalProperty::CreatedAt => "createdAt",
JournalProperty::UpdatedAt => "updatedAt",
}
.into()
}
}
impl JournalProperty {
fn parse(value: &str) -> Option<Self> {
hashify::tiny_map!(value.as_bytes(),
b"id" => JournalProperty::Id,
b"name" => JournalProperty::Name,
b"description" => JournalProperty::Description,
b"enabled" => JournalProperty::Enabled,
b"direction" => JournalProperty::Direction,
b"scope" => JournalProperty::Scope,
b"retentionDays" => JournalProperty::RetentionDays,
b"builtIn" => JournalProperty::BuiltIn,
b"archiveAddress" => JournalProperty::ArchiveAddress,
b"archiveFailures" => JournalProperty::ArchiveFailures,
b"createdBy" => JournalProperty::CreatedBy,
b"createdAt" => JournalProperty::CreatedAt,
b"updatedAt" => JournalProperty::UpdatedAt,
)
}
}
impl FromStr for JournalProperty {
type Err = ();
fn from_str(s: &str) -> Result<Self, Self::Err> {
JournalProperty::parse(s).ok_or(())
}
}
impl Element for JournalValue {
type Property = JournalProperty;
fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {
match key {
Key::Property(JournalProperty::Id) => Id::from_str(value).ok().map(JournalValue::Id),
_ => None,
}
}
fn to_cow(&self) -> Cow<'static, str> {
match self {
JournalValue::Id(id) => id.to_string().into(),
}
}
}
/// The set call's own argument: why, for the audit log.
#[derive(Debug, Clone, Default)]
pub struct JournalSetArguments {
pub reason: Option<String>,
}
impl<'de> DeserializeArguments<'de> for JournalSetArguments {
fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>
where
A: serde::de::MapAccess<'de>,
{
if key == "reason" {
self.reason = map.next_value()?;
} else {
let _ = map.next_value::<serde::de::IgnoredAny>()?;
}
Ok(())
}
}
impl JmapObject for Journal {
type Property = JournalProperty;
type Element = JournalValue;
type Id = Id;
type Filter = ();
type Comparator = ();
type GetArguments = ();
type SetArguments<'de> = JournalSetArguments;
type QueryArguments = ();
type CopyArguments = ();
type ParseArguments = ();
const ID_PROPERTY: Self::Property = JournalProperty::Id;
}
impl From<Id> for JournalValue {
fn from(id: Id) -> Self {
JournalValue::Id(id)
}
}
impl JmapObjectId for JournalValue {
fn as_id(&self) -> Option<Id> {
match self {
JournalValue::Id(id) => Some(*id),
}
}
fn as_any_id(&self) -> Option<AnyId> {
match self {
JournalValue::Id(id) => Some(AnyId::Id(*id)),
}
}
fn as_id_ref(&self) -> Option<&str> {
None
}
fn try_set_id(&mut self, new_id: AnyId) -> bool {
if let AnyId::Id(id) = new_id {
*self = JournalValue::Id(id);
true
} else {
false
}
}
}
impl JmapObjectId for JournalProperty {
fn as_id(&self) -> Option<Id> {
None
}
fn as_any_id(&self) -> Option<AnyId> {
None
}
fn as_id_ref(&self) -> Option<&str> {
None
}
fn try_set_id(&mut self, _: AnyId) -> bool {
false
}
}
@@ -0,0 +1,340 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! The journal's JMAP objects under `urn:inbuxa:jmap` (journaling spec,
//! JR-6, JR-15 to JR-17):
//!
//! - `inbuxa:JournalEntry/get` and `/query`: what was journaled, read-only.
//! `report` (the whole journal report) comes only when asked for.
//! - `inbuxa:JournalExport/set`: create one to get a ZIP of the reports a
//! filter matches.
//! - `inbuxa:JournalVerification/set`: create one to recheck every chain.
//!
//! They share one set of properties. Nested values (an export's filter, a
//! verification's chains) are plain JSON objects.
use crate::{
object::{AnyId, JmapObject, JmapObjectId},
request::deserialize::DeserializeArguments,
};
use jmap_tools::{Element, Key, Property};
use std::{borrow::Cow, str::FromStr};
use types::id::Id;
#[derive(Debug, Clone, Default)]
pub struct JournalEntry;
#[derive(Debug, Clone, Default)]
pub struct JournalExport;
#[derive(Debug, Clone, Default)]
pub struct JournalVerification;
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum JournalEntryProperty {
Id,
ReceivedAt,
Direction,
Sender,
Authenticated,
Recipients,
Subject,
MessageId,
JournalIds,
Held,
Size,
Sha256,
ExpiresAt,
Report,
Filter,
Reason,
BlobId,
Count,
Verified,
Chains,
}
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum JournalEntryValue {
Id(Id),
}
impl Property for JournalEntryProperty {
fn try_parse(parent: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {
// Keys inside a filter or a chain report stay plain keys
match parent {
None => JournalEntryProperty::parse(value),
Some(_) => None,
}
}
fn to_cow(&self) -> Cow<'static, str> {
match self {
JournalEntryProperty::Id => "id",
JournalEntryProperty::ReceivedAt => "receivedAt",
JournalEntryProperty::Direction => "direction",
JournalEntryProperty::Sender => "sender",
JournalEntryProperty::Authenticated => "authenticated",
JournalEntryProperty::Recipients => "recipients",
JournalEntryProperty::Subject => "subject",
JournalEntryProperty::MessageId => "messageId",
JournalEntryProperty::JournalIds => "journalIds",
JournalEntryProperty::Held => "held",
JournalEntryProperty::Size => "size",
JournalEntryProperty::Sha256 => "sha256",
JournalEntryProperty::ExpiresAt => "expiresAt",
JournalEntryProperty::Report => "report",
JournalEntryProperty::Filter => "filter",
JournalEntryProperty::Reason => "reason",
JournalEntryProperty::BlobId => "blobId",
JournalEntryProperty::Count => "count",
JournalEntryProperty::Verified => "verified",
JournalEntryProperty::Chains => "chains",
}
.into()
}
}
impl JournalEntryProperty {
fn parse(value: &str) -> Option<Self> {
hashify::tiny_map!(value.as_bytes(),
b"id" => JournalEntryProperty::Id,
b"receivedAt" => JournalEntryProperty::ReceivedAt,
b"direction" => JournalEntryProperty::Direction,
b"sender" => JournalEntryProperty::Sender,
b"authenticated" => JournalEntryProperty::Authenticated,
b"recipients" => JournalEntryProperty::Recipients,
b"subject" => JournalEntryProperty::Subject,
b"messageId" => JournalEntryProperty::MessageId,
b"journalIds" => JournalEntryProperty::JournalIds,
b"held" => JournalEntryProperty::Held,
b"size" => JournalEntryProperty::Size,
b"sha256" => JournalEntryProperty::Sha256,
b"expiresAt" => JournalEntryProperty::ExpiresAt,
b"report" => JournalEntryProperty::Report,
b"filter" => JournalEntryProperty::Filter,
b"reason" => JournalEntryProperty::Reason,
b"blobId" => JournalEntryProperty::BlobId,
b"count" => JournalEntryProperty::Count,
b"verified" => JournalEntryProperty::Verified,
b"chains" => JournalEntryProperty::Chains,
)
}
}
impl FromStr for JournalEntryProperty {
type Err = ();
fn from_str(s: &str) -> Result<Self, Self::Err> {
JournalEntryProperty::parse(s).ok_or(())
}
}
impl Element for JournalEntryValue {
type Property = JournalEntryProperty;
fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {
match key {
Key::Property(JournalEntryProperty::Id) => {
Id::from_str(value).ok().map(JournalEntryValue::Id)
}
_ => None,
}
}
fn to_cow(&self) -> Cow<'static, str> {
match self {
JournalEntryValue::Id(id) => id.to_string().into(),
}
}
}
/// One condition of an `inbuxa:JournalEntry/query` filter. Several in one
/// filter object must all hold.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum JournalFilter {
/// From this time on (UTC date).
After(String),
/// Before this time (UTC date).
Before(String),
/// Part of the sender's address.
Sender(String),
/// Part of a recipient's address.
Recipient(String),
/// Part of the sender's or a recipient's address.
Address(String),
/// `outgoing`, `incoming` or `internal`.
Direction(String),
/// Words that must all be in the subject.
Text(String),
MessageId(String),
JournalId(Id),
_T(String),
}
impl Default for JournalFilter {
fn default() -> Self {
JournalFilter::_T(String::new())
}
}
impl<'de> DeserializeArguments<'de> for JournalFilter {
fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>
where
A: serde::de::MapAccess<'de>,
{
hashify::fnc_map!(key.as_bytes(),
b"after" => {
*self = JournalFilter::After(map.next_value()?);
},
b"before" => {
*self = JournalFilter::Before(map.next_value()?);
},
b"sender" => {
*self = JournalFilter::Sender(map.next_value()?);
},
b"recipient" => {
*self = JournalFilter::Recipient(map.next_value()?);
},
b"address" => {
*self = JournalFilter::Address(map.next_value()?);
},
b"direction" => {
*self = JournalFilter::Direction(map.next_value()?);
},
b"text" => {
*self = JournalFilter::Text(map.next_value()?);
},
b"messageId" => {
*self = JournalFilter::MessageId(map.next_value()?);
},
b"journalId" => {
*self = JournalFilter::JournalId(map.next_value()?);
},
_ => {
*self = JournalFilter::_T(key.to_string());
let _ = map.next_value::<serde::de::IgnoredAny>()?;
}
);
Ok(())
}
}
/// Entries sort newest first, by `receivedAt`; nothing else.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum JournalComparator {
ReceivedAt,
_T(String),
}
impl Default for JournalComparator {
fn default() -> Self {
JournalComparator::_T(String::new())
}
}
impl<'de> DeserializeArguments<'de> for JournalComparator {
fn deserialize_argument<A>(&mut self, key: &str, map: &mut A) -> Result<(), A::Error>
where
A: serde::de::MapAccess<'de>,
{
if key == "property" {
let value = map.next_value::<Cow<str>>()?;
*self = if value == "receivedAt" {
JournalComparator::ReceivedAt
} else {
JournalComparator::_T(value.into_owned())
};
} else {
let _ = map.next_value::<serde::de::IgnoredAny>()?;
}
Ok(())
}
}
macro_rules! journal_object {
($object:ty, $filter:ty, $comparator:ty) => {
impl JmapObject for $object {
type Property = JournalEntryProperty;
type Element = JournalEntryValue;
type Id = Id;
type Filter = $filter;
type Comparator = $comparator;
type GetArguments = ();
type SetArguments<'de> = ();
type QueryArguments = ();
type CopyArguments = ();
type ParseArguments = ();
const ID_PROPERTY: Self::Property = JournalEntryProperty::Id;
}
};
}
journal_object!(JournalEntry, JournalFilter, JournalComparator);
journal_object!(JournalExport, (), ());
journal_object!(JournalVerification, (), ());
impl From<Id> for JournalEntryValue {
fn from(id: Id) -> Self {
JournalEntryValue::Id(id)
}
}
impl JmapObjectId for JournalEntryValue {
fn as_id(&self) -> Option<Id> {
match self {
JournalEntryValue::Id(id) => Some(*id),
}
}
fn as_any_id(&self) -> Option<AnyId> {
match self {
JournalEntryValue::Id(id) => Some(AnyId::Id(*id)),
}
}
fn as_id_ref(&self) -> Option<&str> {
None
}
fn try_set_id(&mut self, new_id: AnyId) -> bool {
if let AnyId::Id(id) = new_id {
*self = JournalEntryValue::Id(id);
true
} else {
false
}
}
}
impl JmapObjectId for JournalEntryProperty {
fn as_id(&self) -> Option<Id> {
None
}
fn as_any_id(&self) -> Option<AnyId> {
None
}
fn as_id_ref(&self) -> Option<&str> {
None
}
fn try_set_id(&mut self, _: AnyId) -> bool {
false
}
}
@@ -0,0 +1,173 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! `inbuxa:SecurityAcceptance/get` and `/set` under `urn:inbuxa:jmap`: the
//! security to-do items an administrator accepted, with why (security
//! to-do list spec, SS-23 to SS-26). Created and destroyed, never updated.
use crate::object::{AnyId, JmapObject, JmapObjectId};
use jmap_tools::{Element, Key, Property};
use std::{borrow::Cow, str::FromStr};
use types::id::Id;
#[derive(Debug, Clone, Default)]
pub struct SecurityAcceptance;
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum SecurityAcceptanceProperty {
Id,
/// `SS-1` to `SS-18`.
Check,
Subject,
AcceptedValue,
Note,
AcceptedBy,
AcceptedAt,
}
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum SecurityAcceptanceValue {
Id(Id),
}
impl Property for SecurityAcceptanceProperty {
fn try_parse(parent: Option<&Key<'_, Self>>, value: &str) -> Option<Self> {
// Keys inside acceptedValue stay plain keys
match parent {
None => SecurityAcceptanceProperty::parse(value),
Some(_) => None,
}
}
fn to_cow(&self) -> Cow<'static, str> {
match self {
SecurityAcceptanceProperty::Id => "id",
SecurityAcceptanceProperty::Check => "check",
SecurityAcceptanceProperty::Subject => "subject",
SecurityAcceptanceProperty::AcceptedValue => "acceptedValue",
SecurityAcceptanceProperty::Note => "note",
SecurityAcceptanceProperty::AcceptedBy => "acceptedBy",
SecurityAcceptanceProperty::AcceptedAt => "acceptedAt",
}
.into()
}
}
impl SecurityAcceptanceProperty {
fn parse(value: &str) -> Option<Self> {
hashify::tiny_map!(value.as_bytes(),
b"id" => SecurityAcceptanceProperty::Id,
b"check" => SecurityAcceptanceProperty::Check,
b"subject" => SecurityAcceptanceProperty::Subject,
b"acceptedValue" => SecurityAcceptanceProperty::AcceptedValue,
b"note" => SecurityAcceptanceProperty::Note,
b"acceptedBy" => SecurityAcceptanceProperty::AcceptedBy,
b"acceptedAt" => SecurityAcceptanceProperty::AcceptedAt,
)
}
}
impl FromStr for SecurityAcceptanceProperty {
type Err = ();
fn from_str(s: &str) -> Result<Self, Self::Err> {
SecurityAcceptanceProperty::parse(s).ok_or(())
}
}
impl Element for SecurityAcceptanceValue {
type Property = SecurityAcceptanceProperty;
fn try_parse<P>(key: &Key<'_, Self::Property>, value: &str) -> Option<Self> {
match key {
Key::Property(SecurityAcceptanceProperty::Id) => {
Id::from_str(value).ok().map(SecurityAcceptanceValue::Id)
}
_ => None,
}
}
fn to_cow(&self) -> Cow<'static, str> {
match self {
SecurityAcceptanceValue::Id(id) => id.to_string().into(),
}
}
}
impl JmapObject for SecurityAcceptance {
type Property = SecurityAcceptanceProperty;
type Element = SecurityAcceptanceValue;
type Id = Id;
type Filter = ();
type Comparator = ();
type GetArguments = ();
type SetArguments<'de> = ();
type QueryArguments = ();
type CopyArguments = ();
type ParseArguments = ();
const ID_PROPERTY: Self::Property = SecurityAcceptanceProperty::Id;
}
impl From<Id> for SecurityAcceptanceValue {
fn from(id: Id) -> Self {
SecurityAcceptanceValue::Id(id)
}
}
impl JmapObjectId for SecurityAcceptanceValue {
fn as_id(&self) -> Option<Id> {
match self {
SecurityAcceptanceValue::Id(id) => Some(*id),
}
}
fn as_any_id(&self) -> Option<AnyId> {
match self {
SecurityAcceptanceValue::Id(id) => Some(AnyId::Id(*id)),
}
}
fn as_id_ref(&self) -> Option<&str> {
None
}
fn try_set_id(&mut self, new_id: AnyId) -> bool {
if let AnyId::Id(id) = new_id {
*self = SecurityAcceptanceValue::Id(id);
true
} else {
false
}
}
}
impl JmapObjectId for SecurityAcceptanceProperty {
fn as_id(&self) -> Option<Id> {
None
}
fn as_any_id(&self) -> Option<AnyId> {
None
}
fn as_id_ref(&self) -> Option<&str> {
None
}
fn try_set_id(&mut self, _: AnyId) -> bool {
false
}
}
+3
View File
@@ -30,6 +30,9 @@ pub mod inbuxa_inventory_snapshot; // inbuxa: personal-data catalog
pub mod inbuxa_audit; // inbuxa: the audit log
pub mod inbuxa_legal_hold; // inbuxa: legal hold
pub mod inbuxa_mail_rule; // inbuxa: DLP and mail flow rules
pub mod inbuxa_security_acceptance; // inbuxa: accepted security to-do items
pub mod inbuxa_journal; // inbuxa: journaling
pub mod inbuxa_journal_entry; // inbuxa: journaling, search and export
pub mod inbuxa_held_message; // inbuxa: mail held for review
pub mod inbuxa_hold_export; // inbuxa: legal hold exports
pub mod inbuxa_explanation; // inbuxa: "Explain this" with the local model
+9
View File
@@ -88,6 +88,15 @@ impl Response<'_> {
GetResponseMethod::MailRule(response) => {
response.eval_jptr(path, &mut results)
}
GetResponseMethod::SecurityAcceptance(response) => {
response.eval_jptr(path, &mut results)
}
GetResponseMethod::Journal(response) => {
response.eval_jptr(path, &mut results)
}
GetResponseMethod::JournalEntry(response) => {
response.eval_jptr(path, &mut results)
}
GetResponseMethod::HeldMessage(response) => {
response.eval_jptr(path, &mut results)
}
@@ -55,6 +55,9 @@ impl Response<'_> {
GetRequestMethod::AccountLock(request) => request.resolve_references(self)?,
GetRequestMethod::LegalHold(request) => request.resolve_references(self)?,
GetRequestMethod::MailRule(request) => request.resolve_references(self)?,
GetRequestMethod::SecurityAcceptance(request) => request.resolve_references(self)?,
GetRequestMethod::Journal(request) => request.resolve_references(self)?,
GetRequestMethod::JournalEntry(request) => request.resolve_references(self)?,
GetRequestMethod::HeldMessage(request) => request.resolve_references(self)?,
GetRequestMethod::HoldExport(request) => request.resolve_references(self)?,
GetRequestMethod::ProtocolPolicy(request) => request.resolve_references(self)?,
@@ -131,6 +134,18 @@ impl Response<'_> {
SetRequestMethod::MailRule(request) => {
request.resolve_references(self, 1, false)?
}
SetRequestMethod::SecurityAcceptance(request) => {
request.resolve_references(self, 1, false)?
}
SetRequestMethod::Journal(request) => {
request.resolve_references(self, 1, false)?
}
SetRequestMethod::JournalExport(request) => {
request.resolve_references(self, 1, false)?
}
SetRequestMethod::JournalVerification(request) => {
request.resolve_references(self, 1, false)?
}
SetRequestMethod::HeldMessage(request) => {
request.resolve_references(self, 1, false)?
}
+36 -1
View File
@@ -68,7 +68,14 @@ pub enum MethodObject {
ProtocolPolicy,
// inbuxa: DLP and mail flow rules
MailRule,
// inbuxa: accepted security to-do items
SecurityAcceptance,
HeldMessage,
// inbuxa: journaling
Journal,
JournalEntry,
JournalExport,
JournalVerification,
TenantProtocolPolicy,
}
@@ -109,7 +116,12 @@ impl MethodObject {
| MethodObject::LegalHold
| MethodObject::HoldExport
| MethodObject::MailRule
| MethodObject::HeldMessage => Capability::Inbuxa,
| MethodObject::SecurityAcceptance
| MethodObject::HeldMessage
| MethodObject::Journal
| MethodObject::JournalEntry
| MethodObject::JournalExport
| MethodObject::JournalVerification => Capability::Inbuxa,
MethodObject::ProtocolPolicy => Capability::Inbuxa,
MethodObject::TenantProtocolPolicy => Capability::Inbuxa,
}
@@ -307,6 +319,16 @@ impl MethodName {
(MethodFunction::Set, MethodObject::LegalHold) => "inbuxa:LegalHold/set",
(MethodFunction::Get, MethodObject::MailRule) => "inbuxa:MailRule/get",
(MethodFunction::Set, MethodObject::MailRule) => "inbuxa:MailRule/set",
(MethodFunction::Get, MethodObject::SecurityAcceptance) => "inbuxa:SecurityAcceptance/get",
(MethodFunction::Set, MethodObject::SecurityAcceptance) => "inbuxa:SecurityAcceptance/set",
(MethodFunction::Get, MethodObject::Journal) => "inbuxa:Journal/get",
(MethodFunction::Set, MethodObject::Journal) => "inbuxa:Journal/set",
(MethodFunction::Get, MethodObject::JournalEntry) => "inbuxa:JournalEntry/get",
(MethodFunction::Query, MethodObject::JournalEntry) => "inbuxa:JournalEntry/query",
(MethodFunction::Set, MethodObject::JournalExport) => "inbuxa:JournalExport/set",
(MethodFunction::Set, MethodObject::JournalVerification) => {
"inbuxa:JournalVerification/set"
}
(MethodFunction::Get, MethodObject::HeldMessage) => "inbuxa:HeldMessage/get",
(MethodFunction::Set, MethodObject::HeldMessage) => "inbuxa:HeldMessage/set",
(MethodFunction::Get, MethodObject::HoldExport) => "inbuxa:HoldExport/get",
@@ -465,6 +487,14 @@ impl MethodName {
"inbuxa:LegalHold/set" => (MethodObject::LegalHold, MethodFunction::Set),
"inbuxa:MailRule/get" => (MethodObject::MailRule, MethodFunction::Get),
"inbuxa:MailRule/set" => (MethodObject::MailRule, MethodFunction::Set),
"inbuxa:SecurityAcceptance/get" => (MethodObject::SecurityAcceptance, MethodFunction::Get),
"inbuxa:SecurityAcceptance/set" => (MethodObject::SecurityAcceptance, MethodFunction::Set),
"inbuxa:Journal/get" => (MethodObject::Journal, MethodFunction::Get),
"inbuxa:Journal/set" => (MethodObject::Journal, MethodFunction::Set),
"inbuxa:JournalEntry/get" => (MethodObject::JournalEntry, MethodFunction::Get),
"inbuxa:JournalEntry/query" => (MethodObject::JournalEntry, MethodFunction::Query),
"inbuxa:JournalExport/set" => (MethodObject::JournalExport, MethodFunction::Set),
"inbuxa:JournalVerification/set" => (MethodObject::JournalVerification, MethodFunction::Set),
"inbuxa:HeldMessage/get" => (MethodObject::HeldMessage, MethodFunction::Get),
"inbuxa:HeldMessage/set" => (MethodObject::HeldMessage, MethodFunction::Set),
"inbuxa:HoldExport/get" => (MethodObject::HoldExport, MethodFunction::Get),
@@ -539,6 +569,11 @@ impl Display for MethodObject {
MethodObject::AccountLock => "inbuxa:AccountLock",
MethodObject::LegalHold => "inbuxa:LegalHold",
MethodObject::MailRule => "inbuxa:MailRule",
MethodObject::SecurityAcceptance => "inbuxa:SecurityAcceptance",
MethodObject::Journal => "inbuxa:Journal",
MethodObject::JournalEntry => "inbuxa:JournalEntry",
MethodObject::JournalExport => "inbuxa:JournalExport",
MethodObject::JournalVerification => "inbuxa:JournalVerification",
MethodObject::HeldMessage => "inbuxa:HeldMessage",
MethodObject::HoldExport => "inbuxa:HoldExport",
MethodObject::ProtocolPolicy => "inbuxa:ProtocolPolicy",
+10
View File
@@ -125,6 +125,9 @@ pub enum GetRequestMethod {
AccountLock(Box<GetRequest<crate::object::inbuxa_account_lock::AccountLock>>),
LegalHold(Box<GetRequest<crate::object::inbuxa_legal_hold::LegalHold>>),
MailRule(Box<GetRequest<crate::object::inbuxa_mail_rule::MailRule>>),
SecurityAcceptance(Box<GetRequest<crate::object::inbuxa_security_acceptance::SecurityAcceptance>>),
Journal(Box<GetRequest<crate::object::inbuxa_journal::Journal>>),
JournalEntry(Box<GetRequest<crate::object::inbuxa_journal_entry::JournalEntry>>),
HeldMessage(Box<GetRequest<crate::object::inbuxa_held_message::HeldMessage>>),
HoldExport(Box<GetRequest<crate::object::inbuxa_hold_export::HoldExport>>),
ProtocolPolicy(Box<GetRequest<crate::object::inbuxa_protocol_policy::ProtocolPolicy>>),
@@ -163,6 +166,12 @@ pub enum SetRequestMethod<'x> {
AccountLock(Box<SetRequest<'x, crate::object::inbuxa_account_lock::AccountLock>>),
LegalHold(Box<SetRequest<'x, crate::object::inbuxa_legal_hold::LegalHold>>),
MailRule(Box<SetRequest<'x, crate::object::inbuxa_mail_rule::MailRule>>),
SecurityAcceptance(
Box<SetRequest<'x, crate::object::inbuxa_security_acceptance::SecurityAcceptance>>,
),
Journal(Box<SetRequest<'x, crate::object::inbuxa_journal::Journal>>),
JournalExport(Box<SetRequest<'x, crate::object::inbuxa_journal_entry::JournalExport>>),
JournalVerification(Box<SetRequest<'x, crate::object::inbuxa_journal_entry::JournalVerification>>),
HeldMessage(Box<SetRequest<'x, crate::object::inbuxa_held_message::HeldMessage>>),
HoldExport(Box<SetRequest<'x, crate::object::inbuxa_hold_export::HoldExport>>),
ProtocolPolicy(Box<SetRequest<'x, crate::object::inbuxa_protocol_policy::ProtocolPolicy>>),
@@ -197,6 +206,7 @@ pub enum QueryRequestMethod {
ShareNotification(Box<QueryRequest<ShareNotification>>),
Registry(Box<QueryRequest<Registry>>),
AuditEvent(Box<QueryRequest<crate::object::inbuxa_audit::AuditEvent>>),
JournalEntry(Box<QueryRequest<crate::object::inbuxa_journal_entry::JournalEntry>>),
}
#[derive(Debug)]
+58
View File
@@ -653,6 +653,64 @@ impl<'de> Visitor<'de> for CallVisitor {
return Err(de::Error::invalid_length(1, &self));
}
},
// inbuxa: accepted security to-do items
(MethodFunction::Get, MethodObject::SecurityAcceptance) => match seq.next_element() {
Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::SecurityAcceptance(value)),
Err(err) => RequestMethod::invalid(err),
Ok(None) => {
return Err(de::Error::invalid_length(1, &self));
}
},
(MethodFunction::Set, MethodObject::SecurityAcceptance) => match seq.next_element() {
Ok(Some(value)) => RequestMethod::Set(SetRequestMethod::SecurityAcceptance(value)),
Err(err) => RequestMethod::invalid(err),
Ok(None) => {
return Err(de::Error::invalid_length(1, &self));
}
},
// inbuxa: journaling
(MethodFunction::Get, MethodObject::JournalEntry) => match seq.next_element() {
Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::JournalEntry(value)),
Err(err) => RequestMethod::invalid(err),
Ok(None) => {
return Err(de::Error::invalid_length(1, &self));
}
},
(MethodFunction::Query, MethodObject::JournalEntry) => match seq.next_element() {
Ok(Some(value)) => RequestMethod::Query(QueryRequestMethod::JournalEntry(value)),
Err(err) => RequestMethod::invalid(err),
Ok(None) => {
return Err(de::Error::invalid_length(1, &self));
}
},
(MethodFunction::Set, MethodObject::JournalExport) => match seq.next_element() {
Ok(Some(value)) => RequestMethod::Set(SetRequestMethod::JournalExport(value)),
Err(err) => RequestMethod::invalid(err),
Ok(None) => {
return Err(de::Error::invalid_length(1, &self));
}
},
(MethodFunction::Set, MethodObject::JournalVerification) => match seq.next_element() {
Ok(Some(value)) => RequestMethod::Set(SetRequestMethod::JournalVerification(value)),
Err(err) => RequestMethod::invalid(err),
Ok(None) => {
return Err(de::Error::invalid_length(1, &self));
}
},
(MethodFunction::Get, MethodObject::Journal) => match seq.next_element() {
Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::Journal(value)),
Err(err) => RequestMethod::invalid(err),
Ok(None) => {
return Err(de::Error::invalid_length(1, &self));
}
},
(MethodFunction::Set, MethodObject::Journal) => match seq.next_element() {
Ok(Some(value)) => RequestMethod::Set(SetRequestMethod::Journal(value)),
Err(err) => RequestMethod::invalid(err),
Ok(None) => {
return Err(de::Error::invalid_length(1, &self));
}
},
// inbuxa: legal hold
(MethodFunction::Get, MethodObject::LegalHold) => match seq.next_element() {
Ok(Some(value)) => RequestMethod::Get(GetRequestMethod::LegalHold(value)),
+57
View File
@@ -112,6 +112,9 @@ pub enum GetResponseMethod {
AccountLock(GetResponse<crate::object::inbuxa_account_lock::AccountLock>),
LegalHold(GetResponse<crate::object::inbuxa_legal_hold::LegalHold>),
MailRule(GetResponse<crate::object::inbuxa_mail_rule::MailRule>),
SecurityAcceptance(GetResponse<crate::object::inbuxa_security_acceptance::SecurityAcceptance>),
Journal(GetResponse<crate::object::inbuxa_journal::Journal>),
JournalEntry(GetResponse<crate::object::inbuxa_journal_entry::JournalEntry>),
HeldMessage(GetResponse<crate::object::inbuxa_held_message::HeldMessage>),
HoldExport(GetResponse<crate::object::inbuxa_hold_export::HoldExport>),
ProtocolPolicy(GetResponse<crate::object::inbuxa_protocol_policy::ProtocolPolicy>),
@@ -150,6 +153,12 @@ pub enum SetResponseMethod {
AccountLock(Box<SetResponse<crate::object::inbuxa_account_lock::AccountLock>>),
LegalHold(Box<SetResponse<crate::object::inbuxa_legal_hold::LegalHold>>),
MailRule(Box<SetResponse<crate::object::inbuxa_mail_rule::MailRule>>),
SecurityAcceptance(
Box<SetResponse<crate::object::inbuxa_security_acceptance::SecurityAcceptance>>,
),
Journal(Box<SetResponse<crate::object::inbuxa_journal::Journal>>),
JournalExport(Box<SetResponse<crate::object::inbuxa_journal_entry::JournalExport>>),
JournalVerification(Box<SetResponse<crate::object::inbuxa_journal_entry::JournalVerification>>),
HeldMessage(Box<SetResponse<crate::object::inbuxa_held_message::HeldMessage>>),
HoldExport(Box<SetResponse<crate::object::inbuxa_hold_export::HoldExport>>),
Explanation(Box<SetResponse<crate::object::inbuxa_explanation::Explanation>>),
@@ -829,6 +838,23 @@ impl<'x> From<SetResponse<crate::object::inbuxa_held_message::HeldMessage>> for
}
}
// inbuxa: accepted security to-do items
impl<'x> From<GetResponse<crate::object::inbuxa_security_acceptance::SecurityAcceptance>>
for ResponseMethod<'x>
{
fn from(value: GetResponse<crate::object::inbuxa_security_acceptance::SecurityAcceptance>) -> Self {
ResponseMethod::Get(GetResponseMethod::SecurityAcceptance(value))
}
}
impl<'x> From<SetResponse<crate::object::inbuxa_security_acceptance::SecurityAcceptance>>
for ResponseMethod<'x>
{
fn from(value: SetResponse<crate::object::inbuxa_security_acceptance::SecurityAcceptance>) -> Self {
ResponseMethod::Set(SetResponseMethod::SecurityAcceptance(Box::new(value)))
}
}
impl<'x> From<GetResponse<crate::object::inbuxa_mail_rule::MailRule>> for ResponseMethod<'x> {
fn from(value: GetResponse<crate::object::inbuxa_mail_rule::MailRule>) -> Self {
ResponseMethod::Get(GetResponseMethod::MailRule(value))
@@ -841,6 +867,37 @@ impl<'x> From<SetResponse<crate::object::inbuxa_mail_rule::MailRule>> for Respon
}
}
// inbuxa: journaling
impl<'x> From<GetResponse<crate::object::inbuxa_journal_entry::JournalEntry>> for ResponseMethod<'x> {
fn from(value: GetResponse<crate::object::inbuxa_journal_entry::JournalEntry>) -> Self {
ResponseMethod::Get(GetResponseMethod::JournalEntry(value))
}
}
impl<'x> From<SetResponse<crate::object::inbuxa_journal_entry::JournalExport>> for ResponseMethod<'x> {
fn from(value: SetResponse<crate::object::inbuxa_journal_entry::JournalExport>) -> Self {
ResponseMethod::Set(SetResponseMethod::JournalExport(Box::new(value)))
}
}
impl<'x> From<SetResponse<crate::object::inbuxa_journal_entry::JournalVerification>> for ResponseMethod<'x> {
fn from(value: SetResponse<crate::object::inbuxa_journal_entry::JournalVerification>) -> Self {
ResponseMethod::Set(SetResponseMethod::JournalVerification(Box::new(value)))
}
}
impl<'x> From<GetResponse<crate::object::inbuxa_journal::Journal>> for ResponseMethod<'x> {
fn from(value: GetResponse<crate::object::inbuxa_journal::Journal>) -> Self {
ResponseMethod::Get(GetResponseMethod::Journal(value))
}
}
impl<'x> From<SetResponse<crate::object::inbuxa_journal::Journal>> for ResponseMethod<'x> {
fn from(value: SetResponse<crate::object::inbuxa_journal::Journal>) -> Self {
ResponseMethod::Set(SetResponseMethod::Journal(Box::new(value)))
}
}
impl<'x> From<GetResponse<crate::object::inbuxa_legal_hold::LegalHold>> for ResponseMethod<'x> {
fn from(value: GetResponse<crate::object::inbuxa_legal_hold::LegalHold>) -> Self {
ResponseMethod::Get(GetResponseMethod::LegalHold(value))
+50
View File
@@ -116,7 +116,13 @@ impl JmapAuthorization for AccessToken {
Permission::SysDlpPolicyGet
}
}
// inbuxa: journaling (JR-18)
GetRequestMethod::Journal(_) => Permission::SysJournalGet,
GetRequestMethod::JournalEntry(_) => Permission::SysJournalSearch,
GetRequestMethod::HoldExport(_) => Permission::SysLegalHoldExport,
// inbuxa: accepted security items are read by whoever may
// see the server's security settings
GetRequestMethod::SecurityAcceptance(_) => Permission::SysSecurityGet,
// inbuxa: legacy protocols off. It takes listeners away and
// puts them back, so it takes the listener's permissions
GetRequestMethod::ProtocolPolicy(_) => Permission::SysNetworkListenerGet,
@@ -288,6 +294,43 @@ impl JmapAuthorization for AccessToken {
.details("You are not authorized to change mail rules"))
}
}
// inbuxa: journaling (JR-18)
SetRequestMethod::Journal(s) => validate_set(
s,
self,
Permission::SysJournalUpdate,
Permission::SysJournalUpdate,
Permission::SysJournalUpdate,
),
SetRequestMethod::JournalExport(s) => validate_set(
s,
self,
Permission::SysJournalExport,
Permission::SysJournalExport,
Permission::SysJournalExport,
),
SetRequestMethod::JournalVerification(s) => validate_set(
s,
self,
Permission::SysJournalGet,
Permission::SysJournalGet,
Permission::SysJournalGet,
),
// inbuxa: accepting a security to-do item, or removing
// an acceptance; nothing is ever edited
SetRequestMethod::SecurityAcceptance(s) => {
if s.update.as_ref().is_some_and(|u| !u.is_empty()) {
Err(trc::JmapEvent::Forbidden
.into_err()
.details("An acceptance is replaced, not edited"))
} else if self.has_permission(Permission::SysSecurityAccept) {
Ok(())
} else {
Err(trc::JmapEvent::Forbidden
.into_err()
.details("You are not authorized to accept security items"))
}
}
// inbuxa: LH-12, exporting held data
SetRequestMethod::HoldExport(s) => validate_set(
s,
@@ -450,7 +493,12 @@ impl JmapAuthorization for AccessToken {
| MethodObject::LegalHold
| MethodObject::HoldExport
| MethodObject::MailRule
| MethodObject::SecurityAcceptance
| MethodObject::HeldMessage
| MethodObject::Journal
| MethodObject::JournalEntry
| MethodObject::JournalExport
| MethodObject::JournalVerification
| MethodObject::ProtocolPolicy
| MethodObject::TenantProtocolPolicy => Permission::JmapEmailChanges,
// inbuxa: x:MaskedEmail/changes reads what /get reads
@@ -509,6 +557,8 @@ impl JmapAuthorization for AccessToken {
QueryRequestMethod::ShareNotification(_) => Permission::JmapShareNotificationQuery,
// inbuxa: the audit log (AU-9)
QueryRequestMethod::AuditEvent(_) => Permission::SysAuditGet,
// inbuxa: journaling (JR-15)
QueryRequestMethod::JournalEntry(_) => Permission::SysJournalSearch,
QueryRequestMethod::Registry(_) => {
let MethodObject::Registry(object_type) = object else {
unreachable!()
+89
View File
@@ -285,6 +285,18 @@ impl RequestHandler for Server {
SetResponseMethod::MailRule(set_response) => {
set_response.update_created_ids(&mut response);
}
SetResponseMethod::SecurityAcceptance(set_response) => {
set_response.update_created_ids(&mut response);
}
SetResponseMethod::Journal(set_response) => {
set_response.update_created_ids(&mut response);
}
SetResponseMethod::JournalExport(set_response) => {
set_response.update_created_ids(&mut response);
}
SetResponseMethod::JournalVerification(set_response) => {
set_response.update_created_ids(&mut response);
}
SetResponseMethod::HeldMessage(set_response) => {
set_response.update_created_ids(&mut response);
}
@@ -512,6 +524,24 @@ impl RequestHandler for Server {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
crate::inbuxa::mail_rule::get(self, access_token, *req).await?.into()
}
// inbuxa: accepted security to-do items
GetRequestMethod::SecurityAcceptance(mut req) => {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
crate::inbuxa::security_acceptance::get(self, access_token, *req)
.await?
.into()
}
// inbuxa: journaling
GetRequestMethod::Journal(mut req) => {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
crate::inbuxa::journal::get(self, access_token, *req).await?.into()
}
GetRequestMethod::JournalEntry(mut req) => {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
crate::inbuxa::journal_entry::get(self, access_token, session, *req)
.await?
.into()
}
// inbuxa: the audit log (AU-9)
GetRequestMethod::AuditEvent(mut req) => {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
@@ -708,6 +738,13 @@ impl RequestHandler for Server {
.await?
.into()
}
// inbuxa: journaling (JR-15)
QueryRequestMethod::JournalEntry(mut req) => {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
crate::inbuxa::journal_entry::query(self, access_token, session, *req)
.await?
.into()
}
QueryRequestMethod::Registry(mut req) => {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
assert_registry_account(self, method_name.obj, access_token, req.account_id)
@@ -985,6 +1022,45 @@ impl RequestHandler for Server {
.await?
.into()
}
// inbuxa: SS-26, every acceptance made or removed is in the
// audit log
SetRequestMethod::SecurityAcceptance(mut req) => {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
crate::inbuxa::audit::recorded(
self,
access_token,
session,
&method_name.obj.to_string(),
None,
None,
*req,
|req| {
Box::pin(crate::inbuxa::security_acceptance::set(
self,
access_token,
req,
))
},
)
.await?
.into()
}
SetRequestMethod::Journal(mut req) => {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
let reason = req.arguments.reason.clone();
crate::inbuxa::audit::recorded(
self,
access_token,
session,
&method_name.obj.to_string(),
None,
reason,
*req,
|req| Box::pin(crate::inbuxa::journal::set(self, access_token, req)),
)
.await?
.into()
}
SetRequestMethod::AuditExport(mut req) => {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
crate::inbuxa::audit_log::export_set(self, access_token, session, *req)
@@ -997,6 +1073,19 @@ impl RequestHandler for Server {
.await?
.into()
}
// inbuxa: journaling (JR-6, JR-16)
SetRequestMethod::JournalExport(mut req) => {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
crate::inbuxa::journal_entry::export_set(self, access_token, session, *req)
.await?
.into()
}
SetRequestMethod::JournalVerification(mut req) => {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
crate::inbuxa::journal_entry::verification_set(self, access_token, session, *req)
.await?
.into()
}
// inbuxa: inbuxa:Explanation/set ("Explain this")
SetRequestMethod::Explanation(mut req) => {
resolve_account_id(&mut req.account_id, method_name.obj, access_token)?;
+5
View File
@@ -431,6 +431,11 @@ impl IntermediateChangesResponse {
| MethodObject::LegalHold
| MethodObject::HoldExport
| MethodObject::MailRule
| MethodObject::SecurityAcceptance
| MethodObject::Journal
| MethodObject::JournalEntry
| MethodObject::JournalExport
| MethodObject::JournalVerification
| MethodObject::HeldMessage
| MethodObject::ProtocolPolicy
| MethodObject::TenantProtocolPolicy
+25 -1
View File
@@ -217,7 +217,11 @@ async fn before<T: JmapObject>(
if let Some(MaybeResultReference::Value(destroy)) = &request.destroy {
for id in destroy {
let before = stored(server, registry, id).await;
// inbuxa: a fork object is named from its own store, as an update is
let before = match registry {
Some(_) => stored(server, registry, id).await,
None => fork_current(server, object, id).await,
};
let mut described = before.as_ref().map(diff::describe).unwrap_or_default();
if let Some(before) = &before {
described.name = full_name(server, object, before, described.name).await;
@@ -434,6 +438,26 @@ async fn fork_current(server: &Server, object: &str, id: &MaybeInvalid<Id>) -> O
}
MaybeInvalid::Invalid(_) => None,
},
// SS-26: an acceptance named by its check and subject
"inbuxa:SecurityAcceptance" => match id {
MaybeInvalid::Value(id) => {
let acceptance =
security::acceptance::get(data, u32::try_from(id.id()).ok()?)
.await
.ok()??;
let name = match acceptance.subject.as_str() {
"" => acceptance.check.clone(),
subject => format!("{} {subject}", acceptance.check),
};
Some(serde_json::json!({
"name": name,
"check": acceptance.check,
"subject": acceptance.subject,
"note": acceptance.note,
}))
}
MaybeInvalid::Invalid(_) => None,
},
"inbuxa:TenantProtocolPolicy" => match id {
MaybeInvalid::Value(id) => {
security::tenant_protocol_policy::get(data, id.document_id())
+318
View File
@@ -0,0 +1,318 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! `inbuxa:Journal` (journaling spec, JR-9, JR-12, JR-18): journals, seen
//! with `sysJournalGet` and changed with `sysJournalUpdate`, which the
//! request layer checks. Journals are the server's: nobody in a tenant
//! reaches them. The request layer records every change in the audit log.
//! Changing or removing a journal never touches what it has taken.
use common::{Server, auth::AccessToken};
use inbuxa_features::journal::{
self, Journal as Stored,
archive::{self, Failures},
};
use jmap_proto::{
error::set::SetError,
method::{
get::{GetRequest, GetResponse},
set::{SetRequest, SetResponse},
},
object::inbuxa_journal::{Journal, JournalProperty as P, JournalValue},
request::IntoValid,
types::date::UTCDate,
};
use jmap_tools::{Key, Map, Property, Value};
use std::borrow::Cow;
use store::write::now;
use types::id::Id;
type JValue = Value<'static, P, JournalValue>;
const ALL: &[P] = &[
P::Id,
P::Name,
P::Description,
P::Enabled,
P::Direction,
P::Scope,
P::RetentionDays,
P::BuiltIn,
P::ArchiveAddress,
P::ArchiveFailures,
P::CreatedBy,
P::CreatedAt,
P::UpdatedAt,
];
/// Properties the server sets; a client that sends them is refused.
const SERVER_SET: &[P] = &[
P::Id,
P::ArchiveFailures,
P::CreatedBy,
P::CreatedAt,
P::UpdatedAt,
];
fn server_level(access_token: &AccessToken) -> trc::Result<()> {
if access_token.tenant_id().is_some() {
Err(trc::JmapEvent::Forbidden
.into_err()
.details("Journals are the server's."))
} else {
Ok(())
}
}
fn json_to_value(json: serde_json::Value) -> JValue {
match json {
serde_json::Value::Null => Value::Null,
serde_json::Value::Bool(b) => Value::Bool(b),
serde_json::Value::Number(n) => {
if let Some(n) = n.as_u64() {
Value::Number(n.into())
} else if let Some(n) = n.as_i64() {
Value::Number(n.into())
} else {
Value::Number(n.as_f64().unwrap_or_default().into())
}
}
serde_json::Value::String(s) => Value::Str(Cow::Owned(s)),
serde_json::Value::Array(items) => {
Value::Array(items.into_iter().map(json_to_value).collect())
}
serde_json::Value::Object(map) => {
let mut out = Map::with_capacity(map.len());
for (key, value) in map {
out.insert_unchecked(Key::Owned(key), json_to_value(value));
}
Value::Object(out)
}
}
}
fn date(seconds: u64) -> JValue {
Value::Str(UTCDate::from_timestamp(seconds as i64).to_string().into())
}
fn to_value(journal: &Stored, failures: &Failures, properties: &[P]) -> JValue {
let json = serde_json::to_value(journal).unwrap_or_default();
let mut out = Map::with_capacity(properties.len());
for property in properties {
let value = match property {
P::Id => Value::Element(JournalValue::Id(Id::from(journal.id))),
P::CreatedAt => date(journal.created_at),
P::UpdatedAt => date(journal.updated_at),
P::ArchiveAddress => journal
.archive_address
.as_ref()
.map_or(Value::Null, |a| Value::Str(a.clone().into())),
// JR-7: what the console warns about
P::ArchiveFailures => {
let mut out = Map::with_capacity(3);
out.insert_unchecked(Key::Borrowed("count"), Value::Number(failures.count.into()));
out.insert_unchecked(
Key::Borrowed("lastAt"),
if failures.count > 0 {
date(failures.last_at)
} else {
Value::Null
},
);
out.insert_unchecked(
Key::Borrowed("lastReason"),
if failures.count > 0 {
Value::Str(failures.last_reason.clone().into())
} else {
Value::Null
},
);
Value::Object(out)
}
other => json
.get(other.to_cow().as_ref())
.cloned()
.map_or(Value::Null, json_to_value),
};
out.insert_unchecked(Key::Property(property.clone()), value);
}
Value::Object(out)
}
/// A journal as sent: its JSON object, top-level keys only those a client
/// may set.
fn client_json(
value: Value<'_, P, JournalValue>,
) -> Result<serde_json::Map<String, serde_json::Value>, SetError<P>> {
let mut map = serde_json::Map::new();
for (key, value) in value.into_expanded_object() {
match &key {
Key::Property(p) if SERVER_SET.contains(p) => {
return Err(SetError::invalid_properties()
.with_property(p.clone())
.with_description("The server sets this."));
}
Key::Property(p) => {
map.insert(p.to_cow().into_owned(), value.into());
}
_ => {
return Err(SetError::invalid_properties().with_property(key.clone().into_owned()));
}
}
}
Ok(map)
}
fn parse(json: serde_json::Map<String, serde_json::Value>) -> Result<Stored, SetError<P>> {
let journal: Stored =
serde_json::from_value(serde_json::Value::Object(json)).map_err(|err| {
SetError::invalid_properties().with_description(format!("Not a valid journal: {err}"))
})?;
journal.validate().map_err(|invalid| {
let property = invalid.property.parse::<P>().unwrap_or(P::Name);
SetError::invalid_properties()
.with_property(property)
.with_description(invalid.reason)
})?;
Ok(journal)
}
fn journal_id(id: Id) -> Option<u32> {
u32::try_from(id.id()).ok()
}
/// `inbuxa:Journal/get`: every journal, oldest first.
pub async fn get(
server: &Server,
access_token: &AccessToken,
mut request: GetRequest<Journal>,
) -> trc::Result<GetResponse<Journal>> {
server_level(access_token)?;
let properties = request.unwrap_properties(ALL);
let (ids, not_found) = request.unwrap_ids(server.core.jmap.get_max_objects)?;
let mut response = GetResponse {
account_id: request.account_id.into(),
state: None,
list: Vec::new(),
not_found,
};
let journals = journal::all(server.store()).await?;
let wanted: Vec<&Stored> = match ids {
None => journals.iter().collect(),
Some(ids) => {
let mut wanted = Vec::with_capacity(ids.len());
for id in ids {
match journal_id(id).and_then(|id| journals.iter().find(|j| j.id == id)) {
Some(journal) => wanted.push(journal),
None => response.push_not_found(id),
}
}
wanted
}
};
for journal in wanted {
let failures = if properties.contains(&P::ArchiveFailures) {
archive::failures(server.store(), journal.id).await?
} else {
Failures::default()
};
response
.list
.push(to_value(journal, &failures, &properties));
}
Ok(response)
}
/// `inbuxa:Journal/set`: create, change or remove journals.
pub async fn set(
server: &Server,
access_token: &AccessToken,
mut request: SetRequest<'_, Journal>,
) -> trc::Result<SetResponse<Journal>> {
server_level(access_token)?;
let mut response = SetResponse::from_request(&request, server.core.jmap.set_max_objects)?;
let data = server.store();
let actor = server.audit_actor(access_token).await;
for (client_id, value) in request.unwrap_create() {
let stored = match client_json(value).and_then(parse) {
Ok(stored) => stored,
Err(error) => {
response.not_created.append(client_id, error);
continue;
}
};
let at = now();
let stored = Stored {
created_by: actor.name.clone(),
created_at: at,
updated_at: at,
..stored
};
let id = journal::create(data, &stored).await?;
let mut out = Map::with_capacity(1);
out.insert_unchecked(
Key::Property(P::Id),
Value::Element(JournalValue::Id(Id::from(id))),
);
response.created.insert(client_id, Value::Object(out));
}
for (id, value) in request.unwrap_update().into_valid() {
let Some(current) = (match journal_id(id) {
Some(journal_id) => journal::get(data, journal_id).await?,
None => None,
}) else {
response.not_updated.append(id, SetError::not_found());
continue;
};
// The stored journal, with each property sent replacing its own
let mut json = match serde_json::to_value(&current) {
Ok(serde_json::Value::Object(map)) => map,
_ => serde_json::Map::new(),
};
let changes = match client_json(value) {
Ok(changes) => changes,
Err(error) => {
response.not_updated.append(id, error);
continue;
}
};
json.extend(changes);
let next = match parse(json) {
Ok(next) => next,
Err(error) => {
response.not_updated.append(id, error);
continue;
}
};
let next = Stored {
id: current.id,
created_by: current.created_by.clone(),
created_at: current.created_at,
updated_at: now(),
..next
};
if next != current {
journal::update(data, &next).await?;
}
response.updated.append(id, None);
}
for id in request.unwrap_destroy().into_valid() {
let Some(current) = (match journal_id(id) {
Some(journal_id) => journal::get(data, journal_id).await?,
None => None,
}) else {
response.not_destroyed.append(id, SetError::not_found());
continue;
};
journal::delete(data, current.id).await?;
response.destroyed.push(id);
}
Ok(response)
}
+791
View File
@@ -0,0 +1,791 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! The journal over JMAP (journaling spec, JR-6, JR-15 to JR-17):
//!
//! - `inbuxa:JournalEntry/query` and `/get`: searching and reading what was
//! journaled (`sysJournalSearch`). `report` is the whole journal report,
//! only when asked for.
//! - `inbuxa:JournalExport/set`: a ZIP of the reports a filter matches, in
//! the hold export's shape (`sysJournalExport`).
//! - `inbuxa:JournalVerification/set`: rechecks every chain and every report
//! (`sysJournalGet`).
//!
//! Every search, read and export is written to the audit log first; if it
//! can't be, nothing is returned (JR-17). All of it is the server's: nobody
//! in a tenant reaches it.
use common::{Server, auth::AccessToken};
use http_proto::HttpSessionData;
use inbuxa_features::{
audit::{Action, Outcome, Record, Target},
journal::{
Direction,
entries::{self, ChainReport, Entry, EntryId, Filter, MAX_QUERY_LIMIT},
},
};
use jmap_proto::{
error::set::SetError,
method::{
get::{GetRequest, GetResponse},
query::{Filter as QueryFilter, QueryRequest, QueryResponse},
set::{SetRequest, SetResponse},
},
object::inbuxa_journal_entry::{
JournalEntry, JournalEntryProperty as P, JournalEntryValue, JournalExport, JournalFilter,
JournalVerification,
},
request::IntoValid,
types::{date::UTCDate, state::State},
};
use jmap_tools::{Key, Map, Value};
use sha2::{Digest, Sha256};
use std::{
borrow::Cow,
io::{Cursor, Write},
str::FromStr,
};
use types::id::Id;
use zip::{CompressionMethod, ZipWriter, write::SimpleFileOptions};
type JValue = Value<'static, P, JournalEntryValue>;
/// Properties a get returns unless asked otherwise: all but the report.
const LISTED: &[P] = &[
P::Id,
P::ReceivedAt,
P::Direction,
P::Sender,
P::Authenticated,
P::Recipients,
P::Subject,
P::MessageId,
P::JournalIds,
P::Held,
P::Size,
P::Sha256,
P::ExpiresAt,
];
/// Most reports one export holds, and most bytes.
const MAX_EXPORT_ENTRIES: usize = 10_000;
const MAX_EXPORT_BYTES: u64 = 1024 * 1024 * 1024;
/// Most of one report `get` returns as text.
const MAX_REPORT_TEXT: usize = 10 * 1024 * 1024;
fn server_level(access_token: &AccessToken) -> trc::Result<()> {
if access_token.tenant_id().is_some() {
Err(trc::JmapEvent::Forbidden
.into_err()
.details("The journal is the server's."))
} else {
Ok(())
}
}
fn date(seconds: u64) -> JValue {
Value::Str(UTCDate::from_timestamp(seconds as i64).to_string().into())
}
fn text(value: &str) -> JValue {
Value::Str(value.to_string().into())
}
fn json_to_value(json: serde_json::Value) -> JValue {
match json {
serde_json::Value::Null => Value::Null,
serde_json::Value::Bool(b) => Value::Bool(b),
serde_json::Value::Number(n) => match n.as_u64() {
Some(n) => Value::Number(n.into()),
None => Value::Number(n.as_i64().unwrap_or_default().into()),
},
serde_json::Value::String(s) => Value::Str(Cow::Owned(s)),
serde_json::Value::Array(items) => {
Value::Array(items.into_iter().map(json_to_value).collect())
}
serde_json::Value::Object(map) => {
let mut out = Map::with_capacity(map.len());
for (key, value) in map {
out.insert_unchecked(Key::Owned(key), json_to_value(value));
}
Value::Object(out)
}
}
}
fn entry_value(id: EntryId, entry: &Entry, report: Option<&str>, properties: &[P]) -> JValue {
let mut out = Map::with_capacity(properties.len());
for property in properties {
let value = match property {
P::Id => Value::Element(JournalEntryValue::Id(Id::new(id.to_u64()))),
P::ReceivedAt => date(entry.at),
P::Direction => text(entry.direction.as_str()),
P::Sender => text(&entry.sender),
P::Authenticated => Value::Bool(entry.authenticated),
P::Recipients => Value::Array(entry.recipients.iter().map(|r| text(r)).collect()),
P::Subject => text(&entry.subject),
P::MessageId => text(&entry.message_id),
P::JournalIds => Value::Array(
entry
.journals
.iter()
.map(|j| text(&Id::from(*j).to_string()))
.collect(),
),
P::Held => Value::Bool(entry.held),
P::Size => Value::Number(entry.size.into()),
P::Sha256 => text(&entry.sha256),
P::ExpiresAt => date(entry.expires_at),
P::Report => report.map_or(Value::Null, text),
_ => Value::Null,
};
out.insert_unchecked(Key::Property(property.clone()), value);
}
Value::Object(out)
}
/// Writes a record before anything is returned; an error means nothing
/// may be (JR-17).
async fn record(
server: &Server,
access_token: &AccessToken,
session: &HttpSessionData,
action: Action,
target_id: Option<String>,
target_name: Option<String>,
details: String,
reason: Option<String>,
) -> trc::Result<()> {
server
.audit_append(&Record {
at: store::write::now() * 1000,
actor: server.audit_actor(access_token).await,
via: access_token.origin().cloned(),
remote_ip: Some(session.remote_ip),
action,
target: Target {
kind: "inbuxa:JournalEntry".into(),
id: target_id,
name: target_name,
..Default::default()
},
changes: vec![],
details: Some(details),
reason,
outcome: Outcome::success(),
})
.await
.map(|_| ())
.map_err(|err| {
err.details("The audit log couldn't be written, so the journal wasn't read.")
})
}
async fn report_bytes(server: &Server, entry: &Entry) -> trc::Result<Option<Vec<u8>>> {
match entry.blob_hash() {
Some(hash) => {
server
.blob_store()
.get_blob(hash.as_slice(), 0..usize::MAX)
.await
}
None => Ok(None),
}
}
/// `inbuxa:JournalEntry/get`: the entries named. Listing them is recorded
/// once; each report read is recorded on its own.
pub async fn get(
server: &Server,
access_token: &AccessToken,
session: &HttpSessionData,
mut request: GetRequest<JournalEntry>,
) -> trc::Result<GetResponse<JournalEntry>> {
server_level(access_token)?;
let properties = request.unwrap_properties(LISTED);
let (ids, not_found) = request.unwrap_ids(server.core.jmap.get_max_objects)?;
let mut response = GetResponse {
account_id: request.account_id.into(),
state: None,
list: Vec::new(),
not_found,
};
let Some(ids) = ids else {
return Err(trc::JmapEvent::RequestTooLarge
.into_err()
.details("Name the entries to get; use inbuxa:JournalEntry/query to find them."));
};
let mut found = Vec::with_capacity(ids.len());
for id in ids {
let entry_id = EntryId::from_u64(id.id());
match entries::get(server.store(), entry_id).await? {
Some(entry) => found.push((entry_id, entry)),
None => response.push_not_found(id),
}
}
if found.is_empty() {
return Ok(response);
}
let with_report = properties.contains(&P::Report);
if !with_report {
record(
server,
access_token,
session,
Action::BlobAccess,
None,
None,
format!("Listed {} journal entries", found.len()),
None,
)
.await?;
}
for (entry_id, entry) in found {
let report = if with_report {
record(
server,
access_token,
session,
Action::BlobAccess,
Some(Id::new(entry_id.to_u64()).to_string()),
Some(entry.subject.clone()),
format!("Read a journaled message from {}", entry.sender),
None,
)
.await?;
report_bytes(server, &entry).await?.map(|bytes| {
let end = bytes.len().min(MAX_REPORT_TEXT);
String::from_utf8_lossy(&bytes[..end]).into_owned()
})
} else {
None
};
response.list.push(entry_value(
entry_id,
&entry,
report.as_deref(),
&properties,
));
}
Ok(response)
}
fn seconds(value: &str) -> Result<u64, String> {
UTCDate::from_str(value)
.map(|date| date.timestamp().max(0) as u64)
.map_err(|_| format!("{value} isn't a UTC date."))
}
fn direction(value: &str) -> Result<Direction, String> {
match value {
"outgoing" => Ok(Direction::Outgoing),
"incoming" => Ok(Direction::Incoming),
"internal" => Ok(Direction::Internal),
"any" => Ok(Direction::Any),
other => Err(format!("{other} isn't a direction.")),
}
}
/// The conditions of a query filter, all of which must hold. `Or` and
/// `Not` aren't supported.
fn build_filter(conditions: Vec<QueryFilter<JournalFilter>>) -> trc::Result<Filter> {
let unsupported = |why: String| trc::JmapEvent::UnsupportedFilter.into_err().details(why);
let mut filter = Filter::default();
for condition in conditions {
match condition {
QueryFilter::Property(condition) => match condition {
JournalFilter::After(date) => {
filter.after = Some(seconds(&date).map_err(unsupported)?)
}
JournalFilter::Before(date) => {
filter.before = Some(seconds(&date).map_err(unsupported)?)
}
JournalFilter::Sender(s) => filter.sender = Some(s),
JournalFilter::Recipient(r) => filter.recipient = Some(r),
JournalFilter::Address(a) => filter.address = Some(a),
JournalFilter::Direction(d) => {
filter.direction = Some(direction(&d).map_err(unsupported)?)
}
JournalFilter::Text(t) => filter.text = Some(t),
JournalFilter::MessageId(m) => filter.message_id = Some(m),
JournalFilter::JournalId(id) => filter.journal_id = Some(id.document_id()),
JournalFilter::_T(other) => {
return Err(unsupported(format!("Unknown filter property {other}.")));
}
},
QueryFilter::And | QueryFilter::Close => {}
QueryFilter::Or | QueryFilter::Not => {
return Err(unsupported(
"Journal searches take conditions that must all hold; OR and NOT aren't \
supported."
.into(),
));
}
}
}
Ok(filter)
}
fn filter_text(filter: &Filter) -> String {
serde_json::to_string(filter).unwrap_or_default()
}
/// `inbuxa:JournalEntry/query`: newest first. The search is recorded, with
/// its terms, before anything is returned.
pub async fn query(
server: &Server,
access_token: &AccessToken,
session: &HttpSessionData,
request: QueryRequest<JournalEntry>,
) -> trc::Result<QueryResponse> {
server_level(access_token)?;
let filter = build_filter(request.filter)?;
let position = request.position.unwrap_or(0);
if position < 0 || request.anchor.is_some() {
return Err(trc::JmapEvent::UnsupportedFilter
.into_err()
.details("Journal searches page by a position from the start."));
}
let limit = request
.limit
.unwrap_or(MAX_QUERY_LIMIT)
.min(MAX_QUERY_LIMIT);
let count_all = request.calculate_total.unwrap_or(false);
record(
server,
access_token,
session,
Action::BlobAccess,
None,
None,
format!("Searched the journal: {}", filter_text(&filter)),
None,
)
.await?;
let (ids, total) =
entries::query(server.store(), &filter, position as usize, limit, count_all).await?;
Ok(QueryResponse {
account_id: request.account_id,
query_state: State::Initial,
can_calculate_changes: false,
position,
ids: ids.into_iter().map(|id| Id::new(id.to_u64())).collect(),
total: count_all.then_some(total),
limit: Some(limit),
})
}
/// An export's filter, as sent: the query's conditions in one object.
fn export_filter(value: Option<Value<'_, P, JournalEntryValue>>) -> Result<Filter, String> {
let json: serde_json::Value = value
.map(Into::into)
.unwrap_or(serde_json::Value::Object(Default::default()));
let serde_json::Value::Object(map) = json else {
return Err("The filter is an object of conditions.".into());
};
let mut filter = Filter::default();
for (key, value) in map {
let text = || {
value
.as_str()
.map(str::to_string)
.ok_or_else(|| format!("{key} is text."))
};
match key.as_str() {
"after" => filter.after = Some(seconds(&text()?)?),
"before" => filter.before = Some(seconds(&text()?)?),
"sender" => filter.sender = Some(text()?),
"recipient" => filter.recipient = Some(text()?),
"address" => filter.address = Some(text()?),
"direction" => filter.direction = Some(direction(&text()?)?),
"text" => filter.text = Some(text()?),
"messageId" => filter.message_id = Some(text()?),
"journalId" => {
filter.journal_id = Some(
Id::from_str(&text()?)
.map_err(|_| "journalId is a journal's id.".to_string())?
.document_id(),
)
}
other => return Err(format!("Unknown filter property {other}.")),
}
}
Ok(filter)
}
fn csv(field: &str) -> String {
if field.contains([',', '"', '\n', '\r']) {
format!("\"{}\"", field.replace('"', "\"\""))
} else {
field.to_string()
}
}
fn hex(bytes: &[u8]) -> String {
bytes.iter().map(|b| format!("{b:02x}")).collect()
}
/// A ZIP of reports in the hold export's shape: each report as `.eml`,
/// `manifest.csv` with the envelope and a SHA-256 per file, the entries
/// whose report couldn't be read in `exceptions.csv`, and
/// `manifest.sha256` over both. Returns its bytes and how many reports went
/// in.
pub(crate) fn build_zip(
items: &[(EntryId, Entry, Option<Vec<u8>>)],
) -> trc::Result<(Vec<u8>, usize)> {
let fail = |err: zip::result::ZipError| {
trc::StoreEvent::UnexpectedError
.into_err()
.details("Failed to write the export")
.reason(err)
};
let options = SimpleFileOptions::default().compression_method(CompressionMethod::Deflated);
let mut zip = ZipWriter::new(Cursor::new(Vec::new()));
let mut manifest = String::from(
"path,receivedAt,direction,sender,recipients,subject,messageId,queueId,size,sha256\n",
);
let mut exceptions = String::from("entry,receivedAt,sender,subject,reason\n");
let mut written = 0u64;
let mut count = 0;
for (id, entry, bytes) in items {
let received = UTCDate::from_timestamp(entry.at as i64).to_string();
let Some(bytes) = bytes else {
exceptions.push_str(&format!(
"{},{},{},{},{}\n",
Id::new(id.to_u64()),
received,
csv(&entry.sender),
csv(&entry.subject),
"The report couldn't be read."
));
continue;
};
written += bytes.len() as u64;
if written > MAX_EXPORT_BYTES {
return Err(trc::StoreEvent::UnexpectedError.into_err().details(
"The reports are larger than one export can hold (1 GB). Narrow the search.",
));
}
let path = format!(
"reports/{}-{:x}.eml",
received.replace(':', ""),
entry.queue_id
);
zip.start_file(path.as_str(), options).map_err(fail)?;
zip.write_all(bytes).map_err(|e| fail(e.into()))?;
manifest.push_str(&format!(
"{},{},{},{},{},{},{},{:x},{},{}\n",
csv(&path),
received,
entry.direction.as_str(),
csv(&entry.sender),
csv(&entry.recipients.join(" ")),
csv(&entry.subject),
csv(&entry.message_id),
entry.queue_id,
bytes.len(),
hex(&Sha256::digest(bytes))
));
count += 1;
}
let manifest_hash = hex(&Sha256::digest(manifest.as_bytes()));
let exceptions_hash = hex(&Sha256::digest(exceptions.as_bytes()));
zip.start_file("manifest.csv", options).map_err(fail)?;
zip.write_all(manifest.as_bytes())
.map_err(|e| fail(e.into()))?;
zip.start_file("exceptions.csv", options).map_err(fail)?;
zip.write_all(exceptions.as_bytes())
.map_err(|e| fail(e.into()))?;
zip.start_file("manifest.sha256", options).map_err(fail)?;
zip.write_all(
format!("{manifest_hash} manifest.csv\n{exceptions_hash} exceptions.csv\n").as_bytes(),
)
.map_err(|e| fail(e.into()))?;
Ok((zip.finish().map_err(fail)?.into_inner(), count))
}
/// `inbuxa:JournalExport/set`: create `{filter, reason}`; the created
/// object names the ZIP's blob (the caller's), its size, how many reports it
/// holds and its SHA-256. A reason is required; the export is recorded
/// before it's built.
pub async fn export_set(
server: &Server,
access_token: &AccessToken,
session: &HttpSessionData,
mut request: SetRequest<'_, JournalExport>,
) -> trc::Result<SetResponse<JournalExport>> {
server_level(access_token)?;
let mut response = SetResponse::from_request(&request, server.core.jmap.set_max_objects)?;
for (id, _) in request.unwrap_update().into_valid() {
response.not_updated.append(
id,
SetError::forbidden().with_description("Exports can't be changed."),
);
}
for id in request.unwrap_destroy().into_valid() {
response.not_destroyed.append(
id,
SetError::forbidden().with_description("Exports aren't kept to destroy."),
);
}
for (client_id, value) in request.unwrap_create() {
let mut filter_value = None;
let mut reason = None;
let mut invalid = None;
for (key, value) in value.into_expanded_object() {
match (&key, value) {
(Key::Property(P::Filter), value) => filter_value = Some(value.into_owned()),
(Key::Property(P::Reason), Value::Str(r)) => {
reason = Some(r.trim().chars().take(500).collect::<String>())
}
_ => {
invalid = Some(SetError::invalid_properties().with_property(key.into_owned()));
break;
}
}
}
if let Some(error) = invalid {
response.not_created.append(client_id, error);
continue;
}
let Some(reason) = reason.filter(|r| !r.is_empty()) else {
response.not_created.append(
client_id,
SetError::invalid_properties()
.with_property(P::Reason)
.with_description(
"Say why: a reason is required and is kept in the audit log.",
),
);
continue;
};
let filter = match export_filter(filter_value) {
Ok(filter) => filter,
Err(why) => {
response.not_created.append(
client_id,
SetError::invalid_properties()
.with_property(P::Filter)
.with_description(why),
);
continue;
}
};
let (ids, total) =
entries::query(server.store(), &filter, 0, MAX_EXPORT_ENTRIES, true).await?;
if total > MAX_EXPORT_ENTRIES {
response.not_created.append(
client_id,
SetError::invalid_properties()
.with_property(P::Filter)
.with_description(format!(
"{total} entries match; one export holds {MAX_EXPORT_ENTRIES}. Narrow the search."
)),
);
continue;
}
// Recorded first: no export leaves without its record
record(
server,
access_token,
session,
Action::Export,
None,
None,
format!(
"Exported {} journal entries: {}",
ids.len(),
filter_text(&filter)
),
Some(reason),
)
.await?;
let mut items = Vec::with_capacity(ids.len());
for id in ids {
if let Some(entry) = entries::get(server.store(), id).await? {
let bytes = report_bytes(server, &entry).await?;
items.push((id, entry, bytes));
}
}
let (bytes, count) = build_zip(&items)?;
let blob = server
.put_jmap_blob(access_token.account_id(), &bytes)
.await?;
let mut created = Map::with_capacity(5);
created.insert_unchecked(
Key::Property(P::Id),
Value::Element(JournalEntryValue::Id(Id::new(store::write::now()))),
);
created.insert_unchecked(
Key::Property(P::BlobId),
Value::Str(blob.to_string().into()),
);
created.insert_unchecked(
Key::Property(P::Size),
Value::Number((bytes.len() as u64).into()),
);
created.insert_unchecked(
Key::Property(P::Count),
Value::Number((count as u64).into()),
);
created.insert_unchecked(
Key::Property(P::Sha256),
Value::Str(hex(&Sha256::digest(&bytes)).into()),
);
response.created.insert(client_id, Value::Object(created));
}
Ok(response)
}
fn summary(chains: &[ChainReport]) -> String {
if chains.is_empty() {
return "The journal is empty.".into();
}
chains
.iter()
.map(|chain| match (&chain.broken_at, &chain.reason) {
(Some(at), Some(reason)) => format!("node {}: broken at {at}: {reason}", chain.node),
_ => format!(
"node {}: {} entries and {} purged verified ({} to {})",
chain.node, chain.entries, chain.purged, chain.first_seq, chain.last_seq
),
})
.collect::<Vec<_>>()
.join("; ")
}
/// `inbuxa:JournalVerification/set`: create `{}` to recheck every node's
/// chain and every report against its entry (JR-6). Recorded, with what it
/// found.
pub async fn verification_set(
server: &Server,
access_token: &AccessToken,
session: &HttpSessionData,
mut request: SetRequest<'_, JournalVerification>,
) -> trc::Result<SetResponse<JournalVerification>> {
server_level(access_token)?;
let mut response = SetResponse::from_request(&request, server.core.jmap.set_max_objects)?;
for (id, _) in request.unwrap_update().into_valid() {
response.not_updated.append(id, SetError::forbidden());
}
for id in request.unwrap_destroy().into_valid() {
response.not_destroyed.append(id, SetError::forbidden());
}
for (client_id, _) in request.unwrap_create() {
let chains = entries::verify(server.store(), Some(server.blob_store())).await?;
let verified = chains.iter().all(|chain| chain.broken_at.is_none());
let entry = server
.audit_append(&Record {
at: store::write::now() * 1000,
actor: server.audit_actor(access_token).await,
via: access_token.origin().cloned(),
remote_ip: Some(session.remote_ip),
action: Action::Verify,
target: Target {
kind: "inbuxa:JournalEntry".into(),
..Default::default()
},
changes: vec![],
details: Some(summary(&chains)),
reason: None,
outcome: if verified {
Outcome::success()
} else {
Outcome::refused("chainBroken", None)
},
})
.await
.ok();
let mut created = Map::with_capacity(3);
created.insert_unchecked(
Key::Property(P::Id),
Value::Element(JournalEntryValue::Id(Id::new(
entry.map_or(0, |entry| entry.to_u64()),
))),
);
created.insert_unchecked(Key::Property(P::Verified), Value::Bool(verified));
created.insert_unchecked(
Key::Property(P::Chains),
json_to_value(serde_json::to_value(&chains).unwrap_or_default()),
);
response.created.insert(client_id, Value::Object(created));
}
Ok(response)
}
#[cfg(test)]
mod tests {
use super::*;
fn entry(queue_id: u64) -> Entry {
Entry {
queue_id,
at: 1_790_000_000,
direction: Direction::Outgoing,
sender: "[email protected]".into(),
authenticated: true,
recipients: vec!["[email protected]".into()],
subject: "Q3, final".into(),
message_id: "<[email protected]>".into(),
accounts: vec![],
tenants: vec![],
journals: vec![1],
held: false,
blob: String::new(),
size: 0,
sha256: String::new(),
expires_at: 0,
}
}
#[test]
fn exports_list_every_report_and_what_was_missing() {
let items = vec![
(
EntryId { node: 1, seq: 1 },
entry(0x1a),
Some(b"report one".to_vec()),
),
(EntryId { node: 1, seq: 2 }, entry(0x1b), None),
];
let (bytes, count) = build_zip(&items).unwrap();
assert_eq!(count, 1);
let mut zip = zip::ZipArchive::new(Cursor::new(bytes)).unwrap();
let mut read = |name: &str| {
let mut out = String::new();
std::io::Read::read_to_string(&mut zip.by_name(name).unwrap(), &mut out).unwrap();
out
};
let manifest = read("manifest.csv");
assert!(manifest.contains("\"Q3, final\""), "{manifest}");
assert!(manifest.contains(&hex(&Sha256::digest(b"report one"))));
assert!(read("exceptions.csv").contains("couldn't be read"));
let sums = read("manifest.sha256");
assert!(sums.contains(&hex(&Sha256::digest(manifest.as_bytes()))));
}
#[test]
fn export_filters_parse() {
let filter: Value<'_, P, JournalEntryValue> = json_to_value(serde_json::json!({
"sender": "alice", "direction": "outgoing", "journalId": "b",
"after": "2026-09-01T00:00:00Z"
}));
let filter = export_filter(Some(filter)).unwrap();
assert_eq!(filter.sender.as_deref(), Some("alice"));
assert_eq!(filter.direction, Some(Direction::Outgoing));
assert_eq!(filter.journal_id, Some(1));
assert!(filter.after.is_some());
let bad: Value<'_, P, JournalEntryValue> =
json_to_value(serde_json::json!({"colour": "red"}));
assert!(export_filter(Some(bad)).is_err());
}
}
+3
View File
@@ -11,6 +11,9 @@ pub mod access;
pub mod account_lock;
pub mod legal_hold;
pub mod mail_rule;
pub mod security_acceptance;
pub mod journal;
pub mod journal_entry;
pub mod held_message;
pub mod dlp_settings;
pub mod hold_export;
@@ -0,0 +1,257 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! `inbuxa:SecurityAcceptance` (security to-do list spec, SS-23 to SS-26):
//! the security to-do items an administrator accepted, with why, so every
//! administrator sees the same accepted risks. Created and destroyed, never
//! updated (the request gate refuses an update). Seeing them needs what the
//! security page needs; changing them needs `sysSecurityAccept`. Every
//! check is server-wide, so nobody in a tenant reaches them. The request
//! layer records every change in the audit log (SS-26).
use common::{Server, auth::AccessToken};
use inbuxa_features::security::acceptance::{self, Acceptance, Created};
use jmap_proto::{
error::set::SetError,
method::{
get::{GetRequest, GetResponse},
set::{SetRequest, SetResponse},
},
object::inbuxa_security_acceptance::{
SecurityAcceptance, SecurityAcceptanceProperty as P, SecurityAcceptanceValue,
},
request::IntoValid,
types::date::UTCDate,
};
use jmap_tools::{Key, Map, Property, Value};
use std::borrow::Cow;
use store::write::now;
use types::id::Id;
type RValue = Value<'static, P, SecurityAcceptanceValue>;
const ALL: &[P] = &[
P::Id,
P::Check,
P::Subject,
P::AcceptedValue,
P::Note,
P::AcceptedBy,
P::AcceptedAt,
];
/// Properties the server sets; a client that sends them is refused.
const SERVER_SET: &[P] = &[P::Id, P::AcceptedBy, P::AcceptedAt];
fn server_level(access_token: &AccessToken) -> trc::Result<()> {
if access_token.tenant_id().is_some() {
Err(trc::JmapEvent::Forbidden
.into_err()
.details("Security checks are the server's."))
} else {
Ok(())
}
}
fn json_to_value(json: serde_json::Value) -> RValue {
match json {
serde_json::Value::Null => Value::Null,
serde_json::Value::Bool(b) => Value::Bool(b),
serde_json::Value::Number(n) => {
if let Some(n) = n.as_u64() {
Value::Number(n.into())
} else if let Some(n) = n.as_i64() {
Value::Number(n.into())
} else {
Value::Number(n.as_f64().unwrap_or_default().into())
}
}
serde_json::Value::String(s) => Value::Str(Cow::Owned(s)),
serde_json::Value::Array(items) => {
Value::Array(items.into_iter().map(json_to_value).collect())
}
serde_json::Value::Object(map) => {
let mut out = Map::with_capacity(map.len());
for (key, value) in map {
out.insert_unchecked(Key::Owned(key), json_to_value(value));
}
Value::Object(out)
}
}
}
fn to_value(acceptance: &Acceptance, properties: &[P]) -> RValue {
let mut out = Map::with_capacity(properties.len());
for property in properties {
let value = match property {
P::Id => Value::Element(SecurityAcceptanceValue::Id(Id::from(acceptance.id))),
P::Check => Value::Str(acceptance.check.clone().into()),
P::Subject => Value::Str(acceptance.subject.clone().into()),
P::AcceptedValue => json_to_value(acceptance.accepted_value.clone()),
P::Note => Value::Str(acceptance.note.clone().into()),
P::AcceptedBy => Value::Str(acceptance.accepted_by.clone().into()),
P::AcceptedAt => Value::Str(
UTCDate::from_timestamp(acceptance.accepted_at as i64)
.to_string()
.into(),
),
};
out.insert_unchecked(Key::Property(property.clone()), value);
}
Value::Object(out)
}
/// An acceptance as sent, checked whole.
fn parse(value: Value<'_, P, SecurityAcceptanceValue>) -> Result<Acceptance, SetError<P>> {
let mut map = serde_json::Map::new();
for (key, value) in value.into_expanded_object() {
match &key {
Key::Property(p) if SERVER_SET.contains(p) => {
return Err(SetError::invalid_properties()
.with_property(p.clone())
.with_description("The server sets this."));
}
Key::Property(p) => {
map.insert(p.to_cow().into_owned(), value.into());
}
_ => {
return Err(SetError::invalid_properties().with_property(key.clone().into_owned()));
}
}
}
let acceptance: Acceptance =
serde_json::from_value(serde_json::Value::Object(map)).map_err(|err| {
SetError::invalid_properties()
.with_description(format!("Not a valid acceptance: {err}"))
})?;
acceptance.validate().map_err(|invalid| {
let property = invalid.property.parse::<P>().unwrap_or(P::Note);
SetError::invalid_properties()
.with_property(property)
.with_description(invalid.reason)
})?;
Ok(Acceptance {
note: acceptance.note.trim().to_string(),
..acceptance
})
}
fn acceptance_id(id: Id) -> Option<u32> {
u32::try_from(id.id()).ok()
}
/// `inbuxa:SecurityAcceptance/get`: every acceptance, oldest first.
pub async fn get(
server: &Server,
access_token: &AccessToken,
mut request: GetRequest<SecurityAcceptance>,
) -> trc::Result<GetResponse<SecurityAcceptance>> {
server_level(access_token)?;
let properties = request.unwrap_properties(ALL);
let (ids, not_found) = request.unwrap_ids(server.core.jmap.get_max_objects)?;
let mut response = GetResponse {
account_id: request.account_id.into(),
state: None,
list: Vec::new(),
not_found,
};
let all = acceptance::all(server.store()).await?;
match ids {
None => {
response.list = all.iter().map(|a| to_value(a, &properties)).collect();
}
Some(ids) => {
for id in ids {
match acceptance_id(id).and_then(|id| all.iter().find(|a| a.id == id)) {
Some(a) => response.list.push(to_value(a, &properties)),
None => response.push_not_found(id),
}
}
}
}
Ok(response)
}
/// `inbuxa:SecurityAcceptance/set`: accept an item, or remove an acceptance.
pub async fn set(
server: &Server,
access_token: &AccessToken,
mut request: SetRequest<'_, SecurityAcceptance>,
) -> trc::Result<SetResponse<SecurityAcceptance>> {
server_level(access_token)?;
let mut response = SetResponse::from_request(&request, server.core.jmap.set_max_objects)?;
let data = server.store();
let actor = server.audit_actor(access_token).await;
for (client_id, value) in request.unwrap_create() {
let parsed = match parse(value) {
Ok(parsed) => parsed,
Err(error) => {
response.not_created.append(client_id, error);
continue;
}
};
let accepted = Acceptance {
accepted_by: actor.name.clone(),
accepted_at: now(),
..parsed
};
match acceptance::create(data, &accepted).await? {
Created::Id(id) => {
let mut out = Map::with_capacity(3);
out.insert_unchecked(
Key::Property(P::Id),
Value::Element(SecurityAcceptanceValue::Id(Id::from(id))),
);
out.insert_unchecked(
Key::Property(P::AcceptedBy),
Value::Str(accepted.accepted_by.clone().into()),
);
out.insert_unchecked(
Key::Property(P::AcceptedAt),
Value::Str(
UTCDate::from_timestamp(accepted.accepted_at as i64)
.to_string()
.into(),
),
);
response.created.insert(client_id, Value::Object(out));
}
Created::Full => {
response.not_created.append(
client_id,
SetError::over_quota().with_description(format!(
"There are already {} acceptances. Remove some first.",
acceptance::MAX_ACCEPTANCES
)),
);
}
}
}
for (id, _) in request.unwrap_update().into_valid() {
response.not_updated.append(
id,
SetError::forbidden().with_description("An acceptance is replaced, not edited."),
);
}
for id in request.unwrap_destroy().into_valid() {
let found = match acceptance_id(id) {
Some(acceptance_id) => acceptance::get(data, acceptance_id).await?,
None => None,
};
match found {
Some(found) => {
acceptance::delete(data, found.id).await?;
response.destroyed.push(id);
}
None => response.not_destroyed.append(id, SetError::not_found()),
}
}
Ok(response)
}
+9
View File
@@ -146,6 +146,15 @@ pub(crate) async fn log_query(
})?;
response.anchor_found = true;
// inbuxa: the total is only known when the first page reached the end
// of the logs; counting them all would mean reading every file on every
// page. Upstream answered the query cap (5000) as the total, so a
// two-line log read "of 5000".
response.response.total = (req.request.calculate_total.unwrap_or(false)
&& anchor == 0
&& response.response.ids.len() < limit)
.then_some(response.response.ids.len());
Ok(response)
}
+7
View File
@@ -1755,6 +1755,13 @@ pub enum Permission {
SysDlpPolicyUpdate = 677,
SysDlpReviewGet = 678,
SysDlpReviewUpdate = 679,
// inbuxa: journaling
SysJournalGet = 680,
SysJournalUpdate = 681,
SysJournalSearch = 682,
SysJournalExport = 683,
// inbuxa: the security to-do list, accepting an item
SysSecurityAccept = 684,
SysAccountGet = 219,
SysAccountCreate = 220,
SysAccountUpdate = 221,
+16 -1
View File
@@ -7097,6 +7097,11 @@ impl EnumImpl for Permission {
b"sysDlpPolicyUpdate" => Permission::SysDlpPolicyUpdate,
b"sysDlpReviewGet" => Permission::SysDlpReviewGet,
b"sysDlpReviewUpdate" => Permission::SysDlpReviewUpdate,
b"sysJournalGet" => Permission::SysJournalGet,
b"sysJournalUpdate" => Permission::SysJournalUpdate,
b"sysJournalSearch" => Permission::SysJournalSearch,
b"sysJournalExport" => Permission::SysJournalExport,
b"sysSecurityAccept" => Permission::SysSecurityAccept,
b"sysAccountGet" => Permission::SysAccountGet,
b"sysAccountCreate" => Permission::SysAccountCreate,
b"sysAccountUpdate" => Permission::SysAccountUpdate,
@@ -7793,6 +7798,11 @@ impl EnumImpl for Permission {
Permission::SysDlpPolicyUpdate => "sysDlpPolicyUpdate",
Permission::SysDlpReviewGet => "sysDlpReviewGet",
Permission::SysDlpReviewUpdate => "sysDlpReviewUpdate",
Permission::SysJournalGet => "sysJournalGet",
Permission::SysJournalUpdate => "sysJournalUpdate",
Permission::SysJournalSearch => "sysJournalSearch",
Permission::SysJournalExport => "sysJournalExport",
Permission::SysSecurityAccept => "sysSecurityAccept",
Permission::SysAccountGet => "sysAccountGet",
Permission::SysAccountCreate => "sysAccountCreate",
Permission::SysAccountUpdate => "sysAccountUpdate",
@@ -8482,6 +8492,11 @@ impl EnumImpl for Permission {
677 => Some(Permission::SysDlpPolicyUpdate),
678 => Some(Permission::SysDlpReviewGet),
679 => Some(Permission::SysDlpReviewUpdate),
680 => Some(Permission::SysJournalGet),
681 => Some(Permission::SysJournalUpdate),
682 => Some(Permission::SysJournalSearch),
683 => Some(Permission::SysJournalExport),
684 => Some(Permission::SysSecurityAccept),
219 => Some(Permission::SysAccountGet),
220 => Some(Permission::SysAccountCreate),
221 => Some(Permission::SysAccountUpdate),
@@ -8926,7 +8941,7 @@ impl EnumImpl for Permission {
}
}
const COUNT: usize = 680;
const COUNT: usize = 685;
}
impl serde::Serialize for Permission {
@@ -291,6 +291,12 @@ async fn store_maintenance(
trc::error!(err.details("Failed to return unreviewed held mail"));
}
// inbuxa: journaling, JR-13: entries past their retention go,
// except those a legal hold keeps
if let Err(err) = purge_journal(server).await {
trc::error!(err.details("Failed to purge journal entries"));
}
// inbuxa: AU-7: audit records past their retention go; a
// failure leaves them for the next run
if let Err(err) = server.audit_purge().await {
@@ -409,6 +415,54 @@ async fn store_maintenance(
Ok(TaskResult::Success(vec![]))
}
/// inbuxa: journaling, JR-13: removes journal entries past their
/// retention, keeping any whose sender or recipients a legal hold covers
/// (deleted accounts a hold keeps included), and records how many went.
async fn purge_journal(server: &Server) -> trc::Result<()> {
use inbuxa_features::audit::{Action, Actor, Outcome, Record, Target};
let mut held = server.held_accounts().await?;
if !held.is_empty() {
for (account_id, kept) in
inbuxa_features::undelete::data::kept_accounts(server.store()).await?
{
if server.is_kept_held(account_id, &kept).await? {
held.insert(account_id);
}
}
}
let at = store::write::now();
let purged = inbuxa_features::journal::entries::purge(server.store(), at, |entry| {
entry.accounts.iter().any(|account| held.contains(account))
})
.await?;
if purged.removed > 0 || purged.kept_for_hold > 0 {
server
.audit_note(Record {
at: at * 1000,
actor: Actor::system("Journal"),
via: None,
remote_ip: None,
action: Action::Destroy,
target: Target {
kind: "inbuxa:JournalEntry".into(),
id: None,
name: None,
account_id: None,
tenant_id: None,
},
changes: vec![],
details: Some(format!(
"{} past their retention removed; {} kept for a legal hold",
purged.removed, purged.kept_for_hold
)),
reason: None,
outcome: Outcome::success(),
})
.await;
}
Ok(())
}
async fn account_maintenance(
server: &Server,
task: &TaskAccountMaintenance,
+8
View File
@@ -102,6 +102,10 @@ pub struct SessionData {
pub dlp_refusal: Option<DlpRefusal>,
// inbuxa: a mail flow rule's route for this message
pub mailflow_queue: Option<String>,
// inbuxa: journaling (JR-3, JR-10): journals rules sent this message
// to, and recipients rules added, by rule name
pub journal_marks: Vec<u32>,
pub journal_added: Vec<(String, String)>,
}
/// inbuxa: a DATA refusal by DLP rules: blocked, or a warning the sender
@@ -189,6 +193,8 @@ impl SessionData {
dlp_override: None,
dlp_refusal: None,
mailflow_queue: None,
journal_marks: Vec::new(),
journal_added: Vec::new(),
}
}
}
@@ -315,6 +321,8 @@ impl SessionData {
dlp_override: None,
dlp_refusal: None,
mailflow_queue: None,
journal_marks: Vec::new(),
journal_added: Vec::new(),
}
}
}
+15 -3
View File
@@ -763,19 +763,27 @@ impl<T: SessionStream> Session<T> {
}
for change in envelope {
match change {
super::mailflow::EnvelopeChange::AddRecipient(address) => {
super::mailflow::EnvelopeChange::AddRecipient(address, rule) => {
if !self
.data
.rcpt_to
.iter()
.any(|r| r.address_lcase.eq_ignore_ascii_case(&address))
{
self.data.journal_added.push((address.to_lowercase(), rule));
self.data.rcpt_to.push(SessionAddress::new(address));
}
}
super::mailflow::EnvelopeChange::Redirect(addresses) => {
super::mailflow::EnvelopeChange::Redirect(addresses, rule) => {
self.data.journal_added = addresses
.iter()
.map(|a| (a.to_lowercase(), rule.clone()))
.collect();
self.data.rcpt_to = addresses.into_iter().map(SessionAddress::new).collect();
}
super::mailflow::EnvelopeChange::Journal(journal) => {
self.data.journal_marks.push(journal);
}
super::mailflow::EnvelopeChange::Route(queue) => {
self.data.mailflow_queue = Some(queue);
}
@@ -882,7 +890,11 @@ impl<T: SessionStream> Session<T> {
.with_dkim_signers(dkim_signers)
.with_original_raw_message(original_message)
.with_original_authenticated_message(auth_message)
.with_metadata(metadata),
.with_metadata(metadata)
.with_journal(
std::mem::take(&mut self.data.journal_marks),
std::mem::take(&mut self.data.journal_added),
),
)
.await
{
+28 -5
View File
@@ -63,9 +63,12 @@ pub struct HeldDraft {
/// What a transport rule changes about where a message goes.
pub enum EnvelopeChange {
AddRecipient(String),
Redirect(Vec<String>),
/// An address, and the rule that added it.
AddRecipient(String, String),
Redirect(Vec<String>, String),
Route(String),
/// Journaling spec, JR-10: a journal the message goes to.
Journal(u32),
}
/// `[override: reason]` at the start of a subject: the reason, and the
@@ -402,11 +405,17 @@ impl<T: SessionStream> Session<T> {
rewrite::prefix_subject(now, text)
}
RuleAction::AddRecipient { address } => {
changes.push(EnvelopeChange::AddRecipient(address.clone()));
changes.push(EnvelopeChange::AddRecipient(
address.clone(),
matched.name.clone(),
));
None
}
RuleAction::Redirect { addresses } => {
changes.push(EnvelopeChange::Redirect(addresses.clone()));
changes.push(EnvelopeChange::Redirect(
addresses.clone(),
matched.name.clone(),
));
None
}
RuleAction::Route { queue } => {
@@ -423,7 +432,8 @@ impl<T: SessionStream> Session<T> {
}
RuleAction::Block { .. }
| RuleAction::Warn { .. }
| RuleAction::Hold { .. } => None,
| RuleAction::Hold { .. }
| RuleAction::Journal { .. } => None,
};
if next.is_some() {
current = next;
@@ -450,6 +460,19 @@ impl<T: SessionStream> Session<T> {
.await;
}
}
// JR-10: journals any matched rule sends the message to,
// DLP rules included
for matched in &outcome.matched {
for action in &matched.actions {
if let RuleAction::Journal { journal } = action
&& !changes
.iter()
.any(|c| matches!(c, EnvelopeChange::Journal(j) if j == journal))
{
changes.push(EnvelopeChange::Journal(*journal));
}
}
}
match hold {
Some(draft) => Checked::Hold {
draft,
+4
View File
@@ -494,6 +494,10 @@ impl<T: AsyncWrite + AsyncRead + Unpin> Session<T> {
self.data.delivery_by = 0;
self.data.future_release = 0;
self.data.rcpt_oks = 0;
// inbuxa: what mail flow rules decided was for the last message only
self.data.mailflow_queue = None;
self.data.journal_marks.clear();
self.data.journal_added.clear();
}
pub fn reset_tls(&mut self) {
+280
View File
@@ -0,0 +1,280 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! inbuxa: journaling (journaling spec, JR-1 to JR-11): the copy taken as a
//! message is queued, after DLP and transport rules, so it has the envelope
//! the message actually leaves or arrives with; reports to outside archives,
//! and what happens when an archive doesn't take one.
use crate::queue::{
FROM_AUTHENTICATED, FROM_AUTOGENERATED, FROM_DSN, FROM_REPORT, Message, MessageSource, Status,
spool::{QueueParams, SmtpSpool},
};
use common::Server;
use inbuxa_features::{
audit::{Action, Actor, Outcome, Record, Target},
hold::Member,
journal::{
self, Direction,
archive::{self, Pending},
entries::{self, Entry},
report::{self, Envelope, Recipient},
},
mailflow::held::HOLD_SECONDS,
};
use store::write::{BatchBuilder, BlobLink, BlobOp, now};
use types::blob_hash::BlobHash;
/// What mail flow rules decided about a message at DATA (JR-3, JR-10).
#[derive(Debug, Clone, Default)]
pub struct Hints {
/// Journals a rule sent it to.
pub marks: Vec<u32>,
/// Recipients a rule added (lowercase), and the rule's name.
pub added: Vec<(String, String)>,
}
/// Marks a journal report the server queued itself, so it's never
/// journaled (JR-2). Free in the message flags (the MAIL parameters use
/// the low bits, the sources bits 32 to 37).
pub const FROM_JOURNAL: u64 = 1 << 48;
/// Journals `message`, whose queued bytes are `raw`, into every enabled
/// journal that takes it. An error means it may not have been journaled,
/// and the caller must not queue it.
pub async fn capture(
server: &Server,
queue_id: u64,
message: &Message,
raw: &[u8],
hints: &Hints,
) -> trc::Result<()> {
if message.flags & (FROM_JOURNAL | FROM_REPORT) != 0 {
return Ok(());
}
let journals = journal::enabled(server.store()).await?;
if journals.is_empty() {
return Ok(());
}
// Who's here on either side, and which way it goes
let mut members: Vec<Member> = Vec::new();
let mut sender_local = message.flags & FROM_AUTHENTICATED != 0
|| (message.return_path.is_empty() && message.flags & (FROM_DSN | FROM_AUTOGENERATED) != 0);
if !message.return_path.is_empty()
&& let Some(id) = server
.account_id_from_email(&message.return_path, false)
.await?
{
sender_local = true;
if let Some(member) = server.member_of(id).await {
members.push(member);
}
}
let (mut any_local, mut any_remote) = (false, false);
for rcpt in &message.recipients {
let address = rcpt.address.to_lowercase();
let domain = address.rsplit_once('@').map_or("", |(_, d)| d);
let local_domain = server.domain(domain).await.ok().flatten().is_some();
match server.account_id_from_email(&address, false).await? {
Some(id) => {
any_local = true;
if !members.iter().any(|m| m.account == id)
&& let Some(member) = server.member_of(id).await
{
members.push(member);
}
}
None if local_domain => any_local = true,
None => any_remote = true,
}
}
let direction = Direction::of(sender_local, any_remote, any_local);
// A journal takes it through its scope, or because a rule sent it there
let taken: Vec<&journal::Journal> = journals
.iter()
.filter(|j| j.takes(direction, &members) || hints.marks.contains(&j.id))
.collect();
if taken.is_empty() {
return Ok(());
}
// DLP holds a message by putting its release a century off
let at = now();
let held = !message.recipients.is_empty()
&& message
.recipients
.iter()
.all(|rcpt| rcpt.retry.due >= at + HOLD_SECONDS / 2);
let recipients: Vec<Recipient> = message
.recipients
.iter()
.map(|rcpt| Recipient {
address: rcpt.address.to_string(),
orcpt: rcpt.orcpt.as_deref().map(Into::into),
added_by: hints
.added
.iter()
.find(|(address, _)| address.eq_ignore_ascii_case(&rcpt.address))
.map(|(_, rule)| rule.clone()),
})
.collect();
let envelope = Envelope {
sender: &message.return_path,
authenticated: message.flags & FROM_AUTHENTICATED != 0,
recipients: &recipients,
queue_id,
received: message.created,
direction,
held,
};
let host = server.core.network.server_name.as_str();
let (bytes, fields) = report::build(&envelope, raw, &format!("postmaster@{host}"), host);
let mut tenants: Vec<u32> = members.iter().filter_map(|m| m.tenant).collect();
tenants.sort_unstable();
tenants.dedup();
let hash = BlobHash::generate(&bytes);
let entry_for = |journals: &[&journal::Journal]| {
let retention_days = journals
.iter()
.map(|j| j.retention_days)
.max()
.unwrap_or_default();
Entry {
queue_id,
at,
direction,
sender: message.return_path.to_string(),
authenticated: envelope.authenticated,
recipients: recipients.iter().map(|r| r.address.clone()).collect(),
subject: fields.subject.clone(),
message_id: fields.message_id.clone(),
accounts: members.iter().map(|m| m.account).collect(),
tenants: tenants.clone(),
journals: journals.iter().map(|j| j.id).collect(),
held,
blob: entries::hex(hash.as_slice()),
size: bytes.len() as u64,
sha256: entries::sha256(&bytes),
expires_at: at + u64::from(retention_days) * 86_400,
}
};
// The built-in journal: one entry, however many journals keep it there
let built_in: Vec<&journal::Journal> = taken.iter().copied().filter(|j| j.built_in).collect();
if !built_in.is_empty() {
// The report's blob, reserved until the entry links it
let mut batch = BatchBuilder::new();
batch.set(
BlobOp::Link {
hash: hash.clone(),
to: BlobLink::Temporary { until: at + 120 },
},
vec![],
);
server.store().write(batch.build_all()).await?;
server
.blob_store()
.put_blob(hash.as_slice(), &bytes, server.core.email.compression)
.await?;
entries::append(
server.store(),
server.core.network.node_id,
&entry_for(&built_in),
)
.await?;
}
// Outside archives: one report per address (JR-4, JR-7)
let mut addresses: Vec<(String, Vec<&journal::Journal>)> = Vec::new();
for journal in &taken {
if let Some(address) = &journal.archive_address {
let address = address.to_lowercase();
match addresses.iter_mut().find(|(a, _)| *a == address) {
Some((_, journals)) => journals.push(journal),
None => addresses.push((address, vec![journal])),
}
}
}
for (address, journals) in addresses {
// From nobody: an archive's refusal comes back to no one, and the
// queue's own record of it is what counts (settle, below)
let mut report = server.new_message("", MessageSource::Autogenerated, 0);
report.message.flags |= FROM_JOURNAL;
report.add_expanded_recipient(&address, server).await;
let pending = Pending {
address,
entry: entry_for(&journals),
};
archive::set_pending(server.store(), report.queue_id, &pending).await?;
let report_id = report.queue_id;
// Boxed: queueing the report comes back through this function
let queued = Box::pin(report.queue(QueueParams::new(&bytes, 0, server))).await;
if !queued {
archive::clear_pending(server.store(), report_id).await?;
return Err(trc::StoreEvent::UnexpectedError
.into_err()
.details("Failed to queue a journal report"));
}
}
Ok(())
}
/// JR-7: a journal report is leaving the queue. Delivered, its pending
/// record goes; not delivered (refused, expired, or deleted from the
/// queue), it goes into the built-in journal instead, and the journals
/// that sent it count a failure. An error means nothing was settled, and
/// the report must stay queued.
pub async fn settle(server: &Server, queue_id: u64, message: &Message) -> trc::Result<()> {
let store = server.store();
let Some(pending) = archive::pending(store, queue_id).await? else {
return Ok(());
};
let delivered = !message.recipients.is_empty()
&& message
.recipients
.iter()
.all(|rcpt| matches!(rcpt.status, Status::Completed(_)));
if !delivered {
let reason = if message
.recipients
.iter()
.any(|rcpt| matches!(rcpt.status, Status::PermanentFailure(_)))
{
"the archive refused it"
} else {
"it wasn't delivered before leaving the queue"
};
entries::append(store, server.core.network.node_id, &pending.entry).await?;
let at = now();
archive::record_failure(store, &pending.entry.journals, at, reason).await?;
server
.audit_note(Record {
at: at * 1000,
actor: Actor::system("Journal"),
via: None,
remote_ip: None,
action: Action::Create,
target: Target {
kind: "inbuxa:JournalEntry".into(),
id: Some(format!("{:x}", pending.entry.queue_id)),
name: None,
account_id: None,
tenant_id: None,
},
changes: vec![],
details: Some(format!(
"A journal report to {} wasn't delivered ({reason}); kept in the built-in journal",
pending.address
)),
reason: None,
outcome: Outcome::success(),
})
.await;
}
archive::clear_pending(store, queue_id).await
}
+1
View File
@@ -24,6 +24,7 @@ use utils::DomainPart;
pub mod dsn;
pub mod held; // inbuxa: mail held for review
pub mod journal; // inbuxa: journaling
pub mod manager;
pub mod quota;
pub mod spool;
+60
View File
@@ -2,6 +2,8 @@
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <hello@stalw.art>
*
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
*
* Modified by Coffey Labs in 2026 for INBUXA.
*/
use super::{
@@ -368,6 +370,8 @@ pub(crate) struct QueueParams<'x, 'y> {
pub session_id: u64,
pub server: &'y Server,
pub train_spam: Option<(bool, String)>,
// inbuxa: journaling, JR-3, JR-10
pub journal: crate::queue::journal::Hints,
}
impl MessageWrapper {
@@ -388,6 +392,7 @@ impl MessageWrapper {
server,
train_spam,
metadata,
journal,
..
} = params;
let event = self.message.queued_event();
@@ -453,6 +458,26 @@ impl MessageWrapper {
return false;
}
// inbuxa: journaling, JR-1: the copy is taken before the message is
// queued; if it can't be, the message isn't queued either
if let Err(err) = crate::queue::journal::capture(
server,
self.queue_id,
&self.message,
message.as_ref(),
&journal,
)
.await
{
trc::error!(
err.details("Failed to journal a message.")
.span_id(session_id)
.caused_by(trc::location!())
);
return false;
}
trc::event!(
Queue(event),
SpanId = session_id,
@@ -792,6 +817,20 @@ impl MessageWrapper {
}
pub async fn remove(self, server: &Server, prev_event: Option<u64>) -> bool {
// inbuxa: journaling, JR-7: a journal report the archive never took
// goes into the built-in journal before it leaves the queue
if self.message.flags & crate::queue::journal::FROM_JOURNAL != 0
&& let Err(err) =
crate::queue::journal::settle(server, self.queue_id, &self.message).await
{
trc::error!(
err.details("Failed to settle a journal report; it stays queued.")
.span_id(self.span_id)
.caused_by(trc::location!())
);
return false;
}
let mut batch = BatchBuilder::new();
if let Some(prev_event) = prev_event {
@@ -966,6 +1005,19 @@ impl MessageWrapper {
server: &Server,
prev_events: AHashMap<QueueName, u64>,
) -> bool {
// inbuxa: journaling, JR-7, as in `remove`
if self.message.flags & crate::queue::journal::FROM_JOURNAL != 0
&& let Err(err) =
crate::queue::journal::settle(server, self.queue_id, &self.message).await
{
trc::error!(
err.details("Failed to settle a journal report; it stays queued.")
.span_id(self.span_id)
.caused_by(trc::location!())
);
return false;
}
let mut batch = BatchBuilder::new();
for (queue_name, due) in prev_events {
@@ -1121,9 +1173,17 @@ impl<'x, 'y> QueueParams<'x, 'y> {
original_raw_message: None,
original_authenticated_message: None,
metadata: Vec::new(),
journal: Default::default(),
}
}
/// inbuxa: journals mail flow rules sent the message to, and the
/// recipients they added, by rule name.
pub fn with_journal(mut self, marks: Vec<u32>, added: Vec<(String, String)>) -> Self {
self.journal = crate::queue::journal::Hints { marks, added };
self
}
pub fn with_train_spam(mut self, train_spam: Option<(bool, String)>) -> Self {
self.train_spam = train_spam;
self
+1 -1
View File
@@ -81,7 +81,7 @@ fn legacy_setting(name: &str, is_set: impl Fn(&str) -> bool) -> Option<String> {
#[macro_export]
macro_rules! brand_version {
() => {
"2026.9.28.5"
"2026.9.29.1"
};
}
+2
View File
@@ -362,6 +362,8 @@ is written.
| 9 | Per-domain directories | A domain signs in against its own LDAP, SQL or OIDC directory | Added 2026-09-18. Signing in through an OIDC provider as the server's directory is already AGPL; only the per-domain choice is Enterprise. Built 2026-09-19 in `crates/common/src/auth` and `crates/directory`; status in `features/per-domain-directories.md`. |
| — | Seat limits, license keys | Nothing: there's no license | Removed, not rebuilt. |
Not a rebuild: the **security to-do list** is INBUXA's own design (inbuxa-drafts `specs/security-score.md`). The console runs its checks; the server's part is `inbuxa:SecurityAcceptance`, the accepted items (`crates/jmap/src/inbuxa/security_acceptance.rs`), and the `sysSecurityAccept` permission.
## 5. The web front ends
**Which ihasmail.** Public ihasmail stays Stalwart-facing: its code, docs,
+378
View File
@@ -0,0 +1,378 @@
# Feature spec: journaling
Status: **approved 2026-09-28**, with the answers under [Settled](#settled);
**built 2026-09-29** (phases 2–5, see [As built](#as-built)), not yet released.
Not a rebuild of an upstream feature, so it has no line in SPEC.md §4's table.
Rule IDs: **JR-**.
## Provenance
Written for the record SPEC.md §3 rule 3 asks for. Sources, and nothing else:
| Source | License | Used for |
|---|---|---|
| This repository at `94a3a76` (2026-09-28): `crates/smtp/src/inbound/data.rs`, `inbound/rcpt.rs`, `queue/spool.rs`, `outbound/delivery.rs`, `crates/common/src/network/mta.rs`, `crates/features/src/{hold,audit,mailflow,undelete}`, `crates/store/src/write/{mod,blob}.rs`, `crates/jmap/src/inbuxa/hold_export.rs` | AGPL-3.0-only | Where every message passes, what the envelope holds, how holds keep blobs, how the audit chain and hold export work |
| `inbuxa-drafts/queue/journaling.md` | Own | What John asked for, and the gaps to settle |
| The DLP and mail flow rules spec, the audit-hold-lock spec, the personal-data catalog spec | Own | Conditions, the audit log, legal holds, roles, the catalog check |
| RFC 5321, RFC 3461 (DSN, ORCPT), RFC 2046 (`message/rfc822`), RFC 5322 | Public | The envelope, the original recipient of an expanded list, the report's shape |
No Enterprise-only file or snippet was used, and no third-party journaling
product or report format was consulted: the journal report below is our own
layout of the SMTP envelope around the untouched message.
## What it is
A **journal** is a copy of each message the server handles, captured in
transit with its **envelope** (the real sender and every recipient, including
Bcc and the members of lists), kept where nobody can change or remove it
until its retention ends, or sent to an outside archive. It sits beside two
things that exist:
- **Legal hold** keeps what's in chosen mailboxes, including what their owners
delete. It starts when a hold is placed and can't see Bcc or what was sent
from a mailbox that no longer exists.
- **The audit log** records what people and the server did, never the mail.
A journal answers the question neither can: *what went through, to whom,
from the day it was turned on*.
**Out of scope**: journaling mail stored before it's turned on, files,
calendar and contacts, IMAP APPEND (a mail app saving to its own Sent folder
sends nothing), and mail a mail app sends through another server.
Nothing in code, docs, UI text or output claims the product meets a legal or
regulatory standard. The pages say what's captured, where it's kept and for
how long.
## 1. What exists today
Checked by reading the code at `94a3a76`:
| Need | Today |
|---|---|
| One place all mail passes | `MessageWrapper::queue` (`queue/spool.rs` ~L375). SMTP, JMAP submission (`jmap/src/submission/set.rs` builds a local session and runs `queue_message`), inbound mail, Sieve redirects and vacation replies, and DSNs all queue through it. Local and remote delivery both start from the queue. |
| The envelope | At queue time: `mail_from`, every `rcpt_to` (Bcc included), the authenticated account with its groups and tenant, the queue id. Lists are **already expanded** at RCPT (`rcpt_resolve` → `RcptResolution::Expand`, `inbound/rcpt.rs`); the list address survives as each member's ORCPT (`dsn_info`). |
| A copy out | Sieve at DATA, milters and MTA hooks can send one, but all run **before** DLP and transport rules, so they miss recipients the rules add, and a Sieve copy carries no envelope. |
| Keeping a blob nobody can delete | No "undeletable" flag. Blobs are content-addressed (can't be edited); a `BlobLink::Temporary { until }` keeps one until `until`. Legal hold uses `until` = year 9999. |
| A record nobody can quietly change | The audit log's per-node SHA-256 chain (`features/src/audit/log.rs`): each entry carries `prev`, the head is asserted on append, purge leaves a floor hash, `verify` walks it. |
| Export | Hold export (LH-12): a ZIP of `.eml` files, `manifest.csv` with a SHA-256 per file, `manifest.sha256`, capped at 2 GiB. |
| Conditions by sender, recipient, group, tenant | The mail flow engine (`features/src/mailflow/engine.rs`), at DATA. |
## 2. Design
### 2.1 Where the copy is taken (JR-1, JR-2)
**JR-1.** The journal is taken in `MessageWrapper::queue`, after the message
is spooled, behind one `// inbuxa:` marked block. That's after DLP and
transport rules, so the envelope is the one the message actually leaves or
arrives with, and it covers every path that queues mail.
**JR-2.** What isn't journaled: journal reports themselves (they carry a
queue flag, so a report to an outside archive can't journal itself), and the
server's own DMARC and TLS reports. DSNs and Sieve redirects and vacation
replies are journaled (question 4). A message **refused** at DATA (DLP block,
a transport rule's refusal) was never accepted and isn't journaled; the
audit log already records it. A message **held** for DLP review is journaled
when it's queued, which is when it's held, with the hold noted in the entry.
### 2.2 The journal report (JR-3, JR-4)
**JR-3.** Each copy is a **journal report**: a new message whose first part
is `text/plain`, one field a line:
```
Sender: alice@example.com
Signed in as: alice@example.com
Subject: Q3 figures
Message-ID: <…>
Queue ID: 1a2b3c…
Received: 2026-09-28T14:03:11Z
Direction: outgoing
To: bank@elsewhere.example
Cc: bob@example.com
Bcc: carol@example.com
Expanded: finance@example.com -> dan@example.com, erin@example.com
Held for review: yes
```
and whose second part is the message as queued, **byte for byte**, as
`message/rfc822`. `Bcc:` lists envelope recipients that aren't in the
message's To or Cc headers. `Expanded:` groups the members of a list under
the list address, from their ORCPT. Recipients a transport rule added say so
(`Added by rule: <name>`). The field names are fixed English (they're a
record, not interface text), so a script can read them.
**JR-4.** One report per queued message, with the whole envelope, whatever
the scope matched on (§2.4). A message to 40 recipients is one report, not
40.
### 2.3 Where reports go (JR-5 to JR-8)
Each journal has a **destination** (question 1):
**JR-5. The built-in journal.** Records under a new prefix `J` in
`SUBSPACE_INBUXA`: queue id, received time, direction, sender, recipients,
the tenant(s), which journal matched, the report's blob hash and size, its
SHA-256, and the time it may be purged. The report blob is kept by a
`BlobLink::Temporary { until }` set to the end of its retention. There is no
JMAP `set` or `destroy` for entries: nothing in the product changes or
removes one before its time.
**JR-6. The chain.** Each entry carries the SHA-256 of the entry before it,
one chain per node, the same construction as the audit log (and its code,
generalized rather than copied). The console's **Check the journal** walks
it, and every blob's hash against its entry, and says what it found. Someone
with the server's disks can still remove data, and the chain is how that
shows; the docs say exactly that, and don't say it can't happen.
**JR-7. An outside archive.** The report is queued to an address (the
archive's journal mailbox) like any mail, with the queue's retries. A report
the archive refuses permanently, or can't take within the queue's limit, goes
into the built-in journal instead and raises a warning on the Overview
(question 6). Delivery is by the ordinary queue, so TLS and routing settings
apply; a queue route can be chosen for it.
**JR-8. Both**: the built-in journal and an outside archive.
### 2.4 Which mail: journals and their scope (JR-9 to JR-11)
**JR-9.** A **journal** is a named object (`inbuxa:Journal`): on or off, a
destination, a retention, and a scope. The scope is who: **everyone**, or
senders and recipients in chosen **accounts, groups, domains or tenants**,
and which **direction**: outgoing, incoming, internal, any. A message is
journaled once per journal whose scope any sender or recipient is in; two
journals with the same destination never write the same message twice.
**JR-10.** **By what's in it**: a new mail flow rule action, **Journal it**,
names a journal. The rule's conditions (detectors, words, attachments,
headers) decide; the copy is still taken at queue time (the rule only marks
the message). This is the rule-based journaling the queue note called
premium; here it's one more action, not a separate tier (question 2).
**JR-11.** Scope is evaluated from the envelope and directory membership at
queue time (`Server::member_of`), no message parsing, so journaling
everything costs a lookup per recipient and one blob write per message.
### 2.5 Retention and legal hold (JR-12 to JR-14)
**JR-12.** Each journal has a retention in days (question 3). An entry keeps
the retention it was written with: shortening a journal's retention applies
to new entries only, so nobody can empty the journal by editing a number.
Lengthening it applies to new entries too, and the console says so.
**JR-13.** Purge runs in the daily maintenance, removes entries past their
time and drops their blob link, and leaves a floor hash so the chain still
verifies, as the audit log does. An entry whose sender or any recipient is
under a **legal hold** isn't purged while the hold lasts (`holds_on`, read
uncached, as holds are everywhere).
**JR-14.** Deleting an account doesn't remove its journal entries; they end
with their retention (question 7). The privacy catalog says so.
### 2.6 Search, reading, export (JR-15 to JR-17)
**JR-15.** **Management › Compliance › Journal**: search by sender,
recipient, date range, direction, subject words (from the report's header
fields, not the body: no full-text index of the journal in this version).
Results list the envelope; **Read…** opens the report.
**JR-16.** **Export** a search as a ZIP in the hold export's shape: the
reports as `.eml`, `manifest.csv` with the envelope columns and a SHA-256
per file, `manifest.sha256`, the same 2 GiB cap. Export runs as a task and
the result is a blob owned by the person who asked for it.
**JR-17.** Every search, read and export is in the audit log, with who and
the search terms; so is every change to a journal.
### 2.7 Permissions (JR-18)
**JR-18.** New permissions after the DLP set (680 onward):
`sysJournalGet` / `sysJournalUpdate` (see and change journals),
`sysJournalSearch` (search and read entries), `sysJournalExport`. Superuser
only, by default. The officer grant audience adds Get, Search and Export to
the Compliance Officer; administrators configure journals but don't read
them unless granted Search (question 5). Journals are server-level, with a
tenant scope, as DLP rules are; nobody in a tenant reaches them.
### 2.8 Privacy catalog
New objects get catalog entries (`resources/privacy/catalog.toml`):
`inbuxa:Journal` (none), `inbuxa:JournalEntry` (mail content and envelope,
kept for the journal's retention, access audited, not erased with the
account). `privacy-check.py` enforces it.
### 2.9 Mixed versions, clusters, rollback
- Entries and journals live in the shared data store; report blobs in the
blob store. A **node-local** blob store (FileSystem, or RocksDB/SQLite as
the blob store) on a cluster means a node's journal lives on that node;
the console warns when journaling is on and the blob store isn't shared.
- During a rolling upgrade a node on the old version doesn't journal. The
console says so when nodes report different versions; the release notes
say to turn journals on after every node is upgraded.
- Rollback: the new prefix and the queue flag are ignored by an older
version; nothing in the queue's archived format changes (the "journal
report" flag rides in the existing message flags if one is free, else in
a side key by queue id; checked in phase 2 before writing code).
### 2.10 Cost
One extra blob per journaled message (the report wraps the original, so it
doesn't share its hash), plus one small record. The console shows the
journal's size and growth per day on the journal page, from the entries.
## 3. Console
- **Management › Compliance › Journaling**: journals (name, scope,
destination, retention, on/off), described in words like DLP rules
("Journal all mail to and from Finance into the built-in journal, kept
7 years"); **Check the journal**.
- **Management › Compliance › Journal**: search, read, export.
- The mail flow rule editor gains **Journal it**.
- The Overview warns about undelivered outside reports (JR-7) and a
node-local blob store (§2.9).
## 4. Webmail
Nothing. People aren't told a message was journaled, as they aren't told
about legal hold; the docs say journaling exists and what it captures.
## 5. Tests
Unit: the report's fields (Bcc computed from headers, list expansion from
ORCPT, rule-added recipients), scope matching, retention arithmetic, the
chain. Integration (`tests/src/system/`): SMTP and JMAP sends, inbound
mail, internal mail, a list and a Bcc recipient, a DLP-held message, a
Sieve redirect; the report equals the queued bytes; no `set`/`destroy`;
shortening retention doesn't touch existing entries; a hold stops purge;
account deletion leaves entries; an outside archive that refuses falls back
to the built-in journal; export manifest hashes; audit records for search,
read, export.
## 6. Phases
1. This spec, approved.
2. Capture at the queue, the report, the built-in journal with its chain,
retention and purge, holds; `inbuxa:Journal` and `inbuxa:JournalEntry`;
catalog entries; tests.
3. Outside archive and the fallback; **Journal it** in mail flow rules.
4. Search, read and export (a task), audit records.
5. Console pages; docs; a row in `inbuxa-drafts/divergence-log.md`.
Each phase is its own PR with tests; releases as John decides. Like DLP, it
stays out of production until John says.
## As built
Phase 2 (`feature/journal-capture`), where it differs from the design or
fills in what it left open:
- **The chain** is the journal's own (`crates/features/src/journal/
entries.rs`), not the audit log's code shared. Entries expire out of chain
order (each keeps its journal's retention, and holds keep some longer), so
a link names its entry by SHA-256 instead of holding it: purging removes
the entry, its indexes and its report's blob link, and writes a purge
marker; the link stays. An entry missing without a marker is a broken
chain. Purged links at a chain's start are cleared and a floor recorded,
as the audit log does.
- **If the copy can't be taken**, the message isn't queued: the sender gets
a temporary failure and tries again. Nothing leaves unjournaled.
- **The report** says `Authenticated: yes|no` instead of the signed-in
account (the queue doesn't keep which account it was). `Added by rule`
comes with **Journal it** in phase 3. A recipient given with an ORCPT
that names another address counts as expanded from that address.
- **Journal reports** the server queues carry message flag bit 48
(`FROM_JOURNAL`); an older version ignores the bit.
- **Permissions 680–683**: administrators get `sysJournalGet`/`Update`; the
Compliance Officer gets `Get`, `Search` and `Export`. So that an
administrator can still appoint an officer (and grant reading as settled
answer 5 describes), whoever holds `sysJournalUpdate` may grant `Search`
and `Export` without holding them; the role change is in the audit log.
- **`inbuxa:JournalEntry`** (get, query) and **Check the journal** over
JMAP come in phase 4 with search, so every read is audited from the first
version that allows one. Phase 2 has `inbuxa:Journal` only.
Phase 3 (`feature/journal-archive`):
- **Destinations** are two properties of a journal: `builtIn` (true for
journals stored before phase 3) and `archiveAddress`. At least one.
- **Journals only rules use**: a journal whose scope chooses nobody takes
only what a **Journal it** action sends it. The action goes on mail flow
rules, and on a DLP rule beside its block, warn or hold (a blocked
message isn't queued, so it isn't journaled).
- **Reports to an archive** are queued from the empty sender, so a refusal
comes back to no one; a pending record per report says what to keep.
When the queue lets go of a report without delivering it (refused,
expired, or deleted from the queue), the report becomes its own entry in
the built-in journal under the sending journals' retention, even when
another journal already kept the message there, the journal's
`archiveFailures` (count, last time, reason) goes up, and the audit log
records it. If that can't be written, the report stays queued.
- **Added by rule** lists recipients a transport rule added or redirected
to, by rule name, instead of counting them as Bcc.
- A rule's route (and now its journal marks) is cleared between messages
in one SMTP session; before, a second message in the same session kept
the first one's route.
Phase 4 (`feature/journal-search`):
- `inbuxa:JournalEntry/query` (after, before, sender, recipient, address,
direction, subject words, Message-ID, journal; newest first, pages of up
to 500) and `/get` (`report`, the whole journal report up to 10 MB of
text, only when asked for), with `sysJournalSearch`.
- Recording (JR-17) happens before anything is returned, and nothing is
returned if it can't be written: a search with its terms, a listing
once per call, each report read on its own (as `blobAccess`, the
action reads of someone's mail already use), each export with its
reason (`export`), each check (`verify`). No new audit actions, so an
older version reads every record.
- `inbuxa:JournalExport/set` builds the ZIP in the request, like the audit
log's export, rather than as a task: at most 10,000 reports and 1 GB,
and a search that matches more is refused with the count, to narrow.
The ZIP has the reports as `.eml`, `manifest.csv` with the envelope and
a SHA-256 per file, `exceptions.csv` for reports that couldn't be read,
and `manifest.sha256`.
- `inbuxa:JournalVerification/set` rechecks the chains and every report
against its entry, with `sysJournalGet`.
Phase 5 (inbuxa-admin #62, server #119 for the menu, docs inbuxa.org #32):
- One console page, **Management › Compliance › Journal**, with two tabs
instead of the two pages §3 named: **Search** (for those who may search)
and **Journals** (the editor, on/off, delete, archive warnings, and Check
the journal).
- The warnings §3 put on the Overview (undelivered archive reports, a
node-local blob store) aren't there: archive failures show on each
journal, and there's no blob-store warning yet.
- The rule editor's **Journal it**, on mail flow rules and as an optional
second action on DLP rules.
## Known gaps
- A message a person saves to Sent over IMAP, or sends through another
server, never reaches the queue.
- Mail stored before journaling is on isn't journaled (legal hold covers
mailboxes).
- Search reads envelope and header fields, not bodies.
- Group accounts (`GroupAccount`) resolve as one account, not members; their
mail is journaled under the group's address.
## Settled
John, 2026-09-28, all seven as recommended:
1. **Destinations**: the built-in journal, an outside archive by address, or
both, per journal (JR-5, JR-7, JR-8).
2. **Scope**: everyone, or chosen accounts, groups, domains and tenants by
direction, plus a **Journal it** rule action; no standard/premium split
(JR-9, JR-10).
3. **Retention**: no default; 30 days to 10 years, picked when a journal is
turned on; existing entries keep theirs (JR-12).
4. **Which mail**: everything queued, including DSNs, Sieve redirects and
vacation replies, except DMARC/TLS reports and journal reports (JR-2).
5. **Who reads it**: administrators configure; Compliance Officers search,
read and export; administrators read only if granted Search (JR-18).
6. **An outside archive that won't take a report**: kept in the built-in
journal, with a warning (JR-7).
7. **Deleted accounts**: journal entries stay until their retention ends,
and the catalog says so (JR-14).
Binary file not shown.
+66
View File
@@ -113,6 +113,59 @@ exceptions = ["contact", "content"]
actions = ["contact", "content"]
createdBy = ["identifier"]
[object."inbuxa:Journal"]
file = "inbuxa_journal.rs"
default = "none"
whose = ["administrator"]
where = ["data-store"]
scope = "server"
retention = "unbounded"
[object."inbuxa:Journal".properties]
name = ["content"]
description = ["content"]
createdBy = ["identifier"]
[object."inbuxa:SecurityAcceptance"]
file = "inbuxa_security_acceptance.rs"
default = "none"
whose = ["administrator"]
where = ["data-store"]
scope = "server"
retention = "object-life"
[object."inbuxa:SecurityAcceptance".properties]
note = ["content"]
acceptedBy = ["identifier"]
[object."inbuxa:JournalEntry"]
file = "inbuxa_journal_entry.rs"
default = "none"
whose = ["holder", "correspondent"]
where = ["data-store", "blob-store"]
scope = "server"
retention = { setting = "inbuxa:Journal.retentionDays" }
[object."inbuxa:JournalEntry".properties]
sender = ["identifier", "contact"]
recipients = ["identifier", "contact"]
subject = ["content"]
messageId = ["identifier"]
report = ["content", "identifier", "contact", "metadata"]
[object."inbuxa:JournalExport"]
file = "inbuxa_journal_entry.rs"
default = "none"
whose = ["holder", "correspondent"]
where = ["blob-store"]
scope = "server"
retention = { setting = "x:Jmap.uploadTtl" }
[object."inbuxa:JournalExport".properties]
blobId = ["identifier", "contact", "content"]
filter = ["identifier", "contact"]
reason = ["content"]
[object."inbuxa:JournalVerification"]
file = "inbuxa_journal_entry.rs"
default = "none"
[object."inbuxa:LegalHold"]
file = "inbuxa_legal_hold.rs"
default = "none"
@@ -408,6 +461,19 @@ captures = ["x:Email.maxMaskedAddresses"]
leaves_host = false
written_by = ["crates/features/src/masked_email/data.rs"]
# Journaling (journaling spec, JR-5, JR-14): a copy of each message a
# journal takes, with its envelope, kept for the journal's retention even
# after the account is deleted, and longer while a legal hold covers
# someone on it.
[source."journal"]
categories = ["content", "identifier", "contact", "metadata"]
whose = ["holder", "correspondent"]
where = ["data-store", "blob-store"]
scope = "server"
retention = { setting = "inbuxa:Journal.retentionDays" }
leaves_host = false
written_by = ["crates/features/src/journal/entries.rs", "crates/smtp/src/queue/journal.rs"]
[source."outbound-reports"]
categories = ["network", "identifier", "content"]
whose = ["correspondent"]
Binary file not shown.
+1 -1
View File
@@ -1 +1 @@
bLr7o49yK6CtWpENWq5zjgKrbCSzAXW3DZfNAEpnXEE
D8e0s1e4Umau4gRh5MEW24KtsawKGPNvS-6LWrIFbtQ
+931
View File
@@ -0,0 +1,931 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! Journaling (journaling spec, phase 2): journals over JMAP, the copy
//! taken as mail is queued with its whole envelope, the report around the
//! untouched message, retention, purge, and a chain that shows tampering.
use crate::utils::{
account::Account,
server::{TestServer, TestServerBuilder},
smtp::SmtpConnection,
};
use inbuxa_features::journal::{
Direction,
entries::{self, Entry, EntryId},
report,
};
use registry::schema::{
prelude::{ObjectType, Property},
structs::{CustomRoles, Expression, MtaStageAuth, Role, UserRoles},
};
use registry::types::map::Map;
use serde_json::{Value, json};
use std::str::FromStr;
use store::{Deserialize, write::BatchBuilder};
const USING: &[&str] = &[
"urn:ietf:params:jmap:core",
"urn:ietf:params:jmap:mail",
"urn:ietf:params:jmap:submission",
"urn:inbuxa:jmap",
"urn:inbuxa:jmap:registry",
];
async fn call(account: &Account, method: &str, mut arguments: Value) -> (String, Value) {
if arguments.get("accountId").is_none() {
arguments["accountId"] = account.id_string().into();
}
let response = account
.jmap_request(USING, json!([[method, arguments, "0"]]))
.await;
let call = response
.0
.pointer("/methodResponses/0")
.cloned()
.unwrap_or_else(|| panic!("{method}: {}", response.0));
(
call[0].as_str().unwrap_or_default().to_string(),
call[1].clone(),
)
}
/// Sends a message whose headers name `to`, to the envelope `rcpt_to`.
async fn send(
sender: &Account,
identity: &str,
mailbox: &str,
to: &[&str],
rcpt_to: &[&str],
subject: &str,
) -> Value {
let (_, response) = call(
sender,
"Email/set",
json!({"create": {"e": {
"mailboxIds": {mailbox: true},
"from": [{"email": sender.name()}],
"to": to.iter().map(|a| json!({"email": a})).collect::<Vec<_>>(),
"subject": subject,
"bodyValues": {"b": {"value": "The body."}},
"textBody": [{"partId": "b", "type": "text/plain"}]
}}}),
)
.await;
let email = response["created"]["e"]["id"]
.as_str()
.unwrap_or_else(|| panic!("draft: {response}"))
.to_string();
call(
sender,
"EmailSubmission/set",
json!({"create": {"s": {
"emailId": email,
"identityId": identity,
"envelope": {
"mailFrom": {"email": sender.name()},
"rcptTo": rcpt_to.iter().map(|a| json!({"email": a})).collect::<Vec<_>>()
}
}}}),
)
.await
.1
}
async fn all_entries(test: &TestServer) -> Vec<(EntryId, Entry)> {
entries::list(test.server.store(), 0, u64::MAX, 10_000)
.await
.unwrap()
}
async fn entry_for(test: &TestServer, subject: &str) -> Option<(EntryId, Entry)> {
all_entries(test)
.await
.into_iter()
.find(|(_, e)| e.subject == subject)
}
async fn report_of(test: &TestServer, entry: &Entry) -> Vec<u8> {
let hash = entry.blob_hash().expect("blob hash");
test.server
.blob_store()
.get_blob(hash.as_slice(), 0..usize::MAX)
.await
.unwrap()
.expect("report blob")
}
pub async fn test(test: &mut TestServer) {
println!("Running journaling tests...");
let admin = test.account("[email protected]");
let sender = admin
.create_user_account(
"[email protected]",
"journal-sender-secret-7101",
"Journal sender",
&[],
vec![],
)
.await;
let other = admin
.create_user_account(
"[email protected]",
"journal-other-secret-7102",
"Journal other",
&[],
vec![],
)
.await;
let (_, response) = call(
&sender,
"Identity/set",
json!({"create": {"i": {"name": "Sender", "email": "[email protected]"}}}),
)
.await;
let identity = response["created"]["i"]["id"].as_str().unwrap().to_string();
let (_, response) = call(
&sender,
"Mailbox/set",
json!({"create": {"m": {"name": "Journal drafts"}}}),
)
.await;
let mailbox = response["created"]["m"]["id"].as_str().unwrap().to_string();
// Nothing is journaled while there are no journals
let response = send(
&sender,
&identity,
&mailbox,
&["[email protected]"],
&["[email protected]"],
"Before any journal",
)
.await;
assert!(response["created"].get("s").is_some(), "{response}");
assert!(all_entries(test).await.is_empty());
// Journals: checked when written, the server's own properties refused
let (_, response) = call(
&admin,
"inbuxa:Journal/set",
json!({"create": {
"short": {"name": "Short", "enabled": true, "direction": "any",
"scope": {"everyone": true}, "retentionDays": 29},
"both": {"name": "Both", "enabled": true, "direction": "any",
"scope": {"everyone": true, "accounts": [sender.id_string()]},
"retentionDays": 365},
"none": {"name": "Nowhere", "enabled": true, "direction": "any",
"scope": {"everyone": true}, "retentionDays": 365, "builtIn": false},
"badaddr": {"name": "Bad archive", "enabled": true, "direction": "any",
"scope": {"everyone": true}, "retentionDays": 365,
"archiveAddress": "not an address"},
"server": {"name": "Mine", "enabled": true, "direction": "any",
"scope": {"everyone": true}, "retentionDays": 365,
"createdBy": "me"},
"all": {"name": "Everything", "enabled": true, "direction": "any",
"scope": {"everyone": true}, "retentionDays": 365},
"out": {"name": "Sender's outgoing", "enabled": true, "direction": "outgoing",
"scope": {"accounts": [sender.id_string()]}, "retentionDays": 3650}
}}),
)
.await;
for refused in ["short", "both", "none", "badaddr", "server"] {
assert_eq!(
response["notCreated"][refused]["type"], "invalidProperties",
"{refused}: {response}"
);
}
assert_eq!(
response["notCreated"]["short"]["properties"],
json!(["retentionDays"])
);
assert_eq!(
response["notCreated"]["both"]["properties"],
json!(["scope"])
);
assert_eq!(
response["notCreated"]["none"]["properties"],
json!(["builtIn"])
);
assert_eq!(
response["notCreated"]["badaddr"]["properties"],
json!(["archiveAddress"])
);
let everything = response["created"]["all"]["id"]
.as_str()
.unwrap_or_else(|| panic!("{response}"))
.to_string();
let outgoing = response["created"]["out"]["id"]
.as_str()
.unwrap()
.to_string();
let (_, response) = call(&admin, "inbuxa:Journal/get", json!({"ids": null})).await;
let list = response["list"].as_array().unwrap();
assert_eq!(list.len(), 2, "{response}");
assert_eq!(list[0]["name"], "Everything");
assert_eq!(list[0]["createdBy"], "[email protected]");
assert_eq!(list[1]["scope"]["accounts"], json!([sender.id_string()]));
// Each node reads journals again within 30 seconds; this one at once
inbuxa_features::journal::invalidate();
// Internal mail with a Bcc recipient: one entry, the whole envelope
let response = send(
&sender,
&identity,
&mailbox,
&["[email protected]"],
&["[email protected]", "[email protected]"],
"Internal with Bcc",
)
.await;
assert!(response["created"].get("s").is_some(), "{response}");
let (_, entry) = entry_for(test, "Internal with Bcc")
.await
.expect("journaled");
assert_eq!(entry.direction, Direction::Internal);
assert_eq!(entry.sender, "[email protected]");
assert!(entry.authenticated);
assert_eq!(entry.recipients.len(), 2, "{entry:?}");
assert_eq!(
entry.journals.len(),
1,
"internal isn't outgoing: {entry:?}"
);
assert!(!entry.held);
assert_eq!(entry.expires_at, entry.at + 365 * 86_400);
let bytes = report_of(test, &entry).await;
assert_eq!(entries::sha256(&bytes), entry.sha256);
let text = String::from_utf8_lossy(&bytes);
assert!(text.contains("Direction: internal\r\n"), "{text}");
assert!(
text.contains("To: [email protected]\r\n"),
"{text}"
);
assert!(
text.contains("Bcc: [email protected]\r\n"),
"{text}"
);
let original = report::original(&bytes).expect("original part");
let original = String::from_utf8_lossy(original);
assert!(
original.contains("Subject: Internal with Bcc"),
"{original}"
);
assert!(original.contains("The body."), "{original}");
assert!(!original.contains("Bcc:"), "the original is as sent");
// Outgoing: both journals take it, and it's kept for the longer
let response = send(
&sender,
&identity,
&mailbox,
&["[email protected]"],
&["[email protected]"],
"Leaving",
)
.await;
assert!(response["created"].get("s").is_some(), "{response}");
let (leaving_id, entry) = entry_for(test, "Leaving").await.expect("journaled");
assert_eq!(entry.direction, Direction::Outgoing);
assert_eq!(entry.journals.len(), 2, "{entry:?}");
assert_eq!(entry.expires_at, entry.at + 3650 * 86_400);
// Incoming from outside
admin
.registry_create_object(MtaStageAuth {
require: Expression {
else_: "false".to_string(),
..Default::default()
},
..Default::default()
})
.await;
let mut lmtp = SmtpConnection::connect().await;
lmtp.ingest(
"[email protected]",
&["[email protected]"],
"From: [email protected]\r\nTo: [email protected]\r\nSubject: Arriving\r\n\r\nHi.\r\n",
)
.await;
let (_, entry) = entry_for(test, "Arriving").await.expect("journaled");
assert_eq!(entry.direction, Direction::Incoming);
assert!(!entry.authenticated);
assert_eq!(entry.accounts, vec![other.id().document_id()]);
// The chain checks out, reports included
let store = test.server.store();
let blobs = test.server.blob_store();
let reports = entries::verify(store, Some(blobs)).await.unwrap();
assert!(reports.iter().all(|r| r.broken_at.is_none()), "{reports:?}");
let journaled = all_entries(test).await.len() as u64;
assert!(reports.iter().map(|r| r.entries).sum::<u64>() >= journaled);
// An entry changed in the store shows; put back, it checks out again
let key = entries::content_key(leaving_id);
let stored = store
.get_value::<Raw>(key.clone())
.await
.unwrap()
.expect("stored entry")
.0;
let mut forged: Entry = serde_json::from_slice(&stored).unwrap();
forged.recipients = vec!["[email protected]".into()];
let mut batch = BatchBuilder::new();
batch.set(key.class.clone(), serde_json::to_vec(&forged).unwrap());
store.write(batch.build_all()).await.unwrap();
let reports = entries::verify(store, None).await.unwrap();
let broken = reports
.iter()
.find(|r| r.broken_at.is_some())
.expect("broken");
assert_eq!(
broken.broken_at.as_deref(),
Some(leaving_id.to_string().as_str())
);
assert!(
broken
.reason
.as_deref()
.unwrap_or_default()
.contains("changed")
);
let mut batch = BatchBuilder::new();
batch.set(key.class.clone(), stored.clone());
store.write(batch.build_all()).await.unwrap();
assert!(
entries::verify(store, None)
.await
.unwrap()
.iter()
.all(|r| r.broken_at.is_none())
);
// An entry removed without a purge shows too
let mut batch = BatchBuilder::new();
batch.clear(key.class.clone());
store.write(batch.build_all()).await.unwrap();
let reports = entries::verify(store, None).await.unwrap();
assert!(
reports.iter().any(|r| r
.reason
.as_deref()
.unwrap_or_default()
.contains("before its time")),
"{reports:?}"
);
let mut batch = BatchBuilder::new();
batch.set(key.class.clone(), stored);
store.write(batch.build_all()).await.unwrap();
// Retention: nothing is due yet; a year on, what's kept for a hold
// stays, the rest goes, and the chain still checks out
let now = store::write::now();
let purged = entries::purge(store, now, |_| false).await.unwrap();
assert_eq!(purged.removed, 0);
let sender_id = sender.id().document_id();
let later = now + 400 * 86_400;
let purged = entries::purge(store, later, |e| e.accounts.contains(&sender_id))
.await
.unwrap();
assert!(purged.removed >= 1, "{purged:?}");
assert!(purged.kept_for_hold >= 1, "{purged:?}");
assert!(entry_for(test, "Arriving").await.is_none(), "purged");
assert!(entry_for(test, "Internal with Bcc").await.is_some(), "held");
assert!(entry_for(test, "Leaving").await.is_some(), "ten years");
let reports = entries::verify(store, Some(blobs)).await.unwrap();
assert!(reports.iter().all(|r| r.broken_at.is_none()), "{reports:?}");
assert!(reports.iter().map(|r| r.purged).sum::<u64>() >= 1);
// Once the hold is gone the held entry goes too
let purged = entries::purge(store, later, |_| false).await.unwrap();
assert!(purged.removed >= 1, "{purged:?}");
assert!(entry_for(test, "Internal with Bcc").await.is_none());
assert!(
entries::verify(store, Some(blobs))
.await
.unwrap()
.iter()
.all(|r| r.broken_at.is_none())
);
// Changing a journal's retention doesn't touch what it has taken
let before = entry_for(test, "Leaving").await.unwrap().1.expires_at;
let (_, response) = call(
&admin,
"inbuxa:Journal/set",
json!({"update": {outgoing.clone(): {"retentionDays": 30}}}),
)
.await;
assert!(response["updated"].get(&outgoing).is_some(), "{response}");
assert_eq!(
entry_for(test, "Leaving").await.unwrap().1.expires_at,
before
);
// Journals turned off or removed take nothing more; entries stay
let (_, response) = call(
&admin,
"inbuxa:Journal/set",
json!({"update": {everything.clone(): {"enabled": false}}, "destroy": [outgoing]}),
)
.await;
assert!(response["updated"].get(&everything).is_some(), "{response}");
assert_eq!(response["destroyed"].as_array().map(|d| d.len()), Some(1));
inbuxa_features::journal::invalidate();
let count = all_entries(test).await.len();
let response = send(
&sender,
&identity,
&mailbox,
&["[email protected]"],
&["[email protected]"],
"After the journals",
)
.await;
assert!(response["created"].get("s").is_some(), "{response}");
assert_eq!(all_entries(test).await.len(), count);
assert!(entry_for(test, "Leaving").await.is_some());
// Every change to a journal is in the audit log
let (_, response) = call(
&admin,
"inbuxa:AuditEvent/query",
json!({"filter": {"targetKind": "inbuxa:Journal"}}),
)
.await;
assert!(
response["ids"].as_array().map_or(0, |ids| ids.len()) >= 4,
"{response}"
);
}
/// Phase 3: journals only rules send mail to, recipients a rule added,
/// reports sent to an outside archive, and what happens when the archive
/// doesn't take one.
pub async fn archive(test: &mut TestServer) {
println!("Running journal archive tests...");
let admin = test.account("[email protected]");
let sender = admin
.create_user_account(
"[email protected]",
"archive-sender-secret-7201",
"Archive sender",
&[],
vec![],
)
.await;
let vault = admin
.create_user_account(
"[email protected]",
"journal-vault-secret-7202",
"Journal vault",
&[],
vec![],
)
.await;
let (_, response) = call(
&sender,
"Identity/set",
json!({"create": {"i": {"name": "Sender", "email": "[email protected]"}}}),
)
.await;
let identity = response["created"]["i"]["id"].as_str().unwrap().to_string();
let (_, response) = call(
&sender,
"Mailbox/set",
json!({"create": {"m": {"name": "Archive drafts"}}}),
)
.await;
let mailbox = response["created"]["m"]["id"].as_str().unwrap().to_string();
let (_, response) = call(
&admin,
"inbuxa:Journal/set",
json!({"create": {
"rules": {"name": "Only what rules send", "enabled": true, "direction": "any",
"scope": {}, "retentionDays": 30},
"local": {"name": "To the vault", "enabled": true, "direction": "outgoing",
"scope": {"accounts": [sender.id_string()]}, "retentionDays": 30,
"builtIn": false, "archiveAddress": "[email protected]"},
"remote": {"name": "To an outside archive", "enabled": true, "direction": "internal",
"scope": {"accounts": [sender.id_string()]}, "retentionDays": 30,
"builtIn": false, "archiveAddress": "[email protected]"}
}}),
)
.await;
let id = |name: &str| {
response["created"][name]["id"]
.as_str()
.unwrap_or_else(|| panic!("{name}: {response}"))
.to_string()
};
let (rules_only, local, remote) = (id("rules"), id("local"), id("remote"));
let number = |id: &str| types::id::Id::from_str(id).unwrap().document_id();
let (_, response) = call(
&admin,
"inbuxa:MailRule/set",
json!({"create": {"r": {
"name": "Copy and journal", "kind": "transport", "direction": "outgoing",
"conditions": [{"type": "words", "words": ["journal-me"]}],
"actions": [
{"type": "addRecipient", "address": "[email protected]"},
{"type": "journal", "journal": rules_only.clone()}
]
}}}),
)
.await;
let rule = response["created"]["r"]["id"]
.as_str()
.unwrap_or_else(|| panic!("{response}"))
.to_string();
inbuxa_features::journal::invalidate();
// A rule sends it to a journal whose scope takes nobody, and says who
// it added
let response = send(
&sender,
&identity,
&mailbox,
&["[email protected]"],
&["[email protected]"],
"Marked journal-me",
)
.await;
assert!(response["created"].get("s").is_some(), "{response}");
let entry = all_entries(test)
.await
.into_iter()
.map(|(_, e)| e)
.find(|e| e.subject == "Marked journal-me" && e.journals.contains(&number(&rules_only)))
.expect("journaled by the rule");
assert_eq!(entry.journals, vec![number(&rules_only)], "{entry:?}");
let text = String::from_utf8_lossy(&report_of(test, &entry).await).into_owned();
assert!(
text.contains("Added by rule: Copy and journal -> [email protected]\r\n"),
"{text}"
);
assert!(!text.contains("Bcc:"), "{text}");
// The same message went to the outside archive, which can't be reached
// from here: once it leaves the queue (given up on, or deleted), it's
// kept in the built-in journal
let fallback = |entries: &[(EntryId, Entry)]| {
entries
.iter()
.any(|(_, e)| e.subject == "Marked journal-me" && e.journals == vec![number(&remote)])
};
let mut deleted = false;
for _ in 0..100 {
if fallback(&all_entries(test).await) {
break;
}
let (_, response) = call(&admin, "x:QueuedMessage/get", json!({"ids": null})).await;
if let Some(queued) = response["list"]
.as_array()
.unwrap()
.iter()
.find(|m| m.to_string().contains("[email protected]"))
{
let queued_id = queued["id"].as_str().unwrap().to_string();
let (_, response) = call(
&admin,
"x:QueuedMessage/set",
json!({"destroy": [queued_id.clone()]}),
)
.await;
deleted = response["destroyed"] == json!([queued_id]);
}
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
}
let kept: Vec<Entry> = all_entries(test)
.await
.into_iter()
.map(|(_, e)| e)
.filter(|e| e.subject == "Marked journal-me")
.collect();
assert_eq!(kept.len(), 2, "{kept:?}");
assert!(kept.iter().any(|e| e.journals == vec![number(&remote)]));
let (_, response) = call(
&admin,
"inbuxa:Journal/get",
json!({"ids": [remote.clone()]}),
)
.await;
let failures = &response["list"][0]["archiveFailures"];
assert_eq!(failures["count"], 1, "{response}");
assert_eq!(
failures["lastReason"],
if deleted {
"it wasn't delivered before leaving the queue"
} else {
"the archive refused it"
},
"{response}"
);
let (_, response) = call(
&admin,
"inbuxa:Journal/get",
json!({"ids": [local.clone()]}),
)
.await;
assert_eq!(response["list"][0]["archiveFailures"]["count"], 0);
// Delivered to an archive here: the report arrives, and nothing goes
// into the built-in journal for that journal
let response = send(
&sender,
&identity,
&mailbox,
&["[email protected]"],
&["[email protected]"],
"To the vault",
)
.await;
assert!(response["created"].get("s").is_some(), "{response}");
let mut arrived = Vec::new();
for _ in 0..100 {
let (_, response) = call(
&vault,
"Email/query",
json!({"filter": {"subject": "Journal report: To the vault"}}),
)
.await;
arrived = response["ids"].as_array().cloned().unwrap_or_default();
if !arrived.is_empty() {
break;
}
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
}
assert_eq!(arrived.len(), 1, "the report arrived");
assert!(entry_for(test, "To the vault").await.is_none());
assert!(
all_entries(test)
.await
.iter()
.all(|(_, e)| !e.subject.starts_with("Journal report")),
"reports aren't journaled"
);
let (_, response) = call(
&admin,
"inbuxa:Journal/get",
json!({"ids": [local.clone()]}),
)
.await;
assert_eq!(response["list"][0]["archiveFailures"]["count"], 0);
call(&admin, "inbuxa:MailRule/set", json!({"destroy": [rule]})).await;
call(
&admin,
"inbuxa:Journal/set",
json!({"destroy": [rules_only, local, remote]}),
)
.await;
inbuxa_features::journal::invalidate();
}
/// Phase 4: searching, reading and exporting over JMAP, by a Compliance
/// Officer, each recorded; administrators set journals up but don't read
/// them; the chain check.
pub async fn search(test: &mut TestServer) {
println!("Running journal search tests...");
let admin = test.account("[email protected]");
let sender = admin
.create_user_account(
"[email protected]",
"search-sender-secret-7301",
"Search sender",
&[],
vec![],
)
.await;
let (_, response) = call(
&sender,
"Identity/set",
json!({"create": {"i": {"name": "Sender", "email": "[email protected]"}}}),
)
.await;
let identity = response["created"]["i"]["id"].as_str().unwrap().to_string();
let (_, response) = call(
&sender,
"Mailbox/set",
json!({"create": {"m": {"name": "Search drafts"}}}),
)
.await;
let mailbox = response["created"]["m"]["id"].as_str().unwrap().to_string();
let (_, response) = call(
&admin,
"inbuxa:Journal/set",
json!({"create": {"s": {"name": "Search sender", "enabled": true, "direction": "any",
"scope": {"accounts": [sender.id_string()]}, "retentionDays": 30}}}),
)
.await;
let journal_id = response["created"]["s"]["id"]
.as_str()
.unwrap_or_else(|| panic!("{response}"))
.to_string();
inbuxa_features::journal::invalidate();
for subject in ["Budget draft", "Budget final", "Lunch"] {
let response = send(
&sender,
&identity,
&mailbox,
&["[email protected]"],
&["[email protected]"],
subject,
)
.await;
assert!(response["created"].get("s").is_some(), "{response}");
}
// Administrators set journals up but don't read them
let (name, response) = call(
&admin,
"inbuxa:JournalEntry/query",
json!({"filter": {"sender": "search-sender"}}),
)
.await;
assert_eq!(name, "error", "{response}");
// A Compliance Officer does
let mut officer_role = None;
for id in admin
.registry_query_ids(
ObjectType::Role,
Vec::<(&str, &str)>::new(),
Vec::<&str>::new(),
)
.await
{
let role = admin.registry_get::<Role>(id).await;
if role.description == "Compliance Officer" && role.member_tenant_id.is_none() {
officer_role = Some(id);
}
}
let officer = admin
.create_user_account(
"[email protected]",
"journal-officer-secret-7302",
"Officer",
&[],
vec![],
)
.await;
admin
.registry_update_object(
ObjectType::Account,
officer.id(),
json!({Property::Roles: UserRoles::Custom(CustomRoles {
role_ids: Map::new(vec![officer_role.expect("the officer role")]),
})}),
)
.await;
let (_, response) = call(
&officer,
"inbuxa:JournalEntry/query",
json!({"filter": {"sender": "search-sender", "text": "budget"}, "calculateTotal": true}),
)
.await;
assert_eq!(response["total"], 2, "{response}");
let ids = response["ids"].clone();
let (_, response) = call(&officer, "inbuxa:JournalEntry/get", json!({"ids": ids})).await;
let list = response["list"].as_array().unwrap();
assert_eq!(list.len(), 2, "{response}");
assert_eq!(list[0]["subject"], "Budget final", "newest first");
assert_eq!(list[0]["direction"], "outgoing");
assert_eq!(list[0]["journalIds"], json!([journal_id]));
assert!(list[0]["report"].is_null(), "only when asked for");
let first = list[0]["id"].as_str().unwrap().to_string();
let (_, response) = call(
&officer,
"inbuxa:JournalEntry/get",
json!({"ids": [first.clone()], "properties": ["subject", "report"]}),
)
.await;
let report = response["list"][0]["report"].as_str().unwrap_or_default();
assert!(
report.contains("Subject: Journal report: Budget final"),
"{response}"
);
assert!(report.contains("Sender: [email protected]\r\n"));
let (_, response) = call(
&officer,
"inbuxa:JournalEntry/query",
json!({"filter": {"journalId": journal_id, "direction": "incoming"}}),
)
.await;
assert_eq!(response["ids"], json!([]), "{response}");
let (name, _) = call(
&officer,
"inbuxa:JournalEntry/query",
json!({"filter": {"colour": "red"}}),
)
.await;
assert_eq!(name, "error");
// Exports need a reason, and hold every report the filter matches
let (_, response) = call(
&officer,
"inbuxa:JournalExport/set",
json!({"create": {"x": {"filter": {"sender": "search-sender"}}}}),
)
.await;
assert_eq!(
response["notCreated"]["x"]["properties"],
json!(["reason"]),
"{response}"
);
let (_, response) = call(
&officer,
"inbuxa:JournalExport/set",
json!({"create": {"x": {"filter": {"sender": "search-sender"}, "reason": "Case 12"}}}),
)
.await;
let export = &response["created"]["x"];
assert_eq!(export["count"], 3, "{response}");
assert!(export["blobId"].as_str().is_some());
assert_eq!(export["sha256"].as_str().map(str::len), Some(64));
// The chain check, which the officer may run too
let (_, response) = call(
&officer,
"inbuxa:JournalVerification/set",
json!({"create": {"v": {}}}),
)
.await;
assert_eq!(response["created"]["v"]["verified"], true, "{response}");
// The officer changes no journals
let (name, _) = call(
&officer,
"inbuxa:Journal/set",
json!({"destroy": [journal_id.clone()]}),
)
.await;
assert_eq!(name, "error");
// Every search, listing, read, export and check is recorded
let (_, response) = call(
&admin,
"inbuxa:AuditEvent/query",
json!({"filter": {"targetKind": "inbuxa:JournalEntry", "actorId": officer.id_string()}}),
)
.await;
let ids = response["ids"].clone();
let (_, response) = call(&admin, "inbuxa:AuditEvent/get", json!({"ids": ids})).await;
let details: Vec<String> = response["list"]
.as_array()
.unwrap()
.iter()
.map(|e| format!("{} {}", e["action"], e["details"]))
.collect();
for expected in [
"Searched the journal",
"Listed 2 journal entries",
"Read a journaled message from [email protected]",
"Exported 3 journal entries",
"verify",
] {
assert!(
details.iter().any(|d| d.contains(expected)),
"{expected}: {details:?}"
);
}
call(
&admin,
"inbuxa:Journal/set",
json!({"destroy": [journal_id]}),
)
.await;
inbuxa_features::journal::invalidate();
}
struct Raw(Vec<u8>);
impl Deserialize for Raw {
fn deserialize(bytes: &[u8]) -> trc::Result<Self> {
Ok(Raw(bytes.to_vec()))
}
}
#[ignore]
#[tokio::test(flavor = "multi_thread")]
pub async fn journal_tests() {
let mut test = TestServerBuilder::new("journal_tests")
.await
.with_default_listeners()
.await
.build()
.await;
let admin = test.create_admin_account("[email protected]").await;
test.insert_account(admin);
self::test(&mut test).await;
self::archive(&mut test).await;
self::search(&mut test).await;
if test.is_reset() {
test.temp_dir.delete();
}
}
+2
View File
@@ -15,6 +15,8 @@ pub mod account_lock; // inbuxa: account lock with delegation
pub mod legal_hold; // inbuxa: legal hold
pub mod compliance; // inbuxa: the compliance roles
pub mod mail_rules; // inbuxa: DLP and mail flow rules
pub mod security_acceptances; // inbuxa: accepted security to-do items
pub mod journal; // inbuxa: journaling
pub mod audit; // inbuxa: the audit log
pub mod authorization;
pub mod auto_reload; // inbuxa: registry writes apply at once
+233
View File
@@ -0,0 +1,233 @@
/*
* SPDX-FileCopyrightText: 2026 Coffey Labs
*
* SPDX-License-Identifier: AGPL-3.0-only
*/
//! `inbuxa:SecurityAcceptance` (security to-do list spec, SS-23 to SS-26):
//! an accepted item is kept with who, when and why, a note is required,
//! nothing is edited, only administrators may accept, and every acceptance
//! made or removed is in the audit log.
use crate::utils::{
account::Account,
server::{TestServer, TestServerBuilder},
};
use serde_json::{Value, json};
const USING: &[&str] = &[
"urn:ietf:params:jmap:core",
"urn:inbuxa:jmap",
"urn:inbuxa:jmap:registry",
];
async fn call(account: &Account, method: &str, mut arguments: Value) -> (String, Value) {
if arguments.get("accountId").is_none() {
arguments["accountId"] = account.id_string().into();
}
let response = account
.jmap_request(USING, json!([[method, arguments, "0"]]))
.await;
let call = response
.0
.pointer("/methodResponses/0")
.cloned()
.unwrap_or_else(|| panic!("{method}: {}", response.0));
(
call[0].as_str().unwrap_or_default().to_string(),
call[1].clone(),
)
}
async fn list(account: &Account) -> Vec<Value> {
let (name, response) = call(
account,
"inbuxa:SecurityAcceptance/get",
json!({"ids": null}),
)
.await;
assert_eq!(name, "inbuxa:SecurityAcceptance/get", "{response}");
response["list"].as_array().unwrap().clone()
}
pub async fn test(test: &mut TestServer) {
println!("Running security acceptance tests...");
let admin = test.account("[email protected]");
// Accepted, with the server's who and when
let (_, response) = call(
&admin,
"inbuxa:SecurityAcceptance/set",
json!({"create": {
"plain": {
"check": "SS-1",
"subject": "",
"acceptedValue": true,
"note": " Old scanners on the LAN; replaced in March. "
},
"relay": {
"check": "SS-2",
"acceptedValue": {"match": {}, "else": "is_local_ip(remote_ip)"},
"note": "The office printer relays through us."
}
}}),
)
.await;
let plain_id = response["created"]["plain"]["id"]
.as_str()
.unwrap_or_else(|| panic!("accepted: {response}"))
.to_string();
assert_eq!(
response["created"]["plain"]["acceptedBy"], "[email protected]",
"{response}"
);
let relay_id = response["created"]["relay"]["id"]
.as_str()
.unwrap()
.to_string();
let all = list(&admin).await;
assert_eq!(all.len(), 2, "{all:?}");
let plain = all.iter().find(|a| a["id"] == plain_id.as_str()).unwrap();
assert_eq!(plain["check"], "SS-1");
assert_eq!(plain["subject"], "");
assert_eq!(plain["acceptedValue"], true);
assert_eq!(plain["note"], "Old scanners on the LAN; replaced in March.");
assert!(
plain["acceptedAt"]
.as_str()
.is_some_and(|d| d.ends_with('Z')),
"{plain}"
);
let relay = all.iter().find(|a| a["id"] == relay_id.as_str()).unwrap();
assert_eq!(relay["acceptedValue"]["else"], "is_local_ip(remote_ip)");
// A note is required, the check must be one of ours, and what the
// server sets can't be sent
let (_, response) = call(
&admin,
"inbuxa:SecurityAcceptance/set",
json!({"create": {
"nonote": {"check": "SS-1", "acceptedValue": true, "note": " "},
"nocheck": {"check": "SS-99", "acceptedValue": true, "note": "x"},
"by": {"check": "SS-1", "acceptedValue": true, "note": "x", "acceptedBy": "someone"}
}}),
)
.await;
assert_eq!(
response["notCreated"]["nonote"]["properties"][0], "note",
"{response}"
);
assert_eq!(
response["notCreated"]["nocheck"]["properties"][0], "check",
"{response}"
);
assert_eq!(
response["notCreated"]["by"]["properties"][0], "acceptedBy",
"{response}"
);
assert_eq!(list(&admin).await.len(), 2);
// Replaced, never edited
let (name, response) = call(
&admin,
"inbuxa:SecurityAcceptance/set",
json!({"update": {plain_id.as_str(): {"note": "changed"}}}),
)
.await;
assert_eq!(name, "error", "an acceptance was edited: {response}");
// Only administrators: someone without the permissions neither sees
// nor accepts
let user = admin
.create_user_account(
"[email protected]",
"user-secret-8812",
"User",
&[],
vec![],
)
.await;
let (name, response) = call(
&user,
"inbuxa:SecurityAcceptance/set",
json!({"create": {"x": {"check": "SS-1", "acceptedValue": true, "note": "mine"}}}),
)
.await;
assert_eq!(name, "error", "a user accepted an item: {response}");
let (name, response) = call(&user, "inbuxa:SecurityAcceptance/get", json!({"ids": null})).await;
assert_eq!(name, "error", "a user read acceptances: {response}");
// Removed
let (_, response) = call(
&admin,
"inbuxa:SecurityAcceptance/set",
json!({"destroy": [plain_id, "zzzzzz"]}),
)
.await;
assert_eq!(response["destroyed"], json!([plain_id]), "{response}");
assert!(
response["notDestroyed"].get("zzzzzz").is_some(),
"{response}"
);
let remaining = list(&admin).await;
assert_eq!(remaining.len(), 1);
assert_eq!(remaining[0]["check"], "SS-2");
// SS-26: accepted and removed are both in the audit log, with who and
// what
let (_, response) = call(
&admin,
"inbuxa:AuditEvent/query",
json!({"filter": {"targetKind": "inbuxa:SecurityAcceptance"}}),
)
.await;
let ids = response["ids"].clone();
let (_, response) = call(&admin, "inbuxa:AuditEvent/get", json!({"ids": ids})).await;
let events = response["list"].as_array().unwrap();
let created = events
.iter()
.filter(|e| e["action"] == "create" && e["outcome"]["status"] == "success")
.count();
assert_eq!(created, 2, "{response}");
let removed = events
.iter()
.find(|e| e["action"] == "destroy" && e["outcome"]["status"] == "success")
.unwrap_or_else(|| panic!("no removal recorded: {response}"));
assert_eq!(removed["target"]["name"], "SS-1", "{removed}");
assert!(
events
.iter()
.all(|e| e["actor"]["name"] == "[email protected]"),
"{response}"
);
assert!(
response.to_string().contains("Old scanners on the LAN"),
"the note isn't in the record: {response}"
);
// Cleared for the tests that follow
call(
&admin,
"inbuxa:SecurityAcceptance/set",
json!({"destroy": [relay_id]}),
)
.await;
}
#[ignore]
#[tokio::test(flavor = "multi_thread")]
pub async fn security_acceptance_tests() {
let mut test = TestServerBuilder::new("security_acceptance_tests")
.await
.with_default_listeners()
.await
.build()
.await;
let admin = test.create_admin_account("[email protected]").await;
test.insert_account(admin);
self::test(&mut test).await;
if test.is_reset() {
test.temp_dir.delete();
}
}