Every node records DMARC and TLS results for the aggregate reports
The report scheduler dropped DMARC and TLS events on a node whose role lacks outboundMta (upstream never started it there, so they sat in a channel nobody read). Mail received on a front node therefore never reached an aggregate report, which is meant to cover all of a domain's inbound mail, whichever node received it. In rehearsal, five messages received on port 25 on a front node were missing from every report. - The report scheduler records on every node. Recording is a store write the nodes already share, so it needs nothing from the outbound MTA. Building and sending a report (the DmarcReport and TlsReport tasks) stay with outboundMta nodes, as the task manager already enforces. - More nodes now append to one report at once. Appends already guard the report's versioned primary key; a write that loses now retries up to ten times after a short random pause, not three times at once. - The node sending a report deletes it only if it is unchanged since it was read, and reads it again otherwise, so a record another node appends meanwhile goes out with the report instead of being deleted unsent. Test: cluster::front_reports (PostgreSQL and MySQL). A front node's results appear in the report the MTA node sends, alongside eight appended at once from both nodes, and the front node never runs the report task. It fails on main: the front node's results are never recorded.
This commit is contained in:
@@ -2,9 +2,12 @@
|
||||
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
|
||||
*
|
||||
* SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-SEL
|
||||
*
|
||||
* Modified by Coffey Labs in 2026 for INBUXA.
|
||||
*/
|
||||
|
||||
use super::AggregateTimestamp;
|
||||
use super::shared::{MAX_WRITE_RETRIES, Revisioned, write_retry_pause};
|
||||
use crate::{
|
||||
core::Session,
|
||||
queue::RecipientDomain,
|
||||
@@ -349,29 +352,43 @@ impl DmarcReporting for Server {
|
||||
let object_id = ObjectType::DmarcInternalReport.to_id();
|
||||
let key = ValueClass::Registry(RegistryClass::Item { object_id, item_id });
|
||||
|
||||
let Some(report) = self
|
||||
.store()
|
||||
.get_value::<DmarcInternalReport>(ValueKey::from(key.clone()))
|
||||
.await
|
||||
.caused_by(trc::location!())?
|
||||
else {
|
||||
return Ok(());
|
||||
};
|
||||
// Delete report. inbuxa: only the version read here, so a record
|
||||
// another node appends meanwhile is sent with it rather than lost
|
||||
let mut attempt = 0;
|
||||
let report = loop {
|
||||
let Some(Revisioned {
|
||||
revision,
|
||||
value: report,
|
||||
}) = self
|
||||
.store()
|
||||
.get_value::<Revisioned<DmarcInternalReport>>(ValueKey::from(key.clone()))
|
||||
.await
|
||||
.caused_by(trc::location!())?
|
||||
else {
|
||||
return Ok(());
|
||||
};
|
||||
|
||||
// Delete report
|
||||
let mut batch = BatchBuilder::new();
|
||||
batch.clear(key).clear(RegistryClass::PrimaryKey {
|
||||
object_id: object_id.into(),
|
||||
index_id: Property::Domain.to_id(),
|
||||
key: KeySerializer::new(report.domain.len() + U64_LEN)
|
||||
.write(&report.domain)
|
||||
.write(report.policy_identifier)
|
||||
.finalize(),
|
||||
});
|
||||
self.store()
|
||||
.write(batch.build_all())
|
||||
.await
|
||||
.caused_by(trc::location!())?;
|
||||
let mut batch = BatchBuilder::new();
|
||||
batch
|
||||
.assert_value(key.clone(), AssertValue::Hash(revision))
|
||||
.clear(key.clone())
|
||||
.clear(RegistryClass::PrimaryKey {
|
||||
object_id: object_id.into(),
|
||||
index_id: Property::Domain.to_id(),
|
||||
key: KeySerializer::new(report.domain.len() + U64_LEN)
|
||||
.write(&report.domain)
|
||||
.write(report.policy_identifier)
|
||||
.finalize(),
|
||||
});
|
||||
match self.store().write(batch.build_all()).await {
|
||||
Ok(_) => break report,
|
||||
Err(err) if err.is_assertion_failure() && attempt < MAX_WRITE_RETRIES => {
|
||||
attempt += 1;
|
||||
write_retry_pause(attempt).await;
|
||||
}
|
||||
Err(err) => return Err(err.caused_by(trc::location!())),
|
||||
}
|
||||
};
|
||||
|
||||
let span_id = self.inner.data.span_id_gen.generate();
|
||||
let event_from = report.report.date_range_begin.timestamp() as u64;
|
||||
@@ -676,8 +693,11 @@ impl DmarcReporting for Server {
|
||||
break;
|
||||
}
|
||||
Err(err) => {
|
||||
if err.is_assertion_failure() && rety_count < 3 {
|
||||
// inbuxa: another node appended first; try again
|
||||
// after a short pause
|
||||
if err.is_assertion_failure() && rety_count < MAX_WRITE_RETRIES {
|
||||
rety_count += 1;
|
||||
write_retry_pause(rety_count).await;
|
||||
continue;
|
||||
}
|
||||
trc::error!(
|
||||
|
||||
Reference in New Issue
Block a user