x:Metric: say which node wrote each sample
ci / fork-checks (pull_request) Skipped
ci / build (pull_request) Skipped
github/ci (branch) GitHub Actions
ci / github (pull_request) Successful in 7m5s

Each node stores histograms as running totals since it started. A sample
didn't say which node wrote it (the node was only in the id's low bits),
so a reader couldn't diff totals per node, and the console diffed across
nodes: on the three-node production cluster the delivery attempt time
read 14.7 s over the last hour against 0.7 s from the nodes' own figures.

x:Metric/get now returns nodeId alongside timestamp, both from the id.
The telemetry suite checks every sample carries it.
This commit is contained in:
2026-09-30 11:43:13 -07:00
parent 69ef48239a
commit 20abf69d31
4 changed files with 28 additions and 1 deletions
@@ -104,6 +104,12 @@ impl StoredMetric {
pub fn timestamp(&self) -> u64 {
SnowflakeIdGenerator::to_timestamp(self.id)
}
/// The node that wrote the sample. Histogram totals are per node, so a
/// reader diffs them per node.
pub fn node_id(&self) -> u64 {
SnowflakeIdGenerator::to_node_id(self.id)
}
}
/// What the node wrote last, so counters and histograms are written as
+5 -1
View File
@@ -6,7 +6,7 @@
//! `x:Metric/get` and `/query` over the stored history (monitoring spec,
//! "Interfaces"). Samples are server-level (MON-31) and read-only (MON-32).
//! A sample's `timestamp` comes from its id.
//! A sample's `timestamp` and `nodeId` come from its id.
use crate::{
api::query::QueryResponseBuilder,
@@ -54,12 +54,16 @@ fn metric_type(metric: &Metric) -> MetricType {
fn to_value(sample: StoredMetric) -> JmapValue<'static> {
let timestamp = sample.timestamp();
let node_id = sample.node_id();
let mut value = sample.metric.into_value();
if let JmapValue::Object(obj) = &mut value {
obj.insert_unchecked(
Property::Timestamp,
JmapValue::Str(UTCDateTime::from_timestamp(timestamp as i64).to_string().into()),
);
// Histograms are running totals per node; without this a reader
// diffs one node's total against another's
obj.insert_unchecked(Property::NodeId, JmapValue::Number(node_id.into()));
}
value
}
+6
View File
@@ -108,6 +108,12 @@ impl SnowflakeIdGenerator {
(id >> (SEQUENCE_LEN + NODE_ID_LEN)) / 1000 + DEFAULT_EPOCH
}
// inbuxa: the node that made the id, so per-node history (metric
// totals) can be told apart
pub fn to_node_id(id: u64) -> u64 {
id & NODE_ID_MASK
}
#[inline(always)]
pub fn past_id(&self, period: Duration) -> Option<u64> {
self.epoch.elapsed().ok().map(|elapsed| {