x:Metric: say which node wrote each sample
Each node stores histograms as running totals since it started. A sample didn't say which node wrote it (the node was only in the id's low bits), so a reader couldn't diff totals per node, and the console diffed across nodes: on the three-node production cluster the delivery attempt time read 14.7 s over the last hour against 0.7 s from the nodes' own figures. x:Metric/get now returns nodeId alongside timestamp, both from the id. The telemetry suite checks every sample carries it.
This commit is contained in:
@@ -104,6 +104,12 @@ impl StoredMetric {
|
||||
pub fn timestamp(&self) -> u64 {
|
||||
SnowflakeIdGenerator::to_timestamp(self.id)
|
||||
}
|
||||
|
||||
/// The node that wrote the sample. Histogram totals are per node, so a
|
||||
/// reader diffs them per node.
|
||||
pub fn node_id(&self) -> u64 {
|
||||
SnowflakeIdGenerator::to_node_id(self.id)
|
||||
}
|
||||
}
|
||||
|
||||
/// What the node wrote last, so counters and histograms are written as
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
//! `x:Metric/get` and `/query` over the stored history (monitoring spec,
|
||||
//! "Interfaces"). Samples are server-level (MON-31) and read-only (MON-32).
|
||||
//! A sample's `timestamp` comes from its id.
|
||||
//! A sample's `timestamp` and `nodeId` come from its id.
|
||||
|
||||
use crate::{
|
||||
api::query::QueryResponseBuilder,
|
||||
@@ -54,12 +54,16 @@ fn metric_type(metric: &Metric) -> MetricType {
|
||||
|
||||
fn to_value(sample: StoredMetric) -> JmapValue<'static> {
|
||||
let timestamp = sample.timestamp();
|
||||
let node_id = sample.node_id();
|
||||
let mut value = sample.metric.into_value();
|
||||
if let JmapValue::Object(obj) = &mut value {
|
||||
obj.insert_unchecked(
|
||||
Property::Timestamp,
|
||||
JmapValue::Str(UTCDateTime::from_timestamp(timestamp as i64).to_string().into()),
|
||||
);
|
||||
// Histograms are running totals per node; without this a reader
|
||||
// diffs one node's total against another's
|
||||
obj.insert_unchecked(Property::NodeId, JmapValue::Number(node_id.into()));
|
||||
}
|
||||
value
|
||||
}
|
||||
|
||||
@@ -108,6 +108,12 @@ impl SnowflakeIdGenerator {
|
||||
(id >> (SEQUENCE_LEN + NODE_ID_LEN)) / 1000 + DEFAULT_EPOCH
|
||||
}
|
||||
|
||||
// inbuxa: the node that made the id, so per-node history (metric
|
||||
// totals) can be told apart
|
||||
pub fn to_node_id(id: u64) -> u64 {
|
||||
id & NODE_ID_MASK
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
pub fn past_id(&self, period: Duration) -> Option<u64> {
|
||||
self.epoch.elapsed().ok().map(|elapsed| {
|
||||
|
||||
Reference in New Issue
Block a user