Dashboard: diff histogram totals per node #73
Assignees
jcoffey-dev (John Coffey)
Clear assignees
jcoffey-dev (John Coffey)
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: inbuxa/inbuxa-admin#73
Reference in new issue
Block a user
Each node stores its histograms as running totals since it started.
deltaHistogramslined up every node's samples by time and diffed them in one list, so in a cluster it subtracted one node's total from another's. Every timing on the dashboard (ingest, index, DNS, delivery, store read/write) came out wrong.On production, the command center showed 1 stage running slow: delivery attempt time read 14.7 s over the last hour when the nodes' own figures give 0.7 s (6 h: 8.5 s vs 1.0 s; 24 h: 13.3 s vs 5.8 s). These were recomputed from the stored
xsamples.nodeId(inbuxa-server PR,x:Metric), and totals are diffed per metric and node.nodeId(older servers, single node), it behaves as before.helpers.test.ts: per-node diffs with interleaved nodes, a restart, and pass-through for counters/gauges.No new strings.
Needs the server change released and rolled out before it has any effect on production.
Each node stores its histograms as running totals since it started. deltaHistograms lined up every node's samples by time and diffed them in one list, so in a cluster it subtracted one node's total from another's. The timing averages came out wrong: on production the delivery attempt time read 14.7 s over the last hour ("1 stage running slow") when the nodes' own figures give 0.7 s. Samples now carry nodeId (inbuxa-server, x:Metric), and totals are diffed per metric and node. Without nodeId (older servers, single node) it works as before.