Files
jcoffey-dev 2358f4814c
ci / fork-checks (pull_request) Skipped
ci / build (pull_request) Skipped
github/ci (branch) GitHub Actions
ci / github (pull_request) Successful in 9m5s
Cluster: one node at a time does the singleton roles' work (#437)
Every node with taskScheduler queued its own copy of each scheduled
maintenance task, and every node with metricsCalculate evaluated alerts
and sent their emails, so each role had to be on exactly one node, and
losing that node paused the work until it came back.

Each role now has a lease in the store (Xs, Xm in `_`), taken with a
compare-and-swap like the store layout lease. Every node with the role
tries for it each minute; the holder renews it and does the work. A
lease lasts 3 minutes and its holder stops acting 15 seconds before it
runs out, so another node with the role takes over within about 4
minutes of the holder stopping. A clean stop gives the leases up, so a
restart or a roll hands over at the next round. A node without a
cluster coordinator needs no lease: the role alone decides, as before.

The firing alerts are stored too (Xa), so a node that takes over alerts
doesn't send the ones still firing again, and it sets the cluster-wide
gauges before its first evaluation. The daily account and domain counts
no longer wait a day on a node that just took the role over.

New events cluster.singleton-lease-taken and -lost (665, 666), also in
the served schema. `X` is registered in the `_` owners table; the spec's
copy of that table gains the N (passkeys) and O (listener status) rows
it was missing.
2026-11-21 01:07:14 -07:00
..
2026-10-30 08:39:54 -07:00