Service status: one failed copy of a redundant unit or container marks the whole service down #52

Closed
opened 2026-11-16 05:58:34 +00:00 by jcoffey-dev · 1 comment
Owner

A topology service's status is the worst of its parts. For a service matched by a pattern across many hosts (units: ["*/redis-server"], containers: ["*/app-*"]), one copy failing makes the whole service critical: the service map shows it down and the reports count it as downtime.

On the demo, one exited app-04 container among many hosts' app containers takes "App servers" to 89% for the month, and one failed redis-server among the cache hosts takes Redis to 89%. A real team's availability report would show outages that never happened.

Do: when a service has several copies of the same unit or container (the same name on different hosts) and some are still fine, the failed ones make the service degraded (warning), not down. When every copy has failed, or there's only one, it's down as now. The service's word says which: "1 of 6 redis-server down". Checks, Cloudflare and needs are judged as they are now. Document it in formats.md (topology.yaml).

A topology service's status is the worst of its parts. For a service matched by a pattern across many hosts (`units: ["*/redis-server"]`, `containers: ["*/app-*"]`), one copy failing makes the whole service critical: the service map shows it down and the reports count it as downtime. On the demo, one exited app-04 container among many hosts' app containers takes "App servers" to 89% for the month, and one failed redis-server among the cache hosts takes Redis to 89%. A real team's availability report would show outages that never happened. **Do:** when a service has several copies of the same unit or container (the same name on different hosts) and some are still fine, the failed ones make the service degraded (warning), not down. When every copy has failed, or there's only one, it's down as now. The service's word says which: "1 of 6 redis-server down". Checks, Cloudflare and needs are judged as they are now. Document it in formats.md (topology.yaml).
jcoffey-dev added the bug
priority
p1
labels 2026-11-16 05:58:34 +00:00
jcoffey-dev self-assigned this 2026-11-16 05:58:34 +00:00
Author
Owner

Fixed in #53. A unit or container a service has on several hosts is now redundant: a copy down while others still run makes the service degraded, with a word saying how many ("1 of 6 redis-server.service down"). Every copy down, or only one, is down as before. Each copy's own alert is unchanged, so availability reports stop counting outages that never happened. In the next release.

Fixed in #53. A unit or container a service has on several hosts is now redundant: a copy down while others still run makes the service degraded, with a word saying how many ("1 of 6 redis-server.service down"). Every copy down, or only one, is down as before. Each copy's own alert is unchanged, so availability reports stop counting outages that never happened. In the next release.
Sign in to join this conversation.