The simulator made every problem it knew at once and kept it forever:
89 open alerts in the first minute, nothing ever resolved. Now
scripts/fleet/scenario.go holds a plan:
- three stories with a cause and its effects: fa-prod-01 lost a drive;
a PostgreSQL replica falls behind, HAProxy takes it out and the shop's
checkout fails; the legacy datacenter drops out of Akamai GTM and its
CP code fails;
- a few lasting warnings anyone would have;
- problems that come and go (spells counted from the Unix epoch, so a
restarted demo carries on where the clock is): a Redis unit, an
exiting container, a flapping check and block listing, ESXi and AHV
patching, a Proxmox HA error, Pure replication paths, a blade
evacuating, an IBM i inquiry and an ended subsystem, vault-edge losing
a server, a late or shrunken backup, SQL Server Agent stopping,
win-fs01 crashing, an Akamai activation that drags.
Cluster readers (Pure, vSphere, Nutanix, Proxmox, IBM i, the HMC, the
Kubernetes clusters) flap records through fakeCluster.flap and flapSet;
the HTTP stand-ins and host reports ask the spells directly. Gone: the
fan at 0 RPM, the inaccessible VM, the crashed libvirt VM, the always-
crashing pods, the ended and soon-ending Postgres certificates, a dozen
expiring Vault and Akamai certificates, the stuck staging activation.
A backup's folder is now named after the day the copy was made, so a
late one no longer says it's from today (#43).
Closes#35, closes#43.