Files
jcoffey-dev 4f2187b805
ci / fork-checks (pull_request) Skipped
ci / build (pull_request) Skipped
github/ci (branch) GitHub Actions
ci / github (pull_request) Successful in 14m26s
Backups for Kubernetes: export to and import from S3, store checks as a command (K-C11)
For the chart's scheduled backup and store checks (inbuxa-charts#11), and
three bugs the lab's restore test found on the way.

Added
- `--export s3://bucket/prefix` streams the export into multipart
  uploads, in a new folder named for the time (20261119T204500Z), and
  writes export.json last, naming the files: an export without it didn't
  finish. Nothing the size of the store lands on local disk. Parts start
  at 16 MiB and double every thousand up to 512 MiB, so one file can be
  nearly 3 TiB; one part is held at a time.
- `--import s3://bucket/prefix/<name>` reads one back, streamed; from
  `s3://bucket/prefix`, the newest finished one.
- The bucket from INBUXA_EXPORT_S3_ENDPOINT (S3-compatible, path-style;
  AWS without it), INBUXA_EXPORT_S3_REGION, and the keys from
  INBUXA_EXPORT_S3_ACCESS_KEY_FILE and INBUXA_EXPORT_S3_SECRET_KEY_FILE
  (the AWS defaults without them).
- `--store-check quick|full` runs a store check to its end and exits: 0
  nothing wrong, 3 problems found, 2 another check running, 1 failed. It's
  saved with the console's other checks, under a node id no server holds.

Fixed
- A live export could name blobs it didn't have: the store's areas went
  out in a set's order, so the blobs could go first and a message stored
  after them pointed at a blob the export never got. Blobs go last now.
- An import into a fresh store refused to run: every start writes the
  fork's subspace (where the blob store is, permission grants, the log
  files' setting), the import's own start too, and the import took that
  for data. Since the blob store change (#333): 27.1-rc.16 and
  2026.11.9.1. The fork's subspace is no longer counted; its records are
  keyed values an import sets over. The store test now imports over a
  start-up record, and fails without this.
- An import wrote the blobs into the data store, the empty store's blob
  store, even when the imported settings name another (S3, say): the
  restored server then found none of them. Blobs are imported last, into
  the blob store the imported x:BlobStore names.
- The S3 export and import could stop for good on a pod limited to one CPU
  (one runtime worker): a reader or sender waiting on the transfer held
  the worker the transfer needed. A wait hands its worker over first.

Tested: inbuxa-k8s-lab backup-test.sh, on the kind lab (Postgres, Garage,
NATS, five server pods): mail for an account, the chart's backup CronJob,
the database dropped and the blob bucket emptied, the import, and every
mailbox's count and message hashes the same after; quick and full store
checks clean. store tests pass on SQLite and RocksDB; unit tests pass.
2026-11-19 13:47:43 -07:00
..