Re-queue DKIM rotation and certificate renewal a domain is missing

jcoffey-dev commented

2026-10-16 15:09:21 +00:00

Owner

DKIM rotation and certificate renewal are queued only when a domain is saved into automatic mode, and each run queues the next. A domain whose first run never happened, or whose chain broke, kept its keys forever with nothing scheduled. Found on a production server: nine domains set to automatic DKIM had no DkimManagement task, and their keys had no nextTransitionAt, seven of them still on year-old keys. They were rotated by hand on 2026-10-16. This makes sure it can't happen again.

What changes

The task scheduler checks daily, and five minutes after start, that every domain with automatic DKIM has a DkimManagement task and every domain with automatic certificates an AcmeRenewal task, and queues any that are missing. Only the node with the task scheduler role runs it.
A task counts as present in any state, failed included, so a domain whose task keeps failing isn't queued again every day; the failure stays visible in Tasks.
DKIM: only when DNS publishes DKIM, or when the domain has no keys at all. Without publishing, a run neither rotates nor queues a follow-up, so queuing it daily would do nothing.
ACME: a renewal that finds a still-valid certificate reschedules itself for when it falls due (#93), so queuing one orders nothing early and can't run into Let's Encrypt rate limits.
Each re-queued task logs task-manager.task-queued (Info) with the domain and "Re-queued a missing domain task".
Domains are found with a registry query, not list(), which returns no Domain objects (seen in the test: list found 0 where the query found 2).

Tests (RocksDB, local): automation::automation_tests passes (ACME, DKIM, DNS, RFC 2136). Its new orphaned-key part drops a domain's task and its keys' schedule, then checks the domain gets a task, the same keys are scheduled again (no rotation), and a second check queues nothing. It failed before the list() fix. Unit tests for the three conditions. cargo build -p inbuxa --locked, cargo test --workspace --locked --no-run and the fork checks pass.

No new UI strings. No docs change.

DKIM rotation and certificate renewal are queued only when a domain is saved into automatic mode, and each run queues the next. A domain whose first run never happened, or whose chain broke, kept its keys forever with nothing scheduled. Found on a production server: nine domains set to automatic DKIM had no `DkimManagement` task, and their keys had no `nextTransitionAt`, seven of them still on year-old keys. They were rotated by hand on 2026-10-16. This makes sure it can't happen again. **What changes** - The task scheduler checks daily, and five minutes after start, that every domain with automatic DKIM has a `DkimManagement` task and every domain with automatic certificates an `AcmeRenewal` task, and queues any that are missing. Only the node with the task scheduler role runs it. - A task counts as present in any state, failed included, so a domain whose task keeps failing isn't queued again every day; the failure stays visible in Tasks. - DKIM: only when DNS publishes DKIM, or when the domain has no keys at all. Without publishing, a run neither rotates nor queues a follow-up, so queuing it daily would do nothing. - ACME: a renewal that finds a still-valid certificate reschedules itself for when it falls due (#93), so queuing one orders nothing early and can't run into Let's Encrypt rate limits. - Each re-queued task logs `task-manager.task-queued` (Info) with the domain and "Re-queued a missing domain task". - Domains are found with a registry query, not `list()`, which returns no `Domain` objects (seen in the test: `list` found 0 where the query found 2). **Tests** (RocksDB, local): `automation::automation_tests` passes (ACME, DKIM, DNS, RFC 2136). Its new orphaned-key part drops a domain's task and its keys' schedule, then checks the domain gets a task, the same keys are scheduled again (no rotation), and a second check queues nothing. It failed before the `list()` fix. Unit tests for the three conditions. `cargo build -p inbuxa --locked`, `cargo test --workspace --locked --no-run` and the fork checks pass. No new UI strings. No docs change.

jcoffey-dev added 1 commit 2026-10-16 15:09:21 +00:00

Re-queue DKIM rotation and certificate renewal a domain is missing

ci / fork-checks (pull_request) Skipped

Details

ci / build (pull_request) Skipped

Details

github/ci (branch) GitHub Actions

Details

ci / github (pull_request) Successful in 25m6s

Details

393eaa61ca

DKIM rotation and ACME renewal are queued only when a domain is saved
into automatic mode, and each run queues the next. A domain whose first
run never happened, or whose chain broke, kept its keys forever with
nothing scheduled: on one server nine automatic domains had no
DkimManagement task and keys with no nextTransitionAt.

- The task scheduler checks daily, and five minutes after start, that
  every domain with automatic DKIM has a DkimManagement task and every
  domain with automatic certificates an AcmeRenewal task, and queues
  any that are missing. Only the node with the task scheduler role
  runs it.
- A task counts as present in any state, failed included, so a domain
  whose task keeps failing isn't queued again every day.
- DKIM: only when DNS publishes DKIM, or when the domain has no keys at
  all. Without publishing, a run neither rotates nor queues a follow-up.
- ACME: a renewal that finds a still-valid certificate reschedules
  itself for when it falls due (#93), so queuing one orders nothing
  early.
- Each re-queued task is logged as task-manager.task-queued, with the
  domain and "Re-queued a missing domain task".

Tests: automation_tests' new orphaned-key part drops a domain's task
and its keys' schedule, then checks the domain gets a task, the same
keys get a schedule again, and a second check queues nothing.

jcoffey-dev merged commit 9f72a5908c into main

2026-10-16 15:34:52 +00:00

jcoffey-dev deleted branch self-heal-domain-tasks

2026-10-16 15:34:53 +00:00

jcoffey-dev referenced this pull request

2026-10-16 15:52:13 +00:00

Release 2026.10.16 #202

jcoffey-dev referenced this issue from a commit

2026-10-18 21:21:57 +00:00

Leave the startup domain check out of test builds

jcoffey-dev referenced this pull request

2026-10-18 21:22:19 +00:00