Hotfix 2026.9.24.4: report reschedules keep the task queue readable

jcoffey-dev commented

2026-09-25 01:36:29 +00:00

Owner

Hotfix release 2026.9.24.4: the production release 2026.9.24.3 plus the report reschedule fix, and nothing else from main.

Commits on top of v2026.9.24.3 (7735780):

Publish: accept tags on release/ branches* — cherry-picked from #48, so the v2026.9.24.4 tag can publish from this branch (a tag runs the workflow from its own commit).
Report reschedules keep the task queue readable — cherry-picked from #46. One conflict in task_manager/manager.rs, resolved by taking #46's row decoding (unreadable queue rows are logged, skipped and repaired instead of ending the scan) and keeping this release's inline role match; #46's task_locks.is_held check was dropped because the lease work it relies on (#35/#41) isn't in this release.
Release 2026.9.24.4 — brand_version! bump.

Why: setting deliverAt on an internal DMARC/TLS report writes a task-queue row with the report's object type instead of the task type, and the scan error then stops every task due after it on every node. Production runs 2026.9.24.3, which has the bug; with this release a row already written that way is repaired when it comes due.

Checked locally on this branch: cargo check -p inbuxa builds; smtp::reporting::reschedule (RocksDB) — see the PR conversation for the result.

After merge: tag v2026.9.24.4 on release/2026.9.24.4. main's next release needs a version above 2026.9.24.4.

Hotfix release 2026.9.24.4: the production release 2026.9.24.3 plus the report reschedule fix, and nothing else from main. Commits on top of `v2026.9.24.3` (7735780): 1. **Publish: accept tags on release/* branches** — cherry-picked from #48, so the `v2026.9.24.4` tag can publish from this branch (a tag runs the workflow from its own commit). 2. **Report reschedules keep the task queue readable** — cherry-picked from #46. One conflict in `task_manager/manager.rs`, resolved by taking #46's row decoding (unreadable queue rows are logged, skipped and repaired instead of ending the scan) and keeping this release's inline role `match`; #46's `task_locks.is_held` check was dropped because the lease work it relies on (#35/#41) isn't in this release. 3. **Release 2026.9.24.4** — `brand_version!` bump. Why: setting `deliverAt` on an internal DMARC/TLS report writes a task-queue row with the report's object type instead of the task type, and the scan error then stops every task due after it on every node. Production runs 2026.9.24.3, which has the bug; with this release a row already written that way is repaired when it comes due. Checked locally on this branch: `cargo check -p inbuxa` builds; `smtp::reporting::reschedule` (RocksDB) — see the PR conversation for the result. After merge: tag `v2026.9.24.4` on `release/2026.9.24.4`. main's next release needs a version above 2026.9.24.4.

jcoffey-dev added 3 commits 2026-09-25 01:36:30 +00:00

Publish: accept tags on release/* branches for hotfix releases 4e6c8b916e

The publish workflow only built a tag whose commit is on main. That keeps
every image tied to reviewed code, but it means production can only get a
fix together with everything that has landed on main since its release.

A tag on a release/* branch is now accepted too. A hotfix branch starts at
an earlier release tag, takes fixes through pull requests into it (so the
code is still reviewed and CI-tested before it is tagged), bumps
brand_version! and is tagged there. The tag must still equal
v<brand_version!>, and the step prints which branch it was found on.

A tag runs the workflow file from its own commit, so a hotfix branch that
starts before this change needs this commit cherry-picked onto it before
its tag is pushed.

(cherry picked from commit 4b85113262)

Report reschedules keep the task queue readable 0b4aa9c084

Setting deliverAt on an internal DMARC or TLS report wrote the new task
queue row with the report's object type (0x21, 0x6e) instead of the task
type (7, 8), and left the task row at its old due. The task manager's scan
failed on that row with store.data-corruption ("Failed to iterate over task
queue"), and because the error ended the whole scan, every task due after
the row stopped running on every node.

- reschedule_ops writes the new queue row through schedule_task_with_id, so
  it carries the task type and the task row gets the new due. It removes
  the row the task is actually queued under (the task's due, which differs
  from deliverAt once the task has been retried) and any row an earlier
  reschedule left at deliverAt.
- x:DmarcInternalReport/set and x:TlsInternalReport/set lock the report's
  task while they move it, as x:Task/set does, refuse while the report is
  being sent, release the locks however the request ends, and wake the task
  manager.
- The task manager logs a queue row it can't read (id, due, key, value) and
  skips it instead of ending the scan. It then repairs the row from its task:
  the row is rewritten with the task's type, and a row with no task behind
  it is removed. A row holding a report's object type for a report task is
  what the old reschedule wrote: the task is moved to that row's time, as
  the reschedule intended, and its old queue row is removed. Stores that
  already hold such a row recover on their own once it comes due.
- x:Task/query with a type filter skips an unreadable row instead of
  failing.

Test: smtp::reporting::reschedule (RocksDB and PostgreSQL). It fails on
main: x:Task/get shows the old due, and with that check removed, neither
report nor a later task ever runs.

(cherry picked from commit 1a7859a8cc)

Release 2026.9.24.4

ci / fork-checks (pull_request) Failing after 29s

Details

ci / build (pull_request) Canceled after 3m15s

Details

e2ab26ad19

jcoffey-dev added 1 commit 2026-09-25 01:39:47 +00:00

Mark task_manager/manager.rs as modified for the AGPL notice

ci / fork-checks (pull_request) Successful in 22s

Details

ci / build (pull_request) Successful in 3m37s

Details

81ec917d74

jcoffey-dev merged commit f398d95062 into release/2026.9.24.4

2026-09-25 01:43:58 +00:00

jcoffey-dev deleted branch hotfix/2026.9.24.4

2026-09-25 01:43:58 +00:00

Hotfix 2026.9.24.4: report reschedules keep the task queue readable #49