Store check: find blobs that are missing, damaged or unlinked (#322) #324

Merged
jcoffey-dev merged 1 commits from feat/322-verify-store into main 2026-11-09 02:10:35 +00:00
1 Commits
Author SHA1 Message Date
jcoffey-dev dfdc47ae9e Store check: find blobs that are missing, damaged or unlinked (#322)
ci / fork-checks (pull_request) Skipped
ci / build (pull_request) Skipped
github/ci (branch) GitHub Actions
ci / github (pull_request) Successful in 8m5s
An administrator can now ask the server to check its blobs. A quick
check confirms that every blob the data holds a link to is in the blob
store. A full check also reads each one back and compares its BLAKE3
hash with its id. Mail is checked from the message side too, so a
message whose metadata names a blob that nothing links to any more is
reported. That is the case from forum topic 238. The check changes
nothing.

It is an inbuxa:StoreCheck under urn:inbuxa:jmap. Creating one starts
it in the background on the node that was reached. It runs one slice
of the blob index at a time, throttled (1,000 blobs/s and 20 MB/s by
default, 0 for no limit), and saves its progress every 10 seconds.
Only one check runs at a time across the server. A check that goes
quiet for five minutes, or whose node restarts, reads as interrupted.
The first 1,000 problems are kept with the account, object, blob id
and what was wrong, and each problem is also logged as
store.data-corruption. An update with action "stop" stops a check. A
finished check can be destroyed. The latest 20 checks are kept.

The new permission sysStoreCheck (695) is in the superuser and admin
grants. A tenant administrator is refused, because a check reads the
whole server. Starting and stopping a check are audited.

Spec: docs/spec/features/store-check.md. Test: system::store_check.
2026-11-08 19:00:44 -07:00