Changing x:BlobStore moved nothing: every blob written under the old setting stayed behind, and a message whose blob was left there failed to read with nothing to say why (forum t/238). Now:
Noticed at start-up (BM-1, BM-2). Each node compares where its blob store is (kind and location, never credentials or tuning) with where the store last said it was. When they differ, the first node records an inbuxa:BlobStoreChange (from, to, when, which node) and keeps the old store's settings, and the start-up log warns, naming both places. Going back to an earlier store (A to B, then back to A) marks the A-to-B change reverted (BM-4).
Read-through (BM-3). While a change is unresolved, a blob the current store lacks is looked for in the old store. If it's there, it's returned and written to the current store under the same key, and store.blob-read-through (a new event) is logged. Writes and deletes go only to the current store, and nothing is ever deleted from the old one. Stefan's message would have kept working with no repair.
Copy (BM-5, BM-6).{"action": "copy"} walks the blob index (256 slices, throttled like the store check) and copies every committed blob something holds, plus the spam classifier's two fixed-name files, from the old store as stored. Blobs held by a document, a queued message or a kept deleted item are checked against their hash first, and a damaged one isn't copied. The copy ends copied (resolved: the old store's settings are dropped and every node stops reading through within a minute) or incomplete, with the first 1,000 problems kept. Stop, interrupted and failed work as in the store check.
Dismiss (BM-7). Destroy forgets the change and the settings it kept. Nothing is deleted from either store.
A store snapshot is refused while a change is unresolved, since it would miss blobs only the old store has (BM-8).
New permission sysBlobStoreChange (697), for administrators only and granted once to existing admin roles. Copy, stop and dismiss are audited, and get never returns the old store's settings (BM-9).
Design choices to look at
Read-through changes the blob read path. It only does anything on a miss, and only while a change is unresolved. With no unresolved change, the server uses its blob store unwrapped, as before.
The old store's settings, credentials included, are kept in the fork subspace (W) until the change is resolved or dismissed. They're catalogued as the blob-store-change-settings source.
A sharded set counts as one place: the set moves blobs between its members itself (ST-16 to ST-22).
The copy runs after the change, not before. Reading through covers the time in between, so no copy-ahead mode was needed (listed under "Not yet").
Tests
store: read-through and keep, absent in both, writes and deletes touch only the new store, nothing read through once resolved; where a store is (BM-1).
inbuxa-features: a move and a move back, recorded once across nodes, the revert, one copy at a time, a quiet copy interrupted, the resolved state.
system::blob_store_change, against a real server: the t/238 case end to end, covering read-through, refusals, a slow copy stopped (and not dismissable while it runs), an incomplete copy reporting the damaged and the missing blob with their message, a second copy resolving the change, and dismissal.
Fork checks run locally: name, notice, context, privacy, expr-schema --check, and the tools tests.
Not yet
A console page, and a note on the blob store form that existing blobs stay where they are (inbuxa-admin).
An inbuxa-cli command, and a docs page.
Copying ahead of a change, and cleaning up an old store once its change is resolved.
Fixes #333. Spec: `docs/spec/features/blob-store-change.md`.
Changing `x:BlobStore` moved nothing: every blob written under the old setting stayed behind, and a message whose blob was left there failed to read with nothing to say why (forum t/238). Now:
- **Noticed at start-up (BM-1, BM-2).** Each node compares where its blob store is (kind and location, never credentials or tuning) with where the store last said it was. When they differ, the first node records an `inbuxa:BlobStoreChange` (from, to, when, which node) and keeps the old store's settings, and the start-up log warns, naming both places. Going back to an earlier store (A to B, then back to A) marks the A-to-B change `reverted` (BM-4).
- **Read-through (BM-3).** While a change is unresolved, a blob the current store lacks is looked for in the old store. If it's there, it's returned and written to the current store under the same key, and `store.blob-read-through` (a new event) is logged. Writes and deletes go only to the current store, and nothing is ever deleted from the old one. Stefan's message would have kept working with no repair.
- **Copy (BM-5, BM-6).** `{"action": "copy"}` walks the blob index (256 slices, throttled like the store check) and copies every committed blob something holds, plus the spam classifier's two fixed-name files, from the old store as stored. Blobs held by a document, a queued message or a kept deleted item are checked against their hash first, and a damaged one isn't copied. The copy ends `copied` (resolved: the old store's settings are dropped and every node stops reading through within a minute) or `incomplete`, with the first 1,000 problems kept. Stop, interrupted and failed work as in the store check.
- **Dismiss (BM-7).** Destroy forgets the change and the settings it kept. Nothing is deleted from either store.
- A store snapshot is refused while a change is unresolved, since it would miss blobs only the old store has (BM-8).
- New permission `sysBlobStoreChange` (697), for administrators only and granted once to existing admin roles. Copy, stop and dismiss are audited, and `get` never returns the old store's settings (BM-9).
### Design choices to look at
- **Read-through changes the blob read path.** It only does anything on a miss, and only while a change is unresolved. With no unresolved change, the server uses its blob store unwrapped, as before.
- The old store's settings, credentials included, are kept in the fork subspace (`W`) until the change is resolved or dismissed. They're catalogued as the `blob-store-change-settings` source.
- A sharded set counts as one place: the set moves blobs between its members itself (ST-16 to ST-22).
- The copy runs after the change, not before. Reading through covers the time in between, so no copy-ahead mode was needed (listed under "Not yet").
### Tests
- `store`: read-through and keep, absent in both, writes and deletes touch only the new store, nothing read through once resolved; where a store is (BM-1).
- `inbuxa-features`: a move and a move back, recorded once across nodes, the revert, one copy at a time, a quiet copy interrupted, the resolved state.
- `system::blob_store_change`, against a real server: the t/238 case end to end, covering read-through, refusals, a slow copy stopped (and not dismissable while it runs), an incomplete copy reporting the damaged and the missing blob with their message, a second copy resolving the change, and dismissal.
Fork checks run locally: name, notice, context, privacy, `expr-schema --check`, and the tools tests.
### Not yet
- A console page, and a note on the blob store form that existing blobs stay where they are (inbuxa-admin).
- An `inbuxa-cli` command, and a docs page.
- Copying ahead of a change, and cleaning up an old store once its change is resolved.
A node that starts with x:BlobStore somewhere else records an
inbuxa:BlobStoreChange and keeps the old store's settings. Until the
change is resolved, a blob the new store lacks is read from the old one
and kept; an administrator has one node copy the rest, checked by hash,
with what couldn't be copied reported. Snapshots are refused meanwhile.
New permission sysBlobStoreChange and event store.blob-read-through.
Spec: docs/spec/features/blob-store-change.md
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Fixes #333. Spec:
docs/spec/features/blob-store-change.md.Changing
x:BlobStoremoved nothing: every blob written under the old setting stayed behind, and a message whose blob was left there failed to read with nothing to say why (forum t/238). Now:inbuxa:BlobStoreChange(from, to, when, which node) and keeps the old store's settings, and the start-up log warns, naming both places. Going back to an earlier store (A to B, then back to A) marks the A-to-B changereverted(BM-4).store.blob-read-through(a new event) is logged. Writes and deletes go only to the current store, and nothing is ever deleted from the old one. Stefan's message would have kept working with no repair.{"action": "copy"}walks the blob index (256 slices, throttled like the store check) and copies every committed blob something holds, plus the spam classifier's two fixed-name files, from the old store as stored. Blobs held by a document, a queued message or a kept deleted item are checked against their hash first, and a damaged one isn't copied. The copy endscopied(resolved: the old store's settings are dropped and every node stops reading through within a minute) orincomplete, with the first 1,000 problems kept. Stop, interrupted and failed work as in the store check.sysBlobStoreChange(697), for administrators only and granted once to existing admin roles. Copy, stop and dismiss are audited, andgetnever returns the old store's settings (BM-9).Design choices to look at
W) until the change is resolved or dismissed. They're catalogued as theblob-store-change-settingssource.Tests
store: read-through and keep, absent in both, writes and deletes touch only the new store, nothing read through once resolved; where a store is (BM-1).inbuxa-features: a move and a move back, recorded once across nodes, the revert, one copy at a time, a quiet copy interrupted, the resolved state.system::blob_store_change, against a real server: the t/238 case end to end, covering read-through, refusals, a slow copy stopped (and not dismissable while it runs), an incomplete copy reporting the damaged and the missing blob with their message, a second copy resolving the change, and dismissal.Fork checks run locally: name, notice, context, privacy,
expr-schema --check, and the tools tests.Not yet
inbuxa-clicommand, and a docs page.