Store layout step 2 → 3: the audit log, journal, holds and undelete notes get subspaces of their own #368

Merged
jcoffey-dev merged 2 commits from feat/store-layout-step-3 into main 2026-11-11 06:44:15 +00:00
Owner

Store layout step 2 → 3 (docs/spec/features/store-layout.md, SL-15 to SL-20, SL-22). Layout version 3.

What moves. The audit log, the journal's entries and archive, legal holds and undelete's notes move out of _ into subspaces of their own: ib_audit (0x80), ib_journal (0x81), ib_holds (0x82), ib_undelete (0x83). Each key loses the letter that named its feature (Le… → e…). The audit settings (Ls) and the journals themselves (Jj) are settings and stay in _.

Commits

  1. SL-15: every backend gets table and column family names from store::subspace_name. Below 0x80 a subspace keeps its one-byte name, so nothing changes on disk. This also removes the unchecked UTF-8 casts in the RocksDB backend.
  2. The step itself:
    • the four subspaces on PostgreSQL, MySQL, SQLite and RocksDB;
    • the four feature modules writing there;
    • a compliance export family, on by default, so EXPORT_TYPES=registry leaves the moved data out (SL-19);
    • INBUXA_STORE_VERSION = 3;
    • the step, in crates/migration/src/fork_subspaces.rs.

The step

  • It lists the two-byte prefixes in _, reading one key per prefix, and checks each against the owner table (OWNERS, also in the spec). A key nobody claims stops it before it writes anything.
  • It copies each moving prefix. On SQL that's one INSERT … SELECT on the server. Elsewhere it copies 10,000 rows at a time.
  • It checks the copy: row counts, plus the audit log's and journal's chain heads and floors compared row for row.
  • Only then does it set Vt, delete the originals and drop RocksDB's legacy v/c column families (SL-20). Those are now opened only where they still exist.
  • It can run again safely (SL-2). Before Vt it starts over; after Vt it copies whatever is left of the originals again and deletes it.
  • It is not rolling-safe (SL-18), so it waits until no other node is active.

Prod: _ holds 9,629 rows (3.2 MB). The only rows that move are 6,813 audit log rows; prod has no journal, hold or undelete rows. Every prefix on prod is claimed (there's a unit test for that).

Tests

  • store_version on RocksDB, SQLite, PostgreSQL and MySQL:
    • real audit entries plus raw journal, hold and undelete rows, put back in the version 2 form and moved;
    • the audit chain verifies and carries on from its head;
    • settings and other features stay;
    • a second run changes nothing;
    • a run that stopped while deleting finishes;
    • an unclaimed key stops the step with nothing changed;
    • a copy that doesn't match stops it with nothing deleted;
    • 10,050 rows go through the chunked copy.
  • A RocksDB unit test drops v and c, and checks the store reopens without them.
  • store::store_tests (export and import, with the compliance family), journal_tests, legal_hold_tests and undelete_tests pass on RocksDB.
  • audit_log_tests fails at test 3b ("Account id b is invalid" when sharing a mailbox). It fails the same way on main at 983f3783, so it predates this change.
  • Fork checks pass locally.

Not in this PR

  • The docs (inbuxa.org PR to follow): the version 3 row on install/updating and the compliance family on run/backup. They name 27.1-rc.20 as the first release, to be confirmed when it's cut.
  • tools/import-check/check.sh runs on the RC (SL-21).
  • Prod needs every node stopped: stop all three, start one, wait for "Upgraded the store layout to version 3", then start the others. A node that has just booted counts as active for up to 3 minutes, so expect about 4 minutes of full downtime.
Store layout step 2 → 3 (docs/spec/features/store-layout.md, SL-15 to SL-20, SL-22). Layout version 3. **What moves.** The audit log, the journal's entries and archive, legal holds and undelete's notes move out of `_` into subspaces of their own: `ib_audit` (0x80), `ib_journal` (0x81), `ib_holds` (0x82), `ib_undelete` (0x83). Each key loses the letter that named its feature (`Le…` → `e…`). The audit settings (`Ls`) and the journals themselves (`Jj`) are settings and stay in `_`. **Commits** 1. SL-15: every backend gets table and column family names from `store::subspace_name`. Below 0x80 a subspace keeps its one-byte name, so nothing changes on disk. This also removes the unchecked UTF-8 casts in the RocksDB backend. 2. The step itself: - the four subspaces on PostgreSQL, MySQL, SQLite and RocksDB; - the four feature modules writing there; - a `compliance` export family, on by default, so `EXPORT_TYPES=registry` leaves the moved data out (SL-19); - `INBUXA_STORE_VERSION = 3`; - the step, in `crates/migration/src/fork_subspaces.rs`. **The step** - It lists the two-byte prefixes in `_`, reading one key per prefix, and checks each against the owner table (`OWNERS`, also in the spec). A key nobody claims stops it before it writes anything. - It copies each moving prefix. On SQL that's one `INSERT … SELECT` on the server. Elsewhere it copies 10,000 rows at a time. - It checks the copy: row counts, plus the audit log's and journal's chain heads and floors compared row for row. - Only then does it set `Vt`, delete the originals and drop RocksDB's legacy `v`/`c` column families (SL-20). Those are now opened only where they still exist. - It can run again safely (SL-2). Before `Vt` it starts over; after `Vt` it copies whatever is left of the originals again and deletes it. - It is not rolling-safe (SL-18), so it waits until no other node is active. **Prod:** `_` holds 9,629 rows (3.2 MB). The only rows that move are 6,813 audit log rows; prod has no journal, hold or undelete rows. Every prefix on prod is claimed (there's a unit test for that). **Tests** - `store_version` on RocksDB, SQLite, PostgreSQL and MySQL: - real audit entries plus raw journal, hold and undelete rows, put back in the version 2 form and moved; - the audit chain verifies and carries on from its head; - settings and other features stay; - a second run changes nothing; - a run that stopped while deleting finishes; - an unclaimed key stops the step with nothing changed; - a copy that doesn't match stops it with nothing deleted; - 10,050 rows go through the chunked copy. - A RocksDB unit test drops `v` and `c`, and checks the store reopens without them. - `store::store_tests` (export and import, with the compliance family), `journal_tests`, `legal_hold_tests` and `undelete_tests` pass on RocksDB. - `audit_log_tests` fails at test 3b ("Account id b is invalid" when sharing a mailbox). It fails the same way on main at 983f3783, so it predates this change. - Fork checks pass locally. **Not in this PR** - The docs (inbuxa.org PR to follow): the version 3 row on install/updating and the `compliance` family on run/backup. They name 27.1-rc.20 as the first release, to be confirmed when it's cut. - `tools/import-check/check.sh` runs on the RC (SL-21). - Prod needs every node stopped: stop all three, start one, wait for "Upgraded the store layout to version 3", then start the others. A node that has just booted counts as active for up to 3 minutes, so expect about 4 minutes of full downtime.
jcoffey-dev added 2 commits 2026-11-11 05:47:14 +00:00
Every backend built its table or column family name from the subspace
byte, with char::from or an unchecked UTF-8 cast. Every lowercase letter
is taken, so step 2 -> 3 needs subspaces from 0x80 up, which have no
one-byte name and would have made the RocksDB cast undefined.

store::subspace_name now gives each subspace its name. Below 0x80 it is
the byte, as before, so no table or column family changes. From 0x80 up
there are none yet; step 2 -> 3 adds them as ib_ names.
Store layout step 2 -> 3: the fork's large data gets subspaces of its own
ci / fork-checks (pull_request) Skipped
ci / build (pull_request) Skipped
github/ci (branch) GitHub Actions
ci / github (pull_request) Successful in 9m47s
e492880fa2
The audit log, the journal's entries and archive, legal holds and
undelete's notes move out of _ into ib_audit, ib_journal, ib_holds and
ib_undelete, each key without the letter that named its feature. The
audit settings and the journals themselves stay in _, where a
configuration export carries them. Exports carry the moved data in a
compliance family of its own, on by default (SL-19).

The step lists the two-byte prefixes in _ and stops before writing
anything on a key no feature claims (SL-16). It copies each moving
prefix, on SQL with one INSERT ... SELECT, checks counts and the audit
and journal chain heads and floors, and only then deletes the originals
and drops RocksDB's legacy v and c (SL-17, SL-20). It isn't
rolling-safe, so it waits for every other node to stop (SL-18).

INBUXA_STORE_VERSION is 3.
jcoffey-dev merged commit c5333af926 into main 2026-11-11 06:44:15 +00:00
jcoffey-dev deleted branch feat/store-layout-step-3 2026-11-11 06:44:15 +00:00
jcoffey-dev referenced this issue from a commit 2026-11-11 06:46:21 +00:00
Sign in to join this conversation.