Read message archives in batches

jcoffey-dev commented

2026-10-08 18:13:23 +00:00

Owner

Backlog item E from the 2026-10-07 cluster survey: upstream reads message archives one key per round trip, which on the three-node cluster is one PostgreSQL hop over WireGuard per message.

Store

Store::get_values(&[key]) -> Vec<Option<U>>: on PostgreSQL one SELECT k, v FROM <subspace> WHERE k = ANY($1), values returned in the keys' order with None for a missing key; every other backend loops get_value. A replicated store serves the batch from a replica and re-reads from the primary only the keys the replica has not received yet (same read-your-writes rule as get_value).
READ_BATCH_SIZE = 128: callers hand over at most this many keys at once, so a FETCH of a whole mailbox never holds every archive in memory.

Callers rewritten to read in batches

IMAP FETCH: the metadata of each 128-message chunk in one round trip; when the fetch will set \Seen, the message data of the still-unseen messages in a second. Was one or two round trips per message.
IMAP STORE: the message data of each chunk in one round trip.
JMAP Email/get: the metadata of each chunk in one round trip.
Message and mailbox cache refresh (crates/email/src/cache/{email,mailbox}.rs): runs on every node after any write it hears about; a bulk move or flag change of N messages used to cost N round trips per node.

Not changed: the index task builds one document per task, so there is nothing to batch there yet; blob reads are item G.

Tests

New batched_reads case in the store suite (tests/src/store/ops.rs): 300 keys with gaps, a repeated key and keys never written; checks order, Nones, agreement with get_value, the empty batch, and the result after a clear. Passes on PostgreSQL and SQLite.
IMAP suite on PostgreSQL: passes (43 s).
JMAP suite on PostgreSQL: the one pre-existing failure only (email/filter-header-name-value, fails the same way on main), everything else passes.

Round trips, counted with log_statement=all on the test PostgreSQL:

suite	single-key `SELECT v … k = $1`	batched `k = ANY($1)`	all statements
IMAP, main	7,214	0	17,190
IMAP, this branch	6,336	243	16,554
JMAP, this branch	102,224	4,189	582,828

The suites fetch a handful of messages per command, so the batches are small; on a mailbox FETCH of N messages the metadata reads go from N round trips to N/128. The JMAP main baseline follows in a comment.

Cluster numbers follow after a roll, as with #167.

Backlog item E from the 2026-10-07 cluster survey: upstream reads message archives one key per round trip, which on the three-node cluster is one PostgreSQL hop over WireGuard per message. **Store** - `Store::get_values(&[key]) -> Vec<Option<U>>`: on PostgreSQL one `SELECT k, v FROM <subspace> WHERE k = ANY($1)`, values returned in the keys' order with `None` for a missing key; every other backend loops `get_value`. A replicated store serves the batch from a replica and re-reads from the primary only the keys the replica has not received yet (same read-your-writes rule as `get_value`). - `READ_BATCH_SIZE = 128`: callers hand over at most this many keys at once, so a FETCH of a whole mailbox never holds every archive in memory. **Callers rewritten to read in batches** - IMAP FETCH: the metadata of each 128-message chunk in one round trip; when the fetch will set `\Seen`, the message data of the still-unseen messages in a second. Was one or two round trips per message. - IMAP STORE: the message data of each chunk in one round trip. - JMAP Email/get: the metadata of each chunk in one round trip. - Message and mailbox cache refresh (`crates/email/src/cache/{email,mailbox}.rs`): runs on every node after any write it hears about; a bulk move or flag change of N messages used to cost N round trips per node. Not changed: the index task builds one document per task, so there is nothing to batch there yet; blob reads are item G. **Tests** - New `batched_reads` case in the store suite (`tests/src/store/ops.rs`): 300 keys with gaps, a repeated key and keys never written; checks order, `None`s, agreement with `get_value`, the empty batch, and the result after a clear. Passes on PostgreSQL and SQLite. - IMAP suite on PostgreSQL: passes (43 s). - JMAP suite on PostgreSQL: the one pre-existing failure only (`email/filter-header-name-value`, fails the same way on main), everything else passes. **Round trips**, counted with `log_statement=all` on the test PostgreSQL: | suite | single-key `SELECT v … k = $1` | batched `k = ANY($1)` | all statements | |---|---|---|---| | IMAP, main | 7,214 | 0 | 17,190 | | IMAP, this branch | 6,336 | 243 | 16,554 | | JMAP, this branch | 102,224 | 4,189 | 582,828 | The suites fetch a handful of messages per command, so the batches are small; on a mailbox FETCH of N messages the metadata reads go from N round trips to N/128. The JMAP main baseline follows in a comment. Cluster numbers follow after a roll, as with #167.

jcoffey-dev added 1 commit 2026-10-08 18:13:23 +00:00

Read message archives in batches

ci / fork-checks (pull_request) Skipped

Details

ci / build (pull_request) Skipped

Details

github/ci (branch) GitHub Actions

Details

ci / github (pull_request) Successful in 9m26s

Details

f80fe8b873

Store::get_values reads a set of keys in one round trip on PostgreSQL
(k = ANY($1)) and falls back to one get_value per key elsewhere. A
replicated store serves the batch from a replica and re-reads from the
primary the keys the replica has not received yet.

IMAP FETCH and STORE, JMAP Email/get and the message and mailbox cache
refreshers that every node runs after a write now read their archives
READ_BATCH_SIZE (128) at a time instead of one round trip per message.
FETCH also prefetches the message data of the messages it is about to
mark \Seen, saving the second round trip per message.

jcoffey-dev commented

2026-10-08 18:22:17 +00:00

Author

Owner

JMAP suite baseline on main (same count): 108,854 single-key reads, 0 batched, 585,342 statements in all. This branch: 102,224 + 4,189 batched, 582,828 in all. The suite reads mostly one message per Email/get, so the batches average about 2.5 keys; the cluster numbers after a roll are the ones to watch.

jcoffey-dev merged commit d6ca092d4c into main

2026-10-08 19:15:07 +00:00

jcoffey-dev deleted branch perf/batched-reads

2026-10-08 19:15:07 +00:00

jcoffey-dev referenced this pull request

2026-10-08 19:53:59 +00:00

Sort and filter by received date on the message cache #172

jcoffey-dev referenced this pull request

2026-10-08 20:02:43 +00:00

Release 2026.10.9 #173

Read message archives in batches #170