Keep list refreshes within the server's limits and stop repeating them #378

New Issue

Closed

opened 2026-09-16 15:26:13 +00:00 by jcoffey-dev · 0 comments

jcoffey-dev commented

2026-09-16 15:26:13 +00:00

Owner

Summary

Two bugs from a performance review of the mail store: one breaks the list outright, the other makes every push cost more requests than it needs to.

The list went stale past 500 rows. A refresh fetches everything already on screen.

refreshList sent every loaded id to one Email/get. Stalwart refuses a whole call over maxObjectsInGet, and the catch kept the old list without saying anything. The refresh now runs in pages of at most maxObjectsInGet.
In conversation mode, the other messages of each listed thread were back-referenced from Thread/get with no bound, so 50 long threads could break even the first page. They're now fetched in their own capped requests, and only those the app doesn't already hold, since Email/changes keeps cached ones current. When there's no state to follow changes from, every member is fetched, since nothing else will update them.

Each push repeated work.

listQuery in MailView depended on the whole mailboxes map, which every mailbox reload replaces to update the counts. Each reload built a new query, and query() answered it with another full list refresh. The query now depends on a key built from folder ids, names, roles and parents, which is all the query reads, so a reload that only changes counts leaves it alone.
loadThread fetched every message of the open thread in full, up to 2 MB of body each, on every push, including the echo of your own mark-as-read. It now fetches bodies only for messages not already held in full; a body can't change under an id (RFC 8621). This also keeps the existing attachment objects, so the reading pane stops rebuilding what it already rendered. The fetch goes through getEmails, which splits it to the limit, so a thread longer than 500 messages works too.

Not in this PR: Email/queryChanges is still unused. Switching the refresh to it would be a bigger change, and paging is what fixes the failure.

Related issues

None.

Translations

Adds none.

Testing

New store/__tests__/list-refresh.test.ts uses a fake server that enforces the 500-object cap on every /get, back-referenced ids included, as the mock server does.
- It covers: a 1,200-row refresh, a folder that shrank, 50 threads of 20 messages, a thread with every message already in full (only Thread/get is sent, and the existing object is kept), one new message in a thread, and a 1,200-message thread.
- All 6 fail against main and pass with this change.
npm test -w web (1,312 tests), tsc and vite build are clean.
Against npm run dev:mock in Chrome, with a conversation open, I starred one of its messages from a second session and counted the app's JMAP requests:
- main: 7 requests, including three full list refreshes and Email/get with full bodies for the whole thread.
- This branch: 4 requests: mailbox and changes, the one updated message, one list refresh, and Thread/get with nothing else. The star showed up.
Checked in the same tab: the conversation, the list's thread counts and the attachments all render, and in:work search still finds its folder.

Merged 2026-09-16 as coffey-labs/ihasmail@9691a7bbf5

_{Rebuilt from: git history, session transcript.}

## Summary Two bugs from a performance review of the mail store: one breaks the list outright, the other makes every push cost more requests than it needs to. **The list went stale past 500 rows.** A refresh fetches everything already on screen. - `refreshList` sent every loaded id to one `Email/get`. Stalwart refuses a whole call over `maxObjectsInGet`, and the `catch` kept the old list without saying anything. The refresh now runs in pages of at most `maxObjectsInGet`. - In conversation mode, the other messages of each listed thread were back-referenced from `Thread/get` with no bound, so 50 long threads could break even the first page. They're now fetched in their own capped requests, and only those the app doesn't already hold, since `Email/changes` keeps cached ones current. When there's no state to follow changes from, every member is fetched, since nothing else will update them. **Each push repeated work.** - `listQuery` in `MailView` depended on the whole `mailboxes` map, which every mailbox reload replaces to update the counts. Each reload built a new query, and `query()` answered it with another full list refresh. The query now depends on a key built from folder ids, names, roles and parents, which is all the query reads, so a reload that only changes counts leaves it alone. - `loadThread` fetched every message of the open thread in full, up to 2 MB of body each, on every push, including the echo of your own mark-as-read. It now fetches bodies only for messages not already held in full; a body can't change under an id (RFC 8621). This also keeps the existing attachment objects, so the reading pane stops rebuilding what it already rendered. The fetch goes through `getEmails`, which splits it to the limit, so a thread longer than 500 messages works too. **Not in this PR:** `Email/queryChanges` is still unused. Switching the refresh to it would be a bigger change, and paging is what fixes the failure. ## Related issues None. ## Translations Adds none. ## Testing - New `store/__tests__/list-refresh.test.ts` uses a fake server that enforces the 500-object cap on every `/get`, back-referenced ids included, as the mock server does. - It covers: a 1,200-row refresh, a folder that shrank, 50 threads of 20 messages, a thread with every message already in full (only `Thread/get` is sent, and the existing object is kept), one new message in a thread, and a 1,200-message thread. - **All 6 fail against `main` and pass with this change.** - `npm test -w web` (1,312 tests), `tsc` and `vite build` are clean. - Against `npm run dev:mock` in Chrome, with a conversation open, I starred one of its messages from a second session and counted the app's JMAP requests: - **`main`: 7 requests**, including three full list refreshes and `Email/get` with full bodies for the whole thread. - **This branch: 4 requests**: mailbox and changes, the one updated message, one list refresh, and `Thread/get` with nothing else. The star showed up. - Checked in the same tab: the conversation, the list's thread counts and the attachments all render, and `in:work` search still finds its folder. **Merged** 2026-09-16 as coffey-labs/ihasmail@9691a7bbf551 <sub>Rebuilt from: git history, session transcript.</sub>

This repo is archived. You cannot comment on issues.

1 Participants

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: coffey-labs/ihasmail-github-archive#378