Import an iCal file in batches the server will take #215

New Issue

Closed

opened 2026-09-02 13:30:05 +00:00 by jcoffey-dev · 0 comments

jcoffey-dev commented

2026-09-02 13:30:05 +00:00

Owner

An 800 KB export imported nothing at all — reported on #173 after the feature shipped.

Every event in the file went out in a single CalendarEvent/set, and Stalwart refuses a method call carrying more objects than maxObjectsInSet: the whole call, with requestTooLarge, creating none of it. So the import failed at exactly the size that makes importing worth doing. A two-event invitation was fine; a real calendar was not.

{"type":"requestTooLarge","description":"The number of ids requested by the client exceeds the maximum number the server is willing to process in a single method call."}

What changed

The events go out maxObjectsInSet at a time. The client already reads that off the session's core capability and falls back to 500 where a server does not advertise it — the same ceiling and the same chunk helper the mail store batches deletes and flag changes by. Nothing new had to be learned about the limit, and nobody has to split an .ics by hand at an arbitrary line.

Still batches rather than a call per event: createEvent invalidates on the way out and invalidating re-fetches every cached range, which is why the import writes its own set calls in the first place. One invalidate, after the last batch.

A batch that fails after earlier ones have been filed now says how much got in — "1000 of 1200 events were imported before this happened" — and re-reads the calendar so those events are visible. Reporting only that the import failed would send someone looking for events that are already there.

Tests

Seven new ones, against a mock server that refuses an oversized call the way Stalwart does rather than quietly taking the first 500 — which is what makes them more than an assertion about call counts. All seven fail on main.

1200 events split into 500 / 500 / 200, and all 1200 filed
split by what the session advertises (40 → 40 / 40 / 20), not by a constant
every event still distinct across the split
one calendar re-read, not one per batch
a failure in the third batch reports "1000 of 1200", and still re-reads
a failure in the first passes the server's own words through

npm run typecheck, npm test (878 web + 122 server), npm run build all pass.

Not in this PR

ContactCard/set has the same shape in three places — the vCard import, the LDIF import, and a bulk destroy — so the address-book import will fail the same way on a large enough file. That belongs with #174, where the LDIF feedback also asks for duplicate checking.

Merged 2026-09-02 as coffey-labs/ihasmail@6b8ac393c9

_{Rebuilt from: git history, session transcript.}

An 800 KB export imported nothing at all — reported on #173 after the feature shipped. Every event in the file went out in a single `CalendarEvent/set`, and Stalwart refuses a method call carrying more objects than `maxObjectsInSet`: the whole call, with `requestTooLarge`, creating none of it. So the import failed at exactly the size that makes importing worth doing. A two-event invitation was fine; a real calendar was not. ```json {"type":"requestTooLarge","description":"The number of ids requested by the client exceeds the maximum number the server is willing to process in a single method call."} ``` ## What changed The events go out `maxObjectsInSet` at a time. The client already reads that off the session's core capability and falls back to 500 where a server does not advertise it — the same ceiling and the same `chunk` helper the mail store batches deletes and flag changes by. Nothing new had to be learned about the limit, and nobody has to split an `.ics` by hand at an arbitrary line. Still batches rather than a call per event: `createEvent` invalidates on the way out and invalidating re-fetches every cached range, which is why the import writes its own `set` calls in the first place. One invalidate, after the last batch. A batch that fails after earlier ones have been filed now says how much got in — *"1000 of 1200 events were imported before this happened"* — and re-reads the calendar so those events are visible. Reporting only that the import failed would send someone looking for events that are already there. ## Tests Seven new ones, against a mock server that refuses an oversized call the way Stalwart does rather than quietly taking the first 500 — which is what makes them more than an assertion about call counts. All seven fail on `main`. - 1200 events split into 500 / 500 / 200, and all 1200 filed - split by what the session advertises (40 → 40 / 40 / 20), not by a constant - every event still distinct across the split - one calendar re-read, not one per batch - a failure in the third batch reports "1000 of 1200", and still re-reads - a failure in the first passes the server's own words through `npm run typecheck`, `npm test` (878 web + 122 server), `npm run build` all pass. ## Not in this PR `ContactCard/set` has the same shape in three places — the vCard import, the LDIF import, and a bulk destroy — so the address-book import will fail the same way on a large enough file. That belongs with #174, where the LDIF feedback also asks for duplicate checking. **Merged** 2026-09-02 as coffey-labs/ihasmail@6b8ac393c906 <sub>Rebuilt from: git history, session transcript.</sub>

This repo is archived. You cannot comment on issues.

1 Participants

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: coffey-labs/ihasmail-github-archive#215