export: write a message once, in every folder it was in

jcoffey-dev commented

2026-09-30 18:25:16 +00:00

Owner

A source that files one message in several folders -- IMAP and Maildir
copies, Gmail labels -- leaves one archive row per folder, all pointing at the
same blob. Export imported each row on its own, so the message arrived on the
target once per folder. Worse, an interrupted export matched the rest by
Message-ID alone on the next run and skipped them, so their folders were never
added.

Export now folds rows with the same blob into one email, with the union of
their folders and keywords. On a match it adds, with one batched Email/set,
any migrated folder the target copy is missing; folders that exist only on the
target are left alone. Matching is still by Message-ID (or the fallback
digest), with size deciding between several messages that share one, so two
different messages are never folded onto one target email.

The archive is unchanged: the fold happens at export, so imports, resumes and
deletions work exactly as before.

Roadmap item 2. Tests: five unit tests (folding, size pairing, memberships) and three end-to-end mock tests (one message in two folders lands once in both; a resumed export adds the missing folder with Email/set and imports nothing; two different messages sharing a Message-ID are not merged). The upstream blobNotFound test is reworked to a single email: with folding, two emails can no longer share one uploaded blob.

A source that files one message in several folders -- IMAP and Maildir copies, Gmail labels -- leaves one archive row per folder, all pointing at the same blob. Export imported each row on its own, so the message arrived on the target once per folder. Worse, an interrupted export matched the rest by Message-ID alone on the next run and skipped them, so their folders were never added. Export now folds rows with the same blob into one email, with the union of their folders and keywords. On a match it adds, with one batched Email/set, any migrated folder the target copy is missing; folders that exist only on the target are left alone. Matching is still by Message-ID (or the fallback digest), with size deciding between several messages that share one, so two different messages are never folded onto one target email. The archive is unchanged: the fold happens at export, so imports, resumes and deletions work exactly as before. Roadmap item 2. Tests: five unit tests (folding, size pairing, memberships) and three end-to-end mock tests (one message in two folders lands once in both; a resumed export adds the missing folder with Email/set and imports nothing; two different messages sharing a Message-ID are not merged). The upstream blobNotFound test is reworked to a single email: with folding, two emails can no longer share one uploaded blob.

jcoffey-dev added 1 commit 2026-09-30 18:25:17 +00:00

export: write a message once, in every folder it was in

ci / test (pull_request) Skipped

Details

github/ci (branch) GitHub Actions

Details

ci / github (pull_request) Successful in 2m11s

Details

ci / announce (pull_request) Skipped

Details

5ae0625ee1

A source that files one message in several folders -- IMAP and Maildir
copies, Gmail labels -- leaves one archive row per folder, all pointing at the
same blob. Export imported each row on its own, so the message arrived on the
target once per folder. Worse, an interrupted export matched the rest by
Message-ID alone on the next run and skipped them, so their folders were never
added.

Export now folds rows with the same blob into one email, with the union of
their folders and keywords. On a match it adds, with one batched Email/set,
any migrated folder the target copy is missing; folders that exist only on the
target are left alone. Matching is still by Message-ID (or the fallback
digest), with size deciding between several messages that share one, so two
different messages are never folded onto one target email.

The archive is unchanged: the fold happens at export, so imports, resumes and
deletions work exactly as before.

jcoffey-dev referenced this pull request

2026-09-30 18:32:07 +00:00

export: bring matched items up to date on every run #7

jcoffey-dev merged commit 01ac1a5b3e into main

2026-09-30 18:33:10 +00:00

jcoffey-dev deleted branch fix/one-message-many-folders

2026-09-30 18:33:10 +00:00

jcoffey-dev referenced this pull request

2026-09-30 19:40:19 +00:00

export: batch Email/import, upload in parallel, and show progress #10

export: write a message once, in every folder it was in #4