inbuxa/inbuxa-migrate
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity
73 Commits 1 Branch 1 Tag
ae6a481755c7fddecc1c3909df422380c2dfc167
Commit Graph
7 Commits
Author SHA1 Message Date
jcoffey-dev 3cae9464f0 import: bound the memory an IMAP or EWS import holds
ci / test (pull_request) Skipped
Details
github/ci (branch) GitHub Actions
Details
ci / github (pull_request) Successful in 2m50s
Details
ci / announce (pull_request) Skipped
Details 
Two things let a large mailbox fill memory. IMAP workers handed every
fetched message to the archive writer through an unbounded queue, so fast
workers could hold whole folders' worth of bodies while the single writer
caught up. And fetch batches were sized by count only: 20 items per EWS
GetItem, 8 connections at a time, is a few megabytes of ordinary mail and
several gigabytes of large attachments.

- The IMAP event queue is bounded at two events per worker, so a worker
  waits for the writer instead of running ahead of it.
- Fetch batches are bounded by bytes as well as by count, with a shared
  helper, `sync::batch::by_count_and_bytes`: 32 MiB by default, and a
  single larger message goes alone.
  - IMAP learns each new message's RFC822.SIZE on the control connection,
    in 1000-UID metadata fetches, before the body fetch. A server that won't
    say leaves the chunks sized by count. `--fetch-batch-mib` sets the cap.
  - EWS asks FindItem for `item:Size` and packs GetItem batches by it,
    fetched `--ews-connections` batches at a time. `--ews-getitem-batch-mib`
    sets the cap. Items from an incremental SyncFolderItems run carry no
    size and stay batched by count.
- docs/usage.md describes both options.
 2026-09-30 12:32:29 -07:00
Maurus Decimus 70fa688d25 Fix IMAP: import failed after authenticating to Dovecot when a capability contained :, such as IMAPSIEVE=sieve://... (fixes #43) 2026-09-28 11:38:10 +02:00
Maurus Decimus adbd0276a6 EWS fixes 2026-06-10 20:26:41 +02:00
Maurus Decimus f7371cc2ae Fix MS Exchange EWS: add support for version negotiation and other enhancements (closes #6) 2026-06-10 15:27:39 +02:00
Maurus Decimus 295038d8c0 Fix MS Exchange EWS: add support for version negotiation and other enhancements (closes #6) 2026-06-10 14:19:23 +02:00
Maurus Decimus bf0ef2083b Fix MS Exchange EWS: add support for version negotiation and other enhancements (closes #6) 2026-06-10 11:43:09 +02:00
Maurus Decimus 576073f8c9 Initial commit 2026-05-29 18:02:15 +02:00