estimate: count and size a Takeout's mail #69

Merged
jcoffey-dev merged 1 commits from feat/estimate-takeout into main 2026-11-12 16:42:41 +00:00
Owner

Follow-up to #67, stacked on #68 (merge #67, then #68, then this). Brings --estimate to the single most common migration source — a Google Takeout — so an operator leaving Gmail can see the scale before committing.

The problem it solves. A Takeout is a local file dump with no server to ask, so its plain --dry-run only counts files. --estimate now reads the mbox files to count the messages that are actually new and sum their bytes.

email: 24,318 new, 4.2 GB

Faithful, not approximate. It deduplicates by the same blake3 content hash the import uses — so a message that appears in two mboxes counts once, and anything already in the archive is skipped (a re-estimate after a partial import correctly reports what is left). It loads the archive's existing hashes once into a set (as IMAP/Maildir load their ids), then scans. It parses no MIME and writes nothing.

Cost. It reads the mbox bodies — unavoidable for a local file source, there is no metadata to ask — but that is the cheap half of what the import does anyway (no MIME parse, no blob intern, no inserts).

Scope. Mail only. A Takeout's calendars (.ics) and contacts (.vcf) are not itemized yet; the file counts in the dry-run log stand in for them. That's the next follow-up if wanted.

Tests. A new sync_takeout test (runnable, no live server) builds a synthetic Takeout, asserts all 9 messages estimate as new and are sized, then imports and re-estimates to assert nothing new. Smoke-tested end to end incl. the duplicate-message and delta cases. cargo test --locked, fmt, clippy -D warnings all pass.

Follow-up to #67, stacked on #68 (**merge #67, then #68, then this**). Brings `--estimate` to the single most common migration source — a Google Takeout — so an operator leaving Gmail can see the scale before committing. **The problem it solves.** A Takeout is a local file dump with no server to ask, so its plain `--dry-run` only counts *files*. `--estimate` now reads the mbox files to count the messages that are actually **new** and sum their bytes. ``` email: 24,318 new, 4.2 GB ``` **Faithful, not approximate.** It deduplicates by the **same blake3 content hash the import uses** — so a message that appears in two mboxes counts once, and anything already in the archive is skipped (a re-estimate after a partial import correctly reports what is left). It loads the archive's existing hashes once into a set (as IMAP/Maildir load their ids), then scans. It parses no MIME and writes nothing. **Cost.** It reads the mbox bodies — unavoidable for a local file source, there is no metadata to ask — but that is the cheap half of what the import does anyway (no MIME parse, no blob intern, no inserts). **Scope.** Mail only. A Takeout's calendars (.ics) and contacts (.vcf) are not itemized yet; the file counts in the dry-run log stand in for them. That's the next follow-up if wanted. **Tests.** A new `sync_takeout` test (runnable, no live server) builds a synthetic Takeout, asserts all 9 messages estimate as new and are sized, then imports and re-estimates to assert nothing new. Smoke-tested end to end incl. the duplicate-message and delta cases. `cargo test --locked`, fmt, clippy `-D warnings` all pass.
jcoffey-dev changed target branch from feat/estimate-dav to main 2026-11-12 16:42:32 +00:00
jcoffey-dev added 1 commit 2026-11-12 16:42:32 +00:00
estimate: count and size a Takeout's mail
ci / test (pull_request) Skipped
github/ci (branch) GitHub Actions
ci / github (pull_request) Successful in 5m52s
ci / announce (pull_request) Skipped
d64a7e9d3b
A Takeout is a local file dump with no server to ask, so its plain dry
run only counts files. --estimate now reads the mbox files to count the
messages that are actually new -- deduplicated by the same content hash
the import uses, so a message in two mboxes counts once and anything
already in the archive is skipped -- and sums their bytes. It parses no
MIME and writes nothing.

Calendars and contacts in a Takeout are not itemized yet; the file
counts in the dry-run log stand in for them.
jcoffey-dev merged commit 3ad85c79d0 into main 2026-11-12 16:42:41 +00:00
jcoffey-dev deleted branch feat/estimate-takeout 2026-11-12 16:42:42 +00:00
Sign in to join this conversation.