Add a POP3 import source #82

Merged
jcoffey-dev merged 1 commits from feat/pop3-source into main 2026-11-13 19:26:58 +00:00
Owner

Adds POP3 as an import source — the first of the remaining [post] sources.

inbuxa-migrate import pop3 \
  --url pop3s://host[:port] | pop3://host[:port] \
  --user <USER> [--password <PASS>] \
  [--allow-cleartext] [--allow-source-change] \
  <ARCHIVE>

POP3 has no folders and no flags, so the whole maildrop lands in one INBOX, keyed by each message's UIDL. A later run into the same archive adds only what arrived since.

Two things set POP3 apart

  • Additive. A message in the archive but gone from the server is kept, not deleted. POP3 mailboxes are routinely emptied after download — that's the protocol's own model — so a message leaving the server says nothing about whether the archive should keep it. This is the one importer that does not mirror deletions.
  • Never changes the source. No DELE: a run only reads, safe to repeat, leaves the server mailbox untouched.

How it's built

  • New src/pop3/ client (RFC 1939 + STLS from RFC 2595): connect, STLS upgrade, USER/PASS, STAT, UIDL, LIST, RETR, QUIT, with dot-unstuffing and 8-bit-safe line IO. The TLS transport is the IMAP client's Connector/ImapStream, reused — STLS is just upgrade_tls on the stream.
  • src/sync/import_pop3.rs coordinator: delta via UIDL against a new sync_id_pop3 table, per-message savepoints, batched commits, blob dedup + orphan GC, source-change guard — the same shape as the other importers, minus folders/flags/deletions.
  • --estimate reports counts and sizes (STAT/UIDL/LIST give per-message octets before any body fetch).
  • Schema: new sync_id_pop3; the sources kind CHECK gains 'pop3', and the existing "rebuild an older sources table" migration is generalized to widen any recognized pre-pop3 table (it already did this for files). An older archive is refused by read-only inspect/reconcile until the next import brings it up to date — unchanged, established behavior.
  • OAuth (SASL XOAUTH2) is intentionally out of scope for v1; --user/password over TLS (incl. app-specific passwords) covers the overwhelming majority of legacy/ISP POP3. Noted as a possible follow-up.

Tests & verification

  • 12 client tests drive the full protocol against a scripted in-memory peer (login, STAT, UIDL+LIST pairing, RETR with dot-unstuffing, 8-bit bytes, multi-chunk bodies, mid-line EOF). Plus pop3_ids, coordinator (endpoint parsing, dry-run/estimate counts), and an updated sources migration test. cargo fmt --check, clippy -D warnings, cargo test --locked (1250) all green.
  • End-to-end against a throwaway mock POP3 server: --estimate (4 new, 176 B), real import (3 into INBOX), inspect, reconcile clean (exit 0); a second import skipped all 3 (delta); the source-change guard fired (exit 4). Dot-stuffed and 8-bit messages round-tripped intact.

Documented in README (Sources + estimate prose), docs/usage.md (new POP3 section), and CHANGELOG [Unreleased].

Zoho and Zimbra (IMAP presets) are the remaining sources, to follow in a separate PR.

Adds POP3 as an import source — the first of the remaining `[post]` sources. ``` inbuxa-migrate import pop3 \ --url pop3s://host[:port] | pop3://host[:port] \ --user <USER> [--password <PASS>] \ [--allow-cleartext] [--allow-source-change] \ <ARCHIVE> ``` POP3 has no folders and no flags, so the whole maildrop lands in one INBOX, keyed by each message's UIDL. A later run into the same archive adds only what arrived since. ### Two things set POP3 apart - **Additive.** A message in the archive but gone from the server is kept, not deleted. POP3 mailboxes are routinely emptied after download — that's the protocol's own model — so a message leaving the server says nothing about whether the archive should keep it. This is the one importer that does not mirror deletions. - **Never changes the source.** No `DELE`: a run only reads, safe to repeat, leaves the server mailbox untouched. ### How it's built - New `src/pop3/` client (RFC 1939 + STLS from RFC 2595): connect, STLS upgrade, USER/PASS, STAT, UIDL, LIST, RETR, QUIT, with dot-unstuffing and 8-bit-safe line IO. The TLS transport is the **IMAP client's `Connector`/`ImapStream`, reused** — STLS is just `upgrade_tls` on the stream. - `src/sync/import_pop3.rs` coordinator: delta via UIDL against a new `sync_id_pop3` table, per-message savepoints, batched commits, blob dedup + orphan GC, source-change guard — the same shape as the other importers, minus folders/flags/deletions. - `--estimate` reports counts **and** sizes (STAT/UIDL/LIST give per-message octets before any body fetch). - Schema: new `sync_id_pop3`; the `sources` kind CHECK gains `'pop3'`, and the existing "rebuild an older sources table" migration is generalized to widen any recognized pre-`pop3` table (it already did this for `files`). An older archive is refused by read-only `inspect`/`reconcile` until the next import brings it up to date — unchanged, established behavior. - OAuth (SASL XOAUTH2) is intentionally out of scope for v1; `--user`/password over TLS (incl. app-specific passwords) covers the overwhelming majority of legacy/ISP POP3. Noted as a possible follow-up. ### Tests & verification - 12 client tests drive the full protocol against a scripted in-memory peer (login, STAT, UIDL+LIST pairing, RETR with dot-unstuffing, 8-bit bytes, multi-chunk bodies, mid-line EOF). Plus `pop3_ids`, coordinator (endpoint parsing, dry-run/estimate counts), and an updated `sources` migration test. `cargo fmt --check`, `clippy -D warnings`, `cargo test --locked` (1250) all green. - End-to-end against a throwaway mock POP3 server: `--estimate` (4 new, 176 B), real import (3 into INBOX), `inspect`, `reconcile` clean (exit 0); a second import skipped all 3 (delta); the source-change guard fired (exit 4). Dot-stuffed and 8-bit messages round-tripped intact. Documented in README (Sources + estimate prose), `docs/usage.md` (new POP3 section), and CHANGELOG `[Unreleased]`. Zoho and Zimbra (IMAP presets) are the remaining sources, to follow in a separate PR.
jcoffey-dev added 1 commit 2026-11-13 19:16:41 +00:00
Add a POP3 import source
ci / test (pull_request) Skipped
github/ci (branch) GitHub Actions
ci / github (pull_request) Successful in 5m51s
ci / announce (pull_request) Skipped
eeae381c18
import pop3 reads a POP3 mailbox into an archive: the whole maildrop
into one INBOX (POP3 has no folders and no flags), keyed by each
message's UIDL so a later run adds only what arrived since. pop3s:// is
TLS from the first byte; pop3:// upgrades with STLS before the password
is sent, or sends cleartext with --allow-cleartext. The TLS transport is
the IMAP client's, reused.

Two things set it apart from the other sources. It is additive: a
message in the archive but gone from the server is kept, since POP3
mailboxes are routinely emptied after download and a message leaving the
server says nothing about whether the archive should still hold it. And
it never changes the source -- no DELE -- so a run only reads and is safe
to repeat.

--estimate reports counts and sizes from STAT/UIDL/LIST before any body
is fetched. Messages are keyed in a new sync_id_pop3 table; the sources
kind CHECK gains 'pop3', and an older archive's table is rebuilt to
accept it, as it already was for earlier kinds.
jcoffey-dev merged commit d692eb3524 into main 2026-11-13 19:26:58 +00:00
jcoffey-dev deleted branch feat/pop3-source 2026-11-13 19:26:58 +00:00
Sign in to join this conversation.