Files
inbuxa-migrate/CHANGELOG.md
T
jcoffey-dev 62d00525ec
ci / test (pull_request) Skipped
github/ci (branch) GitHub Actions
ci / github (pull_request) Successful in 9m56s
ci / announce (pull_request) Skipped
Version 27.1.0-rc.14
2026-11-13 12:54:18 -07:00

26 KiB

Change Log

All notable changes to this project are recorded here. Versions follow the inbuxa suite: a series (27.1), its patches (27.1.3) and release candidates before a series ships (27.1-rc.1). The tag is the version with a v. Releases before 27.1 were dates (v2026.9.30).

[Unreleased]

[27.1-rc.14]

The fourteenth release candidate of the 27.1 series: POP3, Fastmail, Zoho and Zimbra sources, a reconcile report that a migration was complete, and upstream v1.0.16's Exchange fixes.

Added

  • import pop3 imports from a POP3 server: the whole maildrop into one INBOX, keyed by each message's UIDL so a later run adds only what arrived since. pop3s:// is TLS, pop3:// upgrades with STLS (or sends cleartext with --allow-cleartext). It is additive -- a message gone from the server is kept, since POP3 mailboxes are routinely emptied after download -- and it deletes nothing on the server. --estimate reports counts and sizes from STAT/UIDL/LIST before any body is fetched.
  • import zoho and import zimbra import mail over IMAP, as presets for import imap. zoho fills in Zoho's server for your region (--region, or --host for one not listed); zimbra points at your own Zimbra host (IMAPS on 993, or --starttls on 143). Both take only an address and password (or an app-specific password), reuse the whole IMAP pipeline, and support --estimate.
  • import fastmail imports from a Fastmail account over JMAP. It is import jmap with Fastmail's server and bearer-token sign-in filled in, so it needs only an API token; with no --account-* it imports the account Fastmail marks primary. Mail always; calendars and contacts where that account advertises them.
  • reconcile <archive> reads an archive folder by folder and reports it: a message count and a content checksum for each folder, totals and a checksum for the whole, and an integrity check that every message resolves to stored content and a real folder. It is read-only and touches no network. Compare the per-folder counts against the source mailbox to prove a migration was complete; the checksum shows two archives -- an export and a re-import, say -- hold the very same mail. It exits 8 when the archive is not sound.

Changed

  • Upstream v1.0.16 is merged in.
  • Outlook categories become valid JMAP keywords, the same way IMAP --mime-keywords does it (Catégorie Rouge is cat_gorie_rouge), and duplicates are dropped. Before, only the case was lowered, so Project review gave a keyword JMAP doesn't allow. A read-receipt request no longer adds $notified. This is for EWS and Graph.

Fixed

  • Exchange EWS and Graph: all-day events and birthdays landed a day early east of UTC. The date is now taken in the event's time zone. Outlook's "no date" (year 4500 or later) is no longer imported as a real date. Archives imported before this keep the wrong dates until the item changes at the source, so re-import into a fresh archive to correct them.
  • Exchange EWS: Exchange 2019 servers were asked for a schema version that doesn't exist, so every call was refused once and sent again.
  • Exchange EWS: hidden system folders (Recipient Cache, GAL Contacts) were imported as mailboxes and address books. They are skipped now.
  • Exchange EWS: a mail folder under a calendar or contacts folder could stop a later run with a FOREIGN KEY error. It is filed under its nearest mail folder on every run.
  • Exchange EWS: an attendee's or contact's ItemId could replace the item's own, giving UNIQUE constraint failed. Tasks and posts in a listing shifted the ids around them. One item that failed to parse or store stopped the rest of its folder; it is now logged and counted, and the folder goes on.
  • Exchange EWS: event attachments, your own response to a meeting and an event's private flag were never fetched. They are imported now.
  • Exchange EWS: a folder with no class (Outlook's (This computer only) contacts) was always taken for mail, and contacts, appointments and tasks in a mail folder such as Deleted Items were stored as email. A class-less folder now takes its kind from its items, and mail folders skip non-mail items, also in --estimate.

[27.1-rc.13]

The thirteenth release candidate of the 27.1 series: shell completions and a man page, secrets read from files, a Microsoft 365 request throttle, a shared server profile, and a guide that shows real progress.

Added

  • completions <shell> prints a shell completion script (bash, zsh, fish, elvish or powershell) to standard output, and man prints a man page in roff. Both are generated from the command itself and reach no server.
  • Every secret read from an INBUXA_MIGRATE_* variable can instead be read from a file named by the same variable with _FILE appended, the Docker and systemd secrets convention (INBUXA_MIGRATE_PASSWORD_FILE, INBUXA_MIGRATE_API_KEY_FILE, and so on). One trailing newline is ignored; setting both a variable and its _FILE form is an error. This keeps a secret out of both the environment and the command line.
  • import exchange-graph and import exchange-ews take --max-rps to cap how many requests start per second across all worker connections -- a proactive throttle to stay under a rate-limited Microsoft 365 tenant's limit, alongside the existing honoring of 429/Retry-After. Off by default (uncapped).
  • import jmap and import-directory read the inbuxa server's address and API key from a shared profile, ~/.config/inbuxa/config.toml, so the target need not be given each time. [profile.<name>] sections hold a url and either an api-key or an api-key-file (a path, so the key can live in its own file); --profile or $INBUXA_PROFILE pick a section, and the default is default. inbuxa-cli reads the same file. A --url or an INBUXA_MIGRATE_* variable still wins, so nothing changes when they are given; --url is now optional on these two commands when a profile supplies it.

Changed

  • The guide page shows a real progress bar with a running count and percentage while a preview or move runs, reads the counts from the run's structured progress stream rather than scraping its log, and when a run ends it says how many accounts didn't finish and points at the table. A reload reconnects to a run already in progress -- the guide outlives the browser -- and now says so.

[27.1-rc.12]

The twelfth release candidate of the 27.1 series: upstream v1.0.15, keywords from IMAP Keywords: headers, and fixes for Maildir folder names and EWS --mailbox.

Added

  • IMAP: --mime-keywords adds the phrases of each message's Keywords: header as keywords. Exchange shows its categories over IMAP this way. Each phrase is lowercased and its spaces become _. Letters outside ASCII are dropped, and a phrase can't name a system keyword such as $seen.

Changed

  • Upstream v1.0.15 is merged in.

Fixed

  • Maildir: folder names kept in modified UTF-7 are now decoded in full (.&ZeVnLIqe-.&U,BTFw- is 日本語.台北). Folder selection and role guessing use the decoded name. Before, only the last part of a name was decoded, for display.
  • Exchange EWS: with Basic or delegated OAuth sign-in, --mailbox naming another mailbox the user has full access to imported the user's own mailbox instead. Folders are now addressed in the mailbox --mailbox names.

[27.1-rc.11]

The eleventh release candidate of the 27.1 series: upstream v1.0.14, and transfers that stall no longer wait out the whole limit.

Changed

  • Upstream v1.0.14 is merged in. Its HTTP timeouts are left out, since this project already has its own, and so is its Dockerfile.

Fixed

  • Google Takeout: Portuguese (Brazil) system labels now map to mailbox roles and keywords, or are dropped, like their English counterparts.
  • Google Takeout: a message labeled neither Opened nor Unread is now imported as read. Only English and Portuguese (Brazil) labels are recognized, so an unread label in another language becomes a folder and the message imports as read.
  • Google Takeout: a contact without a UID is stored once rather than once per contact group file. Archives from an earlier version keep the old copies, so re-import into a fresh archive.
  • A download or upload that stalls partway now fails after 5 minutes of silence and is retried, rather than waiting out the limit on the whole body: 30 minutes for a download, over two hours for the largest uploads.

[27.1-rc.10]

The tenth release candidate of the 27.1 series: real file names kept through an account export.

Added

  • export-files keeps the real name of every file, directory and link whose name had to change on disk, in the manifest's new files.names, and import files restores them. A real name that is taken in its directory, or can't be a name, is left as it is on disk with a warning, so a renamed entry never takes another's place; a listed link whose name is taken is not made.

[27.1-rc.9]

The ninth release candidate of the 27.1 series: an account export read back whole.

Added

  • import files reads an account export back into an archive: the tree export-files writes, specified at https://coffeylabs.org/specs/account-export/. Besides the mail, contacts, calendars, Sieve scripts and files, it reads what only the manifest holds: real folder names, roles and subscriptions, the default address book and calendar, calendar colors and time zones, the active script, media types, links and identities. Copies of a message in several folders come back as one message in each. Paths in the manifest are checked to stay inside the export, and no symbolic link is followed. An archive gains the source kind files and a table for what it read at its next import; until then inspect and the other read-only commands call it an older layout.

[27.1-rc.8]

The eighth release candidate of the 27.1 series: what a server needs to run migrations into itself, from its admin console.

Added

  • migrate --progress json writes one JSON event per line on stdout (a step begins or ends, an account ends, the run ends), for a program that runs a migration and shows its progress.
  • plan --answers writes the guide's plans from its answers in a file, so a program can ask the questions itself and run migrate. The answers hold no secrets.
  • --stay-on-url sends every request to the --url host, whatever the server's session advertises, for a server reached at its own loopback or through a tunnel. import jmap, export, export-directory and import-directory take it.
  • import imap --google-key-from-env reads the Google service account's key from $INBUXA_MIGRATE_GOOGLE_KEY, and a plan step can name it as the googleKey secret.

[27.1-rc.7]

The seventh release candidate of the 27.1 series: upstream's 1.0.12 and 1.0.13 merged in, with out-of-office replies, --replace-source, and files kept working with inbuxa.

Added

  • The out-of-office reply comes across: imported from a server's VacationResponse, and exported to the target's, in place of copying the script the server keeps for it.
  • WebDAV, CalDAV, CardDAV: --replace-source removes the other recorded sources of the same kind, and what was imported from them, before importing, so a moved share leaves one tree in the archive.
  • Files keep their executable flag where the server has one. An archive gains the column at its next import; until then inspect and the other read-only commands call it an older layout.

Changed

  • FileNodes are read in either shape servers use: nodeType and type, as inbuxa and other 0.16-line servers spell them, or @type and mediaType (draft-ietf-jmap-filenode-18, as in Stalwart 1.0). An export asks the target which it takes and writes that one. Upstream dropped the older shape; this fork keeps both.
  • The Graph device-code sign-in asks for Files.Read, so the default run reaches OneDrive.
  • iCalendar and vCard parse failures say what was wrong and quote the line.

Fixed

  • IMAP: children of a \Noselect parent folder kept their place instead of landing at the top level.
  • JMAP import no longer holds every blob of a page in memory at once, which large messages could exhaust.
  • CalDAV, CardDAV: items with bare CR line breaks, as Kerio Connect sends, are read.
  • EWS: a bare --auth-bearer takes the API key or token from the environment without asking for --ews-client-id.
  • --account-name as an administrator takes the data account from the principal's accounts, as RFC 9670 defines it.

[27.1-rc.6]

The sixth release candidate of the 27.1 series: whole domains from Microsoft 365 and Google Workspace, with one admin credential and no one's password.

Added

  • import exchange-graph --app-only signs in as the app itself, with its client secret, and reads the mailbox --user names. With an Entra app granted application permissions, a whole Microsoft 365 tenant moves in a plan with no one signing in. The token is renewed before it runs out. A plan's clientSecret now reaches Graph as well as EWS.
  • import imap --google-key signs in to Gmail as a Google Workspace service account with domain-wide delegation, acting for the user --auth-user names. A whole Workspace domain moves in a plan with one key and no one's password. The JWT is signed with the key itself (RS256), and each connection takes a fresh token when the last is close to running out.
  • The guide offers Microsoft 365 (an Entra app, app-only) and Google Workspace (a service account) as sources. Its check signs the app in and names any permission it lacks, or asks Google for a token for the first account.

[27.1-rc.5]

The fifth release candidate of the 27.1 series: the guided mode. A move into inbuxa can now be walked through in the browser, from the source to the final pass.

Added

  • guide walks through a move into inbuxa in the browser. It serves a page on 127.0.0.1 that asks for the source (IMAP with a master user, JMAP with an administrator, or Maildir folders), the accounts (pasted, from a CSV, or read from the source) and the inbuxa server. It can create the missing accounts, previews with a dry run, then moves the mail with live progress and runs a final pass. It writes an ordinary plan and runs it with migrate, so the same move can be finished from a terminal. Secrets stay in memory, and only the browser given the address, with its random token, gets an answer.

[27.1-rc.4]

The fourth release candidate of the 27.1 series: a whole domain moves with one plan file, set up on inbuxa from an exported directory and exported account by account with one administrator's key.

Added

  • import-directory sets up on an inbuxa server what export-directory wrote: tenants, roles, domains, groups, users and mailing lists, with their aliases, quotas, roles and memberships. What the server already has is left as it is, so running it again creates nothing. Each user created gets a generated password, written to a new file readable by its owner only.
  • migrate --plan moves many accounts with one command. The plan names the steps, which are the commands in the docs with {email} and {archive} filled in per account, and the accounts, listed or taken from directory.json. Several accounts run at once, each with its own archive and logs. A second run resumes where each stopped, --again runs the final pass, and report.csv says what moved and why anything didn't. Secrets are named by environment variable, never written in the plan.

Fixed

  • export into inbuxa renames Stalwart's Sieve extensions (vnd.stalwart.while and the rest) to inbuxa's. It looked for inbuxa's names in the server's list of Sieve extensions, which doesn't name them, so scripts went in unchanged. It now goes by the session, which says the server is inbuxa.
  • inspect opens the archive read-only. It used to create an empty archive at a mistyped path and bring an older archive's layout up to date; now the first is an error and the second is refused, and reading a closed archive leaves no files beside it.
  • import carddav takes a card that is in two address books into both. It used to fail on that card on every run. Taking it out of one book, or deleting that book, now leaves it in the other.
  • import caldav keeps every occurrence of a series someone was invited to one occurrence at a time, with no master event. It used to keep the first and drop the rest. export now tells such occurrences apart instead of writing them all onto one event, and export-caldav writes them as one resource, as CalDAV keeps them.

[27.1-rc.3]

The third release candidate of the 27.1 series: contacts, calendars and filters go straight into another server, as mail already could.

Added

  • export-managesieve writes an archive's Sieve scripts into any ManageSieve server and makes the archive's active script the active one. Like export-imap, it writes only what's missing or differs, so running it again writes nothing. A script the server refuses is counted and the rest go on; --dry-run checks each one with CHECKSCRIPT.
  • export-caldav and export-carddav write an archive's calendars and address books into any CalDAV or CardDAV server. The account's home is found from the server's root, collections are matched by name or made, and each event or card is written only when it's missing or differs, under its etag, so running it again writes nothing. Nothing is deleted.

Changed

  • export-files names its layout in manifest.json by the address of its specification, https://coffeylabs.org/specs/account-export/, instead of inbuxa-migrate-files. The layout itself is unchanged (formatVersion 1).

[27.1-rc.2]

The second release candidate of the 27.1 series: three ways out of an archive or a server that need no vendor's store format.

Added

  • export-files writes an archive out as plain files in open formats: mail as a Maildir++ tree, each address book as a .vcf, each calendar as an .ics, Sieve scripts, the file tree, and a manifest.json with what those formats don't hold. It needs no network, writes only into a new or empty directory, and keeps the directory private to its owner.
  • export-imap writes an archive's mail into any IMAP server: folders in the server's own layout (SPECIAL-USE folders for sent, drafts, trash, junk and archive), flags, keywords where the folder keeps them, and arrival times. Like export, it matches what is already there, so running it again writes only what changed.
  • export-directory writes who is on a server: its domains, users, groups, mailing lists, aliases, quotas, tenants and roles, read from the registry with an administrator's login, as directory.json and CSV files that another server can be set up from. Credentials, OTP seeds, API keys and DKIM keys are never asked for. --domain keeps it to some domains.

Changed

  • import maildir reads folder names in modified UTF-7, as Dovecot and Courier write them, and the custom keywords in each folder's dovecot-keywords file (Junk, labels).
  • A FETCH section with a header list, such as BODY[HEADER.FIELDS (MESSAGE-ID)], is read as one item.
  • import takeout unfolds and unescapes a calendar's X-WR-CALNAME and X-WR-CALDESC, so a long name or one with a comma reads back whole.

[27.1-rc.1]

The first release candidate of the 27.1 series, with the rest of the suite.

Changed

  • --version and the User-Agent show the version as tags name it (27.1-rc.1).

[2026.10.26] -- 2026-10-26

Changed

  • --auth-bearer reads INBUXA_MIGRATE_API_KEY, the name inbuxa-cli and the console use. INBUXA_MIGRATE_TOKEN still works; when both are set, the API key wins.

[2026.9.30] -- 2026-09-30

The first release of inbuxa-migrate.

Changed

  • Renamed to inbuxa-migrate: the binary, the crate, and the credential variables, now INBUXA_MIGRATE_PASSWORD, INBUXA_MIGRATE_TOKEN, INBUXA_MIGRATE_EWS_CLIENT_SECRET and INBUXA_MIGRATE_GRAPH_TOKEN. The old names are not read. Archives written by Vandelay are not read either.
  • Registry calls (x:Account, x:Domain and the other x: types) use the capability the server advertises: urn:inbuxa:jmap:registry on inbuxa, urn:stalwart:jmap on a Stalwart source. A server that advertises neither gets a clear error instead of a request it never offered to accept.
  • Released as Linux archives for amd64 and arm64 on the Gitea release page, with SHA256SUMS. The npm, Homebrew, shell and PowerShell installers and the MSI are gone.
  • A second export brings matched items up to date instead of skipping them: flags and folders on mail, changed contacts and events, and edited Sieve scripts.
  • A message filed in several folders is written once, in every one of them, and a resumed export adds any folder it was still missing.
  • Sieve scripts from a Stalwart server keep working on inbuxa: their vnd.stalwart.* extension names are renamed to inbuxa's, each rename is logged, and a script that can't be activated fails the export instead of leaving filtering quietly off.
  • --allow-invalid-certs covers only the server named with --url. Microsoft and Google sign-in are always verified.
  • Export imports mail in batches, uploads in parallel within the target's limits, and prints progress: count, rate and time left.
  • export --dry-run predicts what would fail -- messages over the target's upload limit, objects over its request limit, Sieve scripts it would reject -- lists what would be created, updated and skipped, and exits as the real run would.

Fixed

  • Every connection times out instead of waiting forever on a dropped link, and a timeout is retried like any other transient error.
  • An interrupted IMAP or Maildir import keeps what it wrote, and a message that can't be read is recorded and skipped rather than ending the folder.
  • IMAP and EWS imports hold a bounded amount in memory, in batches capped by size as well as count.
  • The archive is created readable by its owner only, and an existing archive others can read is reported.
  • A dry run no longer writes to the target's address books.

Earlier releases (upstream Vandelay)

[1.0.11] - 2026-09-28

Added

Changed

  • Replaced base64 with encodify.

Fixed

  • IMAP: import failed after authenticating to Dovecot when a capability contained :, such as IMAPSIEVE=sieve://... (fixes #43)

[1.0.10] - 2026-08-27

Added

  • MS Exchange Graph:
    • OneDrive files import.
    • Graph contact photos, categories and IM addresses.
    • Graph event file attachments.

Changed

Fixed

  • MS Exchange Graph:
    • Import the default Contacts folder, recover series exceptions (fixes #39)
    • Graph deleted recurrence occurrences were not excluded.
    • bySetPosition was emitted on every Graph recurrence rule.
    • Graph read, flagged and category state was dropped.
    • A contact folder reachable by two paths aborted the run.
    • IM addresses were dropped on export for lacking a uri.
    • A redirect warning logged a OneDrive download credential.

[1.0.9] - 2026-08-22

Added

Changed

Fixed

  • IMAP mailbox names were decoded as Latin-1 instead of UTF-8, so non-ASCII folders failed to SELECT (#32 #37).
  • CalDAV discovery aborted when the given URL answered PROPFIND with 501 instead of trying the next discovery step (#35).
  • An unparsable receivedAt dropped the whole email, and a FileNode without nodeType was skipped (#36).
  • Maildir import took received_at from the file mtime, which does not survive a copy or restore (#38).
  • An EWS recurrence EndDate carrying a UTC offset produced a malformed until and an unbounded series (#33).
  • Export took the default calendar or address book away from a target account that already had one, and an archive could hold more than one default per type.
  • Exchange participant addresses were replaced by a synthetic URN whenever RoutingType was not SMTP, and a Graph X500 reference was emitted as a malformed mailto: (#33).

[1.0.8] - 2026-08-15

Added

  • A warning when the session advertises apiUrl, uploadUrl or downloadUrl on a different origin than the one connected to, naming both origins (#29).
  • The per-type report is printed when a run aborts.

Changed

Fixed

  • A transient serverUnavailable from Email/import permanently dropped that message from the migration instead of retrying it.
  • The Exchange Graph importer stored a whole converted event as a recurrenceOverrides value (#31).
  • Exporting a ContactCard carrying a photo blob failed against Stalwart, and a CalendarEvent enclosure sent as a blobId was silently dropped by the server; the bytes of both are now inlined as a base64 data: URI (#30).
  • A connection failure against the session-advertised apiUrl was reported as a per-type partial failure (exit 5, "consistent and resumable") instead of aborting (#29).
  • Session apiUrl, uploadUrl and downloadUrl given as relative references were used unresolved (#29).

[1.0.7] - 2026-07-26

Added

Changed

Fixed

  • Improve verbosity (#4 #19).
  • WebDAV import materialised the account root collection as a directory named after the account displayname (#18).
  • Report user friendly error message when urn:ietf:params:jmap:principals is not supported and no accountId is provided (#21).
  • Report which email failed to import when the blob is too large (#22).
  • IMAP import failed with "LIST mailbox name missing" when a mailbox name is a purely numeric unquoted atom (#26).

[1.0.6] - 2026-07-12

Added

Changed

Fixed

  • Self heal on blobNotFound errors when exporting data (#13).
  • Mapping existing special mailbox fails after alreadyExists response (#17).

[1.0.5] - 2026-06-27

Added

Changed

Fixed

  • Strict RFC822.SIZE == BODY[] length check discards good mail.

[1.0.4] - 2026-06-21

Added

Changed

Fixed

  • Include correct JMAP capabilities in using.
  • Failures are double-counted.

[1.0.3] - 2026-06-15

Added

Changed

Fixed

  • Mailbox roles must be unique per archive (#8).
  • Google takeout: Decode MIME-encoded values in X-Gmail-Labels (#7).

[1.0.2] - 2026-06-11

Added

Changed

Fixed

  • IMAP: Import fails with BAD on servers that advertise LIST-EXTENDED without SPECIAL-USE.
  • MS Exchange EWS: add support for version negotiation and other fixes (#6).

[1.0.1] - 2026-06-04

Added

Changed

Fixed

  • MS Exchange Graph: duplicate ids and incorrect JSCalendar mapping issues.

[1.0.0] - 2026-05-29

Added

  • Initial release.

Changed

Fixed