inbuxa/inbuxa-migrate
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Rewrite the README for inbuxa-migrate

Browse Source 
The README now says what the tool is in the suite's plain voice: the
archive in the middle, convergent reruns, --dry-run, the archive as a
backup, the sources it reads (Stalwart among the JMAP ones), the
INBUXA_MIGRATE_* credential variables, installing from the Gitea release
with SHA256SUMS, building and testing, and the license with the lineage
line. Upstream's logo, badges and install channels are gone.

The full command and flag reference moves to docs/usage.md, with the
renamed binary and variables. The CHANGELOG gains a section for this first
release, above upstream's history.
This commit is contained in:
jcoffey-dev
2026-09-30 10:15:07 -07:00
parent 42413563ef
commit bfea4a5467
2 changed files with 96 additions and 328 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+21 -1
View File
@@ -1,6 +1,26 @@
# Change Log # Change Log
All notable changes to this project will be documented in this file. This project adheres to [Semantic Versioning](http://semver.org/). All notable changes to this project are recorded here. Versions are dates:
release `v2026.9.30` is version `2026.9.30`.
## [Unreleased] -- 2026.9.30
### Changed
- Renamed to inbuxa-migrate: the binary, the crate, and the credential
variables, now `INBUXA_MIGRATE_PASSWORD`, `INBUXA_MIGRATE_TOKEN`,
`INBUXA_MIGRATE_EWS_CLIENT_SECRET` and `INBUXA_MIGRATE_GRAPH_TOKEN`. The old
names are not read. Archives written by Vandelay are not read either.
- Registry calls (`x:Account`, `x:Domain` and the other `x:` types) use the
capability the server advertises: `urn:inbuxa:jmap:registry` on inbuxa,
`urn:stalwart:jmap` on a Stalwart source. A server that advertises neither
gets a clear error instead of a request it never offered to accept.
- Released as Linux archives for amd64 and arm64 on the Gitea release page,
with `SHA256SUMS`. The npm, Homebrew, shell and PowerShell installers and
the MSI are gone.
---
# Earlier releases (upstream Vandelay)
## [1.0.11] - 2026-09-28 ## [1.0.11] - 2026-09-28
README.md
+74 -326
View File
@@ -1,355 +1,103 @@
<h1 align="center">Vandelay</h1> # inbuxa-migrate
<h3 align="center"> > [!NOTE]
The JMAP importer-exporter > Development happens on [git.coffeylabs.org/inbuxa/inbuxa-migrate](https://git.coffeylabs.org/inbuxa/inbuxa-migrate); the copy on GitHub is a read-only mirror.
<br/> > Report issues at **[git.coffeylabs.org/inbuxa/inbuxa-migrate/issues](https://git.coffeylabs.org/inbuxa/inbuxa-migrate/issues)**, and join discussions at **[community.coffeylabs.org](https://community.coffeylabs.org)**.
<sub>One-shot account migration and backup across JMAP, IMAP, CalDAV, CardDAV, WebDAV, ManageSieve, Maildir, Google Takeout and Microsoft Exchange.</sub>
</h3>
<br> One program that moves an account into **inbuxa**: its mail, calendars,
contacts, identities, Sieve scripts and files.
<p align="center"> inbuxa-migrate import imap … alice.sqlite an account into an archive
<a href="https://github.com/stalwartlabs/vandelay/actions/workflows/release.yml"><img src="https://img.shields.io/github/actions/workflow/status/stalwartlabs/vandelay/release.yml?style=flat-square" alt="build"></a> inbuxa-migrate inspect alice.sqlite what landed
&nbsp; inbuxa-migrate export … alice.sqlite the archive into inbuxa
<a href="https://github.com/stalwartlabs/vandelay/releases/latest"><img src="https://img.shields.io/github/v/release/stalwartlabs/vandelay?style=flat-square" alt="release"></a>
&nbsp;
<a href="https://www.npmjs.com/package/@stalwartlabs/vandelay"><img src="https://img.shields.io/npm/v/@stalwartlabs/vandelay?style=flat-square&logo=npm" alt="npm"></a>
&nbsp;
<a href="https://github.com/stalwartlabs/vandelay/releases"><img src="https://img.shields.io/github/downloads/stalwartlabs/vandelay/total?style=flat-square" alt="downloads"></a>
&nbsp;
<img src="https://img.shields.io/badge/platforms-macOS%20%7C%20Linux%20%7C%20Windows-blue?style=flat-square" alt="platforms">
&nbsp;
<a href="#license"><img src="https://img.shields.io/badge/license-MIT%2FApache--2.0-blue?style=flat-square" alt="license"></a>
</p>
<br> Between the two sides sits an archive: one SQLite file holding one account.
`import` fills it from the old server; `export` writes it into the new one.
Import and export never talk to each other, only to the archive, so the old
server can be gone before the new one exists.
<table> Both commands converge. An interrupted run picks up where it stopped, and a
<tr> later `import` into the same archive adds what arrived since. Every command
<td> takes `--dry-run` and then only reports the plan. `inspect` reads an archive
and changes nothing.
- Jerry: Well, what does **he** do? The archive is also a backup. Import on a schedule, keep the file, and
- George: He's an importer. restore it later with `export`. An archive remembers which account filled it
- Jerry: Just imports, no exports? and refuses a different one unless told otherwise.
- George: He's an **importer/exporter**, okay?
</td> Export speaks JMAP only. It writes into inbuxa, or into any JMAP server that
<td> advertises the types being written. The full command reference is in
<img src="assets/importer-exporter.jpg" alt="He's an importer-exporter."> [docs/usage.md](docs/usage.md).
</td>
</tr>
</table>
&nbsp; ## Sources
## About - **JMAP** -- any JMAP account, including an inbuxa or Stalwart server.
- **IMAP** -- mail only. Folders are chosen by name or by pattern.
- **CalDAV, CardDAV** -- calendars and events, address books and contacts.
- **WebDAV** -- a plain file collection, kept as a file tree.
- **ManageSieve** -- Sieve scripts only, with the active one recorded.
- **Maildir** -- a local Maildir++ tree. No network.
- **Google Takeout** -- the `.mbox`, `.ics` and `.vcf` files in a Takeout
export, or in any directory tree laid out that way.
- **Exchange Server** -- an on-premises mailbox, through EWS.
- **Exchange Online** -- through Microsoft Graph. EWS is being retired in
Exchange Online: blocked from 1 October 2026 unless an administrator allows
the client, and switched off on 1 April 2027. Public folders are only
reachable through EWS.
Vandelay is a one-shot account-migration utility for JMAP, the JMAP analogue of `imapsync` generalised to every JMAP data type (mail, contacts, calendars, identities, sieve scripts, file storage). It imports an account from a wide range of source protocols into a local SQLite "archive", then exports that archive into a target JMAP server. Import and export never talk to each other, only to the SQLite archive. One archive holds exactly one account. ## Credentials
Because the archive is a self-contained SQLite file that fully describes one account, vandelay doubles as a per-account backup tool: run an import on a schedule to capture a fresh snapshot, keep the resulting SQLite file as your backup, and restore it later by running an export against a JMAP target. Secrets are read from the environment, or asked for at a prompt:
## Features INBUXA_MIGRATE_PASSWORD a password for --auth-basic
INBUXA_MIGRATE_TOKEN a bearer token for --auth-bearer
INBUXA_MIGRATE_EWS_CLIENT_SECRET the OAuth client secret for app-only EWS
INBUXA_MIGRATE_GRAPH_TOKEN a Microsoft Graph access token
- **Many source protocols:** The command line takes them too, but a secret there ends up in the shell's
- JMAP history and in the process list. Use the environment or the prompt.
- IMAP
- CalDAV
- CardDAV
- WebDAV
- ManageSieve
- Maildir++
- Google Takeout
- Microsoft Exchange Server (on-premises) via EWS
- Microsoft Exchange Online via Graph
- **One target protocol:** JMAP, with type-by-type stateless re-matching on every run.
- **Convergent:** Re-running an interrupted import or export picks up where it left off without bookkeeping flags.
- **Multi-threaded, no async runtime:** Blocking HTTP with per-server concurrency caps respected automatically.
- **Content-addressed blobs:** Emails, sieve scripts and file-storage payloads are stored once by BLAKE3 hash and deduplicated across the archive.
- **Dry-run everywhere:** Every command supports `--dry-run` to compute the full plan without writing.
- **Source-change protection:** An archive remembers which account it was filled from; pointing it at a different one fails unless explicitly permitted.
- **Read-only inspection:** A built-in `inspect` command dumps any object type from an archive for verification.
## Install ## Installing
```sh Linux, on amd64 or arm64. Each release carries one archive per architecture
# macOS / Linux and a `SHA256SUMS` file:
curl --proto '=https' --tlsv1.2 -LsSf \
https://github.com/stalwartlabs/vandelay/releases/latest/download/vandelay-installer.sh | sh
# Homebrew base=https://git.coffeylabs.org/inbuxa/inbuxa-migrate/releases/latest/download
brew install stalwartlabs/tap/vandelay curl -fLO "$base/inbuxa-migrate-linux-amd64.tar.gz"
curl -fLO "$base/SHA256SUMS"
sha256sum --check --ignore-missing SHA256SUMS
tar -xzf inbuxa-migrate-linux-amd64.tar.gz
./inbuxa-migrate --version
# Windows On arm64, fetch `inbuxa-migrate-linux-arm64.tar.gz` instead. The binary is
powershell -ExecutionPolicy Bypass -c "irm https://github.com/stalwartlabs/vandelay/releases/latest/download/vandelay-installer.ps1 | iex" the whole program; put it anywhere on the path.
# npm ## Building and testing
npm install -g @stalwartlabs/vandelay
# From source cargo build --release the binary is target/release/inbuxa-migrate
cargo install --path . cargo test the default suite
```
A signed `.msi` is also published with each release. The default suite needs no network and no Docker. Unit tests and scripted
mocks stand in for JMAP, DAV, EWS and Graph servers.
## Quick start The live tests are marked `#[ignore]`. Each boots a throwaway container --
Stalwart, Dovecot, Cyrus, Radicale, Baikal or Apache `mod_dav` -- so they
need a working Docker daemon. Run them one binary at a time, and always with
one test thread:
A typical run is two commands, one to capture a source account into a local SQLite archive and one to push that archive into a JMAP target. cargo test --test sync_jmap -- --ignored --test-threads=1
```sh
# 1. Import an IMAP mailbox into a fresh archive.
export VANDELAY_PASSWORD='source-app-password'
vandelay import imap \
--url imaps://imap.example.com \
--auth-basic [email protected] \
alice.sqlite
# 2. Peek at what landed.
vandelay inspect alice.sqlite # per-type summary
vandelay inspect alice.sqlite mailbox # mailbox tree
vandelay inspect alice.sqlite email --limit 5
# 3. Push the archive into a target JMAP server.
export VANDELAY_PASSWORD='target-password'
vandelay export \
--url https://jmap.example.org \
--auth-basic [email protected] \
--account-name [email protected] \
alice.sqlite
```
Both commands are convergent: rerun either to resume an interrupted run, or rerun `import` later to pick up new mail since the last snapshot. Use `--dry-run` on either side to compute the full plan without writing.
## CLI quick reference
```
vandelay <import|export|inspect> [args...]
```
All actions accept a set of global flags (verbosity, worker pool size, retry policy, TLS handling) in addition to action-specific ones. The most useful are:
| Flag | Purpose |
| --- | --- |
| `-j, --threads <N>` | Worker pool size (default: logical CPUs). |
| `--dry-run` | Compute the full plan; perform no writes. |
| `-v`, `-vv`, `-vvv` | Increase log verbosity. |
| `-q, --quiet` | Warnings and errors only. |
| `--max-retries <N>` | Max retries per request on transient failures (default 5). |
| `--allow-invalid-certs` | Accept self-signed / invalid TLS certs. |
Credentials should be supplied via the `VANDELAY_PASSWORD` / `VANDELAY_TOKEN` / `VANDELAY_EWS_CLIENT_SECRET` / `VANDELAY_GRAPH_TOKEN` environment variables, or via an interactive prompt; passing them on the command line is supported but not recommended.
### Import
```
vandelay import <source> [source-args...] <ARCHIVE>
```
Reads a source account into the local SQLite `ARCHIVE` (created if absent). Every importer accepts `--allow-source-change` to override the archive's source-identity guard.
#### JMAP
```
vandelay import jmap \
--url <URL> \
(--auth-basic <USER> [--auth-password <PASS>] | --auth-bearer [TOKEN]) \
(--account-id <ID> | --account-name <NAME>) \
[--objects <list>] \
<ARCHIVE>
```
Imports a single JMAP account. `--objects` accepts a comma-separated list of object tokens (`mailbox,email,calendar,calendarevent,addressbook,contactcard,identity,sievescript,participantidentity,filenode`); default is everything the server advertises.
#### IMAP
```
vandelay import imap \
--url imap(s)://host[:port] \
(--auth-basic <USER> [--auth-password <PASS>] | --auth-bearer [TOKEN] --auth-user <USER>) \
[--include <REGEX>...] [--exclude <REGEX>...] [--exclude-special <ROLE>...] \
[--folder <NAME>...] [--subscribed-only] [--noautomap] \
[--include-deleted] [--allow-cleartext] [--compress] \
[--fetch-batch <N>] [--imap-connections <1..8>] \
<ARCHIVE>
```
Imports mail (and only mail) from any IMAP server. Folder selection is via `--include`/`--exclude` regexes (mutually exclusive with the exact-match `--folder`); `--exclude-special` drops by SPECIAL-USE role.
#### CalDAV
```
vandelay import caldav \
--url <http(s)://host[/path]> \
(--auth-basic <USER> [--auth-password <PASS>] | --auth-bearer [TOKEN]) \
[--allow-cleartext] [--dav-connections <1..8>] [--multiget-batch <N>] \
<ARCHIVE>
```
Discovers the user's CalDAV principal (or accepts a URL pointing straight at a calendar-home or calendar), then imports calendars and events.
#### CardDAV
```
vandelay import carddav \
--url <http(s)://host[/path]> \
(--auth-basic <USER> [--auth-password <PASS>] | --auth-bearer [TOKEN]) \
[--allow-cleartext] [--dav-connections <1..8>] [--multiget-batch <N>] \
<ARCHIVE>
```
Same shape as `caldav`, but for address books and contacts.
#### WebDAV
```
vandelay import webdav \
--url <http(s)://host[/path]> \
(--auth-basic <USER> [--auth-password <PASS>] | --auth-bearer [TOKEN]) \
[--allow-cleartext] [--dav-connections <1..8>] [--multiget-batch <N>] \
<ARCHIVE>
```
Imports a plain WebDAV file collection as a JMAP `FileNode` tree.
#### ManageSieve
```
vandelay import managesieve \
--url sieve(s)://host[:port] \
(--auth-basic <USER> [--auth-password <PASS>] | --auth-bearer [TOKEN] --auth-user <USER>) \
[--allow-cleartext] \
<ARCHIVE>
```
Imports sieve scripts only. Each script is content-addressed in the blob table; the active script is recorded.
#### Maildir
```
vandelay import maildir <MAILDIR> <ARCHIVE> \
[--include <REGEX>...] [--exclude <REGEX>...] [--folder <NAME>...] \
[--noautomap] [--include-deleted]
```
Reads a local Maildir++ tree (a directory with `cur/`, `new/`, `tmp/`). No network. Folder selection mirrors the IMAP importer.
#### Google Takeout
```
vandelay import takeout <PATH> <ARCHIVE> [--noautomap]
```
Scans a directory tree recursively for `.mbox`, `.ics` and `.vcf` files and imports them. Tailored to Google Takeout layouts but works on any such tree; system-label role assignment can be disabled with `--noautomap`.
#### Microsoft Exchange (EWS)
```
vandelay import exchange-ews \
[--url <EWS-ENDPOINT>] [--mailbox <SMTP>] \
[--mailbox-kind primary|archive|public-folders] \
(--auth-basic <USER> [--auth-password <PASS>] \
| --auth-bearer [TOKEN] [--ews-tenant <T> --ews-client-id <ID> \
(--ews-device-code | --ews-client-secret <SECRET>)]) \
[--ews-connections <1..8>] [--ews-getitem-batch <N>] [--ews-attachment-batch <N>] \
[--ews-no-syncfolderitems] \
<ARCHIVE>
```
Imports a mailbox via EWS from an on-premises Exchange Server. Autodiscover is used when `--url` is omitted (a `--mailbox` SMTP address is then required). Supports Basic, pre-acquired bearer, interactive device-code OAuth, and app-only client-credentials OAuth.
For Exchange Online, use `exchange-graph` instead. Microsoft is retiring EWS in Exchange Online: from 1 October 2026 it is blocked unless a tenant administrator sets `EwsEnabled` to `True` and adds the client id to `EwsAllowedAppIDs`, and on 1 April 2027 it is switched off for every tenant. On-premises Exchange Server is not affected.
#### Microsoft Exchange (Graph)
```
vandelay import exchange-graph \
(--client-id <UUID> [--tenant <ID>] | --access-token [TOKEN]) \
[--user <UPN|UUID>] \
[--mailbox-kind primary|archive] \
[--objects mail,calendar,contacts] \
[--event-body-format text|html] \
[--graph-connections <1..16>] [--top <1..1000>] \
<ARCHIVE>
```
Imports a mailbox from Exchange Online via Microsoft Graph. Without `--access-token`, the interactive device-code flow is used. `public-folders` is rejected here: Graph does not expose public folders, so they can only be imported with `exchange-ews`, which for Exchange Online is subject to the retirement described above.
### Export
```
vandelay export \
--url <URL> \
(--auth-basic <USER> [--auth-password <PASS>] | --auth-bearer [TOKEN]) \
(--account-id <ID> | --account-name <NAME>) \
[--objects <list>] [--prune [--yes]] \
<ARCHIVE>
```
Stateless re-export of `ARCHIVE` into a target JMAP server account. The default behaviour is upsert-only: matched items are updated, unmatched local items are created, but pre-existing target items not covered by the archive are left alone.
`--prune` enables destructive reconciliation: target objects that do not match anything in the archive are deleted. The confirmation prompt can be skipped with `--yes` for automation. Export speaks JMAP only; no other target protocols are currently supported.
### Inspect
```
vandelay inspect <ARCHIVE> [TYPE] [--limit <N>] [--offset <N>]
```
Read-only dump of a local archive. This command never opens a network connection and never writes to the archive.
- Omit `TYPE` for a per-type summary (counts of every object kind plus blob storage stats).
- Pass an object type to dump it: `mailbox`, `email`, `identity`, `sievescript`, `addressbook`, `contactcard`, `calendar`, `calendarevent`, `participantidentity`, `filenode`.
- `mailbox` and `filenode` render as a tree (`--limit`/`--offset` are ignored); all other types use a paginated list and respect `--limit` and `--offset`.
## Testing
The default suite is hermetic (unit tests plus `mockito`-scripted JMAP/DAV/EWS/Graph behaviours) and needs no network or Docker:
```sh
cargo build
cargo clippy --all-targets
cargo test
```
### Live and integration tests (Docker required)
Live integration tests against a Stalwart server and container-based tests against third-party servers (Dovecot, Cyrus, Radicale, Baikal, Apache `mod_dav`) are gated behind `--ignored`. **They require a running Docker daemon**: each test binary boots its own throwaway container via `testcontainers` (images are pulled automatically on first run), so Docker must be installed and `docker info` must succeed before invoking them.
Run them per binary, and always with `--test-threads=1`:
```sh
cargo test --test sync_jmap -- --ignored --test-threads=1 # live JMAP import/export/convergence/prune
cargo test --test sync_imap -- --ignored --test-threads=1
cargo test --test sync_managesieve -- --ignored --test-threads=1
cargo test --test sync_maildir -- --ignored --test-threads=1
cargo test --test sync_caldav -- --ignored --test-threads=1
cargo test --test sync_carddav -- --ignored --test-threads=1
cargo test --test sync_webdav -- --ignored --test-threads=1
cargo test --test live_stalwart -- --ignored --test-threads=1
cargo test --test seed_smoke -- --ignored --test-threads=1
cargo test --test seed_only -- --ignored --test-threads=1
# Third-party-server tests (one container each):
cargo test --test integration_radicale -- --ignored --test-threads=1
cargo test --test integration_baikal -- --ignored --test-threads=1
cargo test --test integration_webdav -- --ignored --test-threads=1
cargo test --test integration_dovecot -- --ignored --test-threads=1 cargo test --test integration_dovecot -- --ignored --test-threads=1
cargo test --test integration_cyrus -- --ignored --test-threads=1
# Slow tests The tests in a binary share one container and one disposable domain, so a
cargo test --test mock_jmap -- --ignored second thread would trip over the first. [docs/usage.md](docs/usage.md) lists
``` every live test binary.
`--test-threads=1` is mandatory, not just advisory: within a binary every test shares a single per-binary container, and each test provisions then tears down the same disposable `vandelay.org` domain (and opens the archive with SQLite `EXCLUSIVE` locking). Separate binaries are isolated (each boots its own container on dynamic host ports), so plain `cargo test --test <name>` invocations are safe to run one after another.
## License ## License
Licensed under either of Apache-2.0 OR MIT, at your option. The texts are in [LICENSES](LICENSES).
* Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0) Copyright (C) 2020, Stalwart Labs LLC<br>
* MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT) Copyright (C) 2026, John Coffey
at your option. Forked from Vandelay, originally developed by Stalwart Labs, and distributed
under the same Apache-2.0 OR MIT terms.
## Copyright
Copyright (C) 2020, Stalwart Labs LLC