plan --answers, --stay-on-url and --google-key-from-env #52
No files matched your search
@@ -11,6 +11,16 @@ candidates before a series ships (`27.1-rc.1`). The tag is the version with a
|
||||
- `migrate --progress json` writes one JSON event per line on stdout (a
|
||||
step begins or ends, an account ends, the run ends), for a program that
|
||||
runs a migration and shows its progress.
|
||||
- `plan --answers` writes the guide's plans from its answers in a file, so
|
||||
a program can ask the questions itself and run `migrate`. The answers
|
||||
hold no secrets.
|
||||
- `--stay-on-url` sends every request to the `--url` host, whatever the
|
||||
server's session advertises, for a server reached at its own loopback or
|
||||
through a tunnel. `import jmap`, `export`, `export-directory` and
|
||||
`import-directory` take it.
|
||||
- `import imap --google-key-from-env` reads the Google service account's
|
||||
key from `$INBUXA_MIGRATE_GOOGLE_KEY`, and a plan step can name it as
|
||||
the `googleKey` secret.
|
||||
|
||||
## [27.1-rc.7]
|
||||
|
||||
|
||||
@@ -84,6 +84,8 @@ Secrets are read from the environment, or asked for at a prompt:
|
||||
(INBUXA_MIGRATE_TOKEN still works)
|
||||
INBUXA_MIGRATE_EWS_CLIENT_SECRET the OAuth client secret for app-only EWS
|
||||
INBUXA_MIGRATE_GRAPH_TOKEN a Microsoft Graph access token
|
||||
INBUXA_MIGRATE_GOOGLE_KEY a Google service account's JSON key,
|
||||
for --google-key-from-env
|
||||
|
||||
The command line takes them too, but a secret there ends up in the shell's
|
||||
history and in the process list. Use the environment or the prompt.
|
||||
|
||||
+37
-2
@@ -13,7 +13,7 @@ and how to install it.
|
||||
## Commands and global flags
|
||||
|
||||
```
|
||||
inbuxa-migrate <import|export|export-imap|export-managesieve|export-caldav|export-carddav|export-files|export-directory|import-directory|migrate|guide|inspect> [args...]
|
||||
inbuxa-migrate <import|export|export-imap|export-managesieve|export-caldav|export-carddav|export-files|export-directory|import-directory|migrate|guide|plan|inspect> [args...]
|
||||
```
|
||||
|
||||
Every command takes a few global flags -- verbosity, worker pool size, retry
|
||||
@@ -38,7 +38,17 @@ is where on-premises Autodiscover looks, and then only the EWS endpoint
|
||||
Autodiscover finds.
|
||||
|
||||
Every other host is verified as usual, including a host the server
|
||||
redirects to or names for its API, uploads or downloads. The Microsoft and
|
||||
redirects to or names for its API, uploads or downloads.
|
||||
|
||||
### Staying on the address given
|
||||
|
||||
A JMAP server's session names the addresses for its API, uploads and
|
||||
downloads, and the tool uses them, warning when they're on another host
|
||||
than `--url`. `--stay-on-url` sends every request to the `--url` host
|
||||
instead, keeping each path. It's for a server reached at an address it
|
||||
doesn't advertise: its own loopback, an SSH tunnel, a node behind a load
|
||||
balancer. `import jmap`, `export`, `export-directory` and
|
||||
`import-directory` take it. The Microsoft and
|
||||
Google sign-in and cloud endpoints are always verified, with or without the
|
||||
flag: a certificate that fails there is an attack or a broken network, never
|
||||
a server to trust.
|
||||
@@ -112,6 +122,9 @@ and UID, and the rest of the folder carries on.
|
||||
signs in as a Workspace service account acting for the user `--auth-user`
|
||||
names, so no one's password is needed. In a plan that is
|
||||
`--url imaps://imap.gmail.com --google-key key.json --auth-user {email}`.
|
||||
`--google-key-from-env` reads the key's JSON from
|
||||
`$INBUXA_MIGRATE_GOOGLE_KEY` instead of a file; in a plan, the step names
|
||||
the variable that holds it as `"secrets": {"googleKey": "..."}`.
|
||||
|
||||
- In Google Cloud, create a service account and a JSON key for it.
|
||||
- In the Workspace Admin console, under domain-wide delegation, allow the
|
||||
@@ -735,6 +748,28 @@ directory picks up where it was.
|
||||
- `--no-open` prints the address instead of opening a browser.
|
||||
`--port` picks the port; by default any free one is used.
|
||||
|
||||
## Plans from answers
|
||||
|
||||
```
|
||||
inbuxa-migrate plan --answers <FILE> [--source-directory <DIR>] <DIR>
|
||||
```
|
||||
|
||||
Writes what the guide would, for a program that asks the questions itself
|
||||
and runs `migrate`: `<DIR>/plan.json`, `<DIR>/preview-plan.json` and,
|
||||
when accounts are to be created, the directory the setup reads. The
|
||||
answers are the guide's, as JSON: `source` (`kind`, `url`, `login`,
|
||||
`path`, `cleartext`, `tenant`, `clientId`, `keyFile`), `accounts`,
|
||||
`create`, and `target` (`url`, `stayOnUrl`, `allowInvalidCerts`).
|
||||
`--source-directory` is what `export-directory` wrote from a JMAP source,
|
||||
so its accounts are set up as they were.
|
||||
|
||||
The answers hold no secrets, and `plan` refuses them if they do. It says
|
||||
on stdout, as one JSON object, which plans it wrote, where the move's
|
||||
passwords will go, and which variables to set when running them:
|
||||
`MIGRATE_SOURCE_SECRET` for the source's password, API key, client secret
|
||||
or Google key (for Google, a `keyFile` left empty means the key is the
|
||||
secret), and `MIGRATE_INBUXA_TOKEN` for inbuxa's.
|
||||
|
||||
## Inspect
|
||||
|
||||
```
|
||||
|
||||
+99
-11
@@ -16,7 +16,7 @@ use crate::exchange_ews::types::MailboxKind;
|
||||
use crate::exchange_graph::types::{EventBodyFormat, MailboxKind as GraphMailboxKind, Surfaces};
|
||||
use crate::export_directory::ExportDirectoryConfig;
|
||||
use crate::export_files::ExportFilesConfig;
|
||||
use crate::guide::GuideConfig;
|
||||
use crate::guide::{GuideConfig, PlanConfig};
|
||||
use crate::import_directory::ImportDirectoryConfig;
|
||||
use crate::inspect::InspectConfig;
|
||||
use crate::jmap::account::AccountSelector;
|
||||
@@ -100,6 +100,10 @@ enum Command {
|
||||
about = "Walk through a move to inbuxa in the browser: a page on this machine writes the plan and runs it"
|
||||
)]
|
||||
Guide(Box<GuideArgs>),
|
||||
#[command(
|
||||
about = "Write the guide's plans from its answers, for a program that runs migrate itself"
|
||||
)]
|
||||
Plan(Box<PlanArgs>),
|
||||
#[command(about = "Show the contents of a local archive (read-only)")]
|
||||
Inspect(Box<InspectArgs>),
|
||||
}
|
||||
@@ -187,6 +191,12 @@ struct JmapImportArgs {
|
||||
#[arg(long, value_name = "URL", help = "Base URL or full JMAP session URL")]
|
||||
url: String,
|
||||
|
||||
#[arg(
|
||||
long,
|
||||
help = "Send every request to the --url host, even when the server's session names another (such as its public address)"
|
||||
)]
|
||||
stay_on_url: bool,
|
||||
|
||||
#[arg(long, value_name = "USER", help = "HTTP Basic user")]
|
||||
auth_basic: Option<String>,
|
||||
|
||||
@@ -235,7 +245,7 @@ struct JmapImportArgs {
|
||||
}
|
||||
|
||||
#[derive(Args)]
|
||||
#[command(group(clap::ArgGroup::new("auth").required(true).args(["auth_basic", "auth_bearer", "google_key"])))]
|
||||
#[command(group(clap::ArgGroup::new("auth").required(true).args(["auth_basic", "auth_bearer", "google_key", "google_key_from_env"])))]
|
||||
struct ImapImportArgs {
|
||||
#[arg(
|
||||
long,
|
||||
@@ -280,6 +290,12 @@ struct ImapImportArgs {
|
||||
)]
|
||||
google_key: Option<PathBuf>,
|
||||
|
||||
#[arg(
|
||||
long,
|
||||
help = "Like --google-key, with the key's JSON in $INBUXA_MIGRATE_GOOGLE_KEY"
|
||||
)]
|
||||
google_key_from_env: bool,
|
||||
|
||||
#[arg(long, help = "Permit credentials on imap:// without STARTTLS")]
|
||||
allow_cleartext: bool,
|
||||
|
||||
@@ -373,6 +389,12 @@ struct ExportArgs {
|
||||
#[arg(long, value_name = "URL", help = "Base URL or full JMAP session URL")]
|
||||
url: String,
|
||||
|
||||
#[arg(
|
||||
long,
|
||||
help = "Send every request to the --url host, even when the server's session names another (such as its public address)"
|
||||
)]
|
||||
stay_on_url: bool,
|
||||
|
||||
#[arg(long, value_name = "USER", help = "HTTP Basic user")]
|
||||
auth_basic: Option<String>,
|
||||
|
||||
@@ -580,6 +602,12 @@ struct ExportDirectoryArgs {
|
||||
#[arg(long, value_name = "URL", help = "Base URL or full JMAP session URL")]
|
||||
url: String,
|
||||
|
||||
#[arg(
|
||||
long,
|
||||
help = "Send every request to the --url host, even when the server's session names another (such as its public address)"
|
||||
)]
|
||||
stay_on_url: bool,
|
||||
|
||||
#[arg(long, value_name = "USER", help = "HTTP Basic user (an administrator)")]
|
||||
auth_basic: Option<String>,
|
||||
|
||||
@@ -641,6 +669,12 @@ struct ImportDirectoryArgs {
|
||||
)]
|
||||
url: String,
|
||||
|
||||
#[arg(
|
||||
long,
|
||||
help = "Send every request to the --url host, even when the server's session names another (such as its public address)"
|
||||
)]
|
||||
stay_on_url: bool,
|
||||
|
||||
#[arg(long, value_name = "USER", help = "HTTP Basic user (an administrator)")]
|
||||
auth_basic: Option<String>,
|
||||
|
||||
@@ -773,6 +807,29 @@ struct GuideArgs {
|
||||
dir: PathBuf,
|
||||
}
|
||||
|
||||
#[derive(Args)]
|
||||
struct PlanArgs {
|
||||
#[arg(
|
||||
long,
|
||||
value_name = "FILE",
|
||||
help = "The guide's answers, as JSON, without secrets"
|
||||
)]
|
||||
answers: PathBuf,
|
||||
|
||||
#[arg(
|
||||
long,
|
||||
value_name = "DIR",
|
||||
help = "A directory export-directory wrote from the source, to set up its accounts as they were"
|
||||
)]
|
||||
source_directory: Option<PathBuf>,
|
||||
|
||||
#[arg(
|
||||
value_name = "DIR",
|
||||
help = "Where the plans go; migrate runs them there"
|
||||
)]
|
||||
dir: PathBuf,
|
||||
}
|
||||
|
||||
#[derive(Args)]
|
||||
struct InspectArgs {
|
||||
#[arg(value_name = "ARCHIVE", help = "Local SQLite archive")]
|
||||
@@ -815,6 +872,7 @@ pub enum Action {
|
||||
ImportDirectory(ImportDirectoryConfig),
|
||||
Migrate(MigrateConfig),
|
||||
Guide(GuideConfig),
|
||||
Plan(PlanConfig),
|
||||
ExportImap(CommonConfig, ExportImapConfig),
|
||||
ExportManageSieve(CommonConfig, ExportManageSieveConfig),
|
||||
ExportDav(CommonConfig, ExportDavConfig),
|
||||
@@ -960,6 +1018,11 @@ impl Cli {
|
||||
open: !args.no_open,
|
||||
program: std::env::current_exe()?,
|
||||
})),
|
||||
Command::Plan(args) => Ok(Action::Plan(PlanConfig {
|
||||
answers: args.answers,
|
||||
source_directory: args.source_directory,
|
||||
dir: args.dir,
|
||||
})),
|
||||
Command::ExportImap(args) => resolve_export_imap(*args),
|
||||
Command::ExportManageSieve(args) => resolve_export_managesieve(*args),
|
||||
Command::ExportCaldav(args) => resolve_export_dav(*args, DavKindArg::Caldav),
|
||||
@@ -1225,6 +1288,7 @@ fn resolve_jmap_import(args: JmapImportArgs) -> Result<Action, Error> {
|
||||
common,
|
||||
ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: args.stay_on_url,
|
||||
url: args.url,
|
||||
auth,
|
||||
account,
|
||||
@@ -1237,8 +1301,14 @@ fn resolve_jmap_import(args: JmapImportArgs) -> Result<Action, Error> {
|
||||
|
||||
fn resolve_imap_import(args: ImapImportArgs) -> Result<Action, Error> {
|
||||
let common = common_config(&args.global, args.archive);
|
||||
let auth = match &args.google_key {
|
||||
Some(key) => {
|
||||
let auth = match (&args.google_key, args.google_key_from_env) {
|
||||
(None, false) => resolve_imap_auth(
|
||||
args.auth_basic.as_deref(),
|
||||
args.auth_password.as_deref(),
|
||||
args.auth_bearer.as_ref(),
|
||||
args.auth_user.as_deref(),
|
||||
)?,
|
||||
(path, _) => {
|
||||
if args.auth_password.is_some() {
|
||||
return Err(Error::Usage(
|
||||
"--auth-password is only valid together with --auth-basic".to_owned(),
|
||||
@@ -1247,19 +1317,16 @@ fn resolve_imap_import(args: ImapImportArgs) -> Result<Action, Error> {
|
||||
let user = args.auth_user.as_deref().ok_or_else(|| {
|
||||
Error::Usage("--auth-user names the account --google-key signs in as".to_owned())
|
||||
})?;
|
||||
let account = std::sync::Arc::new(crate::google::ServiceAccount::load(key)?);
|
||||
let account = std::sync::Arc::new(match path {
|
||||
Some(path) => crate::google::ServiceAccount::load(path)?,
|
||||
None => crate::google::ServiceAccount::from_env()?,
|
||||
});
|
||||
ImapAuth::Google(std::sync::Arc::new(crate::google::Delegated::new(
|
||||
account,
|
||||
user,
|
||||
crate::google::MAIL_SCOPE,
|
||||
)))
|
||||
}
|
||||
None => resolve_imap_auth(
|
||||
args.auth_basic.as_deref(),
|
||||
args.auth_password.as_deref(),
|
||||
args.auth_bearer.as_ref(),
|
||||
args.auth_user.as_deref(),
|
||||
)?,
|
||||
};
|
||||
if !args.folder.is_empty() && (!args.include.is_empty() || !args.exclude.is_empty()) {
|
||||
return Err(Error::Usage(
|
||||
@@ -1353,6 +1420,7 @@ fn resolve_export(args: ExportArgs) -> Result<Action, Error> {
|
||||
common,
|
||||
ExportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: args.stay_on_url,
|
||||
url: args.url,
|
||||
auth,
|
||||
account,
|
||||
@@ -1436,6 +1504,7 @@ fn resolve_export_directory(args: ExportDirectoryArgs) -> Result<Action, Error>
|
||||
args.auth_bearer.as_ref(),
|
||||
)?;
|
||||
Ok(Action::ExportDirectory(ExportDirectoryConfig {
|
||||
stay_on_url: args.stay_on_url,
|
||||
url: args.url,
|
||||
auth,
|
||||
dir: args.dir,
|
||||
@@ -1454,6 +1523,7 @@ fn resolve_import_directory(args: ImportDirectoryArgs) -> Result<Action, Error>
|
||||
args.auth_bearer.as_ref(),
|
||||
)?;
|
||||
Ok(Action::ImportDirectory(ImportDirectoryConfig {
|
||||
stay_on_url: args.stay_on_url,
|
||||
url: args.url,
|
||||
auth,
|
||||
dir: args.dir,
|
||||
@@ -2290,6 +2360,24 @@ mod tests {
|
||||
Err(Error::Usage(m)) => assert!(m.contains("/nonexistent/k.json"), "{m}"),
|
||||
_ => panic!("a missing key file is a usage error"),
|
||||
}
|
||||
assert!(parse(&["--google-key", "k.json", "--google-key-from-env"]).is_err());
|
||||
unsafe { env::remove_var(crate::google::KEY_VAR) };
|
||||
match parse(&["--google-key-from-env", "--auth-user", "a@x"])
|
||||
.unwrap()
|
||||
.resolve()
|
||||
{
|
||||
Err(Error::Usage(m)) => assert!(m.contains(crate::google::KEY_VAR), "{m}"),
|
||||
_ => panic!("an unset key variable is a usage error"),
|
||||
}
|
||||
unsafe { env::set_var(crate::google::KEY_VAR, r#"{"type":"user"}"#) };
|
||||
match parse(&["--google-key-from-env", "--auth-user", "a@x"])
|
||||
.unwrap()
|
||||
.resolve()
|
||||
{
|
||||
Err(Error::Usage(m)) => assert!(m.contains("not a service account"), "{m}"),
|
||||
_ => panic!("a key that isn't a service account's is a usage error"),
|
||||
}
|
||||
unsafe { env::remove_var(crate::google::KEY_VAR) };
|
||||
}
|
||||
|
||||
fn webdav_source_change(extra: &[&str]) -> Result<SourceChange, clap::Error> {
|
||||
|
||||
@@ -89,6 +89,8 @@ pub struct ExportDirectoryConfig {
|
||||
pub dry_run: bool,
|
||||
pub max_retries: u32,
|
||||
pub allow_invalid_certs: bool,
|
||||
/// `--stay-on-url`: every request goes to `url`'s host.
|
||||
pub stay_on_url: bool,
|
||||
pub logger: Logger,
|
||||
}
|
||||
|
||||
@@ -101,7 +103,10 @@ pub fn run(config: ExportDirectoryConfig) -> Result<Summary, Error> {
|
||||
CertOverride::for_url(config.allow_invalid_certs, &config.url),
|
||||
);
|
||||
client.set_logger(logger);
|
||||
let session = Session::discover(&client, &config.url)?;
|
||||
let mut session = Session::discover(&client, &config.url)?;
|
||||
if config.stay_on_url {
|
||||
session = session.kept_on(&config.url);
|
||||
}
|
||||
for mismatch in session.origin_mismatches(&config.url) {
|
||||
logger.warn(&mismatch.to_string());
|
||||
}
|
||||
|
||||
@@ -49,7 +49,20 @@ impl fmt::Debug for ServiceAccount {
|
||||
}
|
||||
}
|
||||
|
||||
/// Where `--google-key-from-env` reads the key: the JSON itself, so a
|
||||
/// program running the tool needn't write it to a file.
|
||||
pub const KEY_VAR: &str = "INBUXA_MIGRATE_GOOGLE_KEY";
|
||||
|
||||
impl ServiceAccount {
|
||||
/// The key in [`KEY_VAR`].
|
||||
pub fn from_env() -> Result<Self, Error> {
|
||||
let text = std::env::var(KEY_VAR)
|
||||
.ok()
|
||||
.filter(|t| !t.trim().is_empty())
|
||||
.ok_or_else(|| Error::Usage(format!("--google-key-from-env: ${KEY_VAR} is not set")))?;
|
||||
Self::parse(&text).map_err(|e| Error::Usage(format!("${KEY_VAR}: {e}")))
|
||||
}
|
||||
|
||||
pub fn load(path: &Path) -> Result<Self, Error> {
|
||||
let text = std::fs::read_to_string(path)
|
||||
.map_err(|e| Error::Usage(format!("{}: {e}", path.display())))?;
|
||||
|
||||
+122
-31
@@ -48,10 +48,15 @@ pub enum SourceKind {
|
||||
Google,
|
||||
}
|
||||
|
||||
impl SourceKind {
|
||||
/// Whether the source signs in with a secret typed into the page.
|
||||
fn has_secret(self) -> bool {
|
||||
matches!(self, Self::Imap | Self::Jmap | Self::Microsoft)
|
||||
impl Source {
|
||||
/// Whether the source signs in with a secret typed into the page. A
|
||||
/// Google key given as a secret, not a file, is one too.
|
||||
pub fn has_secret(&self) -> bool {
|
||||
match self.kind {
|
||||
SourceKind::Imap | SourceKind::Jmap | SourceKind::Microsoft => true,
|
||||
SourceKind::Google => self.key_file.trim().is_empty(),
|
||||
SourceKind::Maildir => false,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -71,9 +76,10 @@ pub struct Source {
|
||||
/// Microsoft 365: the Entra tenant and the app's client id.
|
||||
pub tenant: String,
|
||||
pub client_id: String,
|
||||
/// Google: the service account's JSON key file.
|
||||
/// Google: the service account's JSON key file. Empty when the key
|
||||
/// itself is the secret, as from a program running the tool.
|
||||
pub key_file: String,
|
||||
/// The password, API key or client secret.
|
||||
/// The password, API key, client secret or Google key.
|
||||
#[serde(skip_serializing)]
|
||||
pub secret: String,
|
||||
}
|
||||
@@ -82,15 +88,34 @@ pub struct Source {
|
||||
#[serde(rename_all = "camelCase", default)]
|
||||
pub struct Target {
|
||||
pub url: String,
|
||||
/// `--stay-on-url`: for a server reached at an address other than the
|
||||
/// one it advertises, such as its own loopback.
|
||||
pub stay_on_url: bool,
|
||||
/// `--allow-invalid-certs`, for the same.
|
||||
pub allow_invalid_certs: bool,
|
||||
#[serde(skip_serializing)]
|
||||
pub token: String,
|
||||
}
|
||||
|
||||
impl Target {
|
||||
/// The flags every step that writes into inbuxa takes.
|
||||
fn flags(&self) -> Vec<&'static str> {
|
||||
let mut flags = Vec::new();
|
||||
if self.stay_on_url {
|
||||
flags.push("--stay-on-url");
|
||||
}
|
||||
if self.allow_invalid_certs {
|
||||
flags.push("--allow-invalid-certs");
|
||||
}
|
||||
flags
|
||||
}
|
||||
}
|
||||
|
||||
impl Form {
|
||||
/// The secrets a run needs, by variable.
|
||||
pub fn env(&self) -> Vec<(&'static str, String)> {
|
||||
let mut env = vec![(TARGET_TOKEN_VAR, self.target.token.clone())];
|
||||
if self.source.kind.has_secret() {
|
||||
if self.source.has_secret() {
|
||||
env.push((SOURCE_SECRET_VAR, self.source.secret.clone()));
|
||||
}
|
||||
env
|
||||
@@ -107,8 +132,26 @@ impl Form {
|
||||
}
|
||||
}
|
||||
|
||||
/// Everything a plan needs, or what's missing, in the page's words.
|
||||
/// Everything a run needs, or what's missing, in the page's words.
|
||||
pub fn check(&self) -> Result<(), String> {
|
||||
self.check_answers()?;
|
||||
let s = &self.source;
|
||||
if s.has_secret() && s.secret.is_empty() {
|
||||
return Err(match s.kind {
|
||||
SourceKind::Microsoft => "Give the app's client secret.".into(),
|
||||
SourceKind::Google => "Give the service account's key file.".into(),
|
||||
_ => "Give the old server's password or API key.".into(),
|
||||
});
|
||||
}
|
||||
if self.target.token.is_empty() {
|
||||
return Err("Give the inbuxa server's address and an administrator's API key.".into());
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Everything a plan needs, secrets aside: a plan only names the
|
||||
/// variables they'll be in.
|
||||
pub fn check_answers(&self) -> Result<(), String> {
|
||||
let s = &self.source;
|
||||
match s.kind {
|
||||
SourceKind::Imap | SourceKind::Jmap if s.url.trim().is_empty() => {
|
||||
@@ -127,22 +170,12 @@ impl Form {
|
||||
{
|
||||
return Err("Give the Microsoft 365 tenant and the app's client id.".into());
|
||||
}
|
||||
SourceKind::Google if s.key_file.trim().is_empty() => {
|
||||
return Err("Give the service account's key file.".into());
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
if s.kind.has_secret() && s.secret.is_empty() {
|
||||
return Err(if s.kind == SourceKind::Microsoft {
|
||||
"Give the app's client secret.".into()
|
||||
} else {
|
||||
"Give the old server's password or API key.".into()
|
||||
});
|
||||
}
|
||||
if self.accounts.is_empty() {
|
||||
return Err("List the accounts to move.".into());
|
||||
}
|
||||
if self.target.url.trim().is_empty() || self.target.token.is_empty() {
|
||||
if self.target.url.trim().is_empty() {
|
||||
return Err("Give the inbuxa server's address and an administrator's API key.".into());
|
||||
}
|
||||
Ok(())
|
||||
@@ -180,6 +213,11 @@ impl Form {
|
||||
args.extend(["--user", "{email}"]);
|
||||
secrets = json!({"clientSecret": SOURCE_SECRET_VAR});
|
||||
}
|
||||
SourceKind::Google if s.key_file.trim().is_empty() => {
|
||||
args.extend(["import", "imap", "--url", GMAIL_IMAP]);
|
||||
args.extend(["--google-key-from-env", "--auth-user", "{email}"]);
|
||||
secrets = json!({"googleKey": SOURCE_SECRET_VAR});
|
||||
}
|
||||
SourceKind::Google => {
|
||||
args.extend(["import", "imap", "--url", GMAIL_IMAP]);
|
||||
args.extend(["--google-key", s.key_file.trim(), "--auth-user", "{email}"]);
|
||||
@@ -190,24 +228,25 @@ impl Form {
|
||||
}
|
||||
|
||||
fn export_step(&self) -> Value {
|
||||
json!({
|
||||
"name": "export",
|
||||
"args": ["export", "--url", self.target.url.trim(), "--auth-bearer",
|
||||
"--account-name", "{email}", "{archive}"],
|
||||
"secrets": {"token": TARGET_TOKEN_VAR},
|
||||
})
|
||||
let mut args = vec!["export", "--url", self.target.url.trim(), "--auth-bearer"];
|
||||
args.extend(self.target.flags());
|
||||
args.extend(["--account-name", "{email}", "{archive}"]);
|
||||
json!({"name": "export", "args": args, "secrets": {"token": TARGET_TOKEN_VAR}})
|
||||
}
|
||||
|
||||
fn setup(&self, passwords: &str) -> Vec<Value> {
|
||||
if !self.create {
|
||||
return Vec::new();
|
||||
}
|
||||
vec![json!({
|
||||
"name": "accounts",
|
||||
"args": ["import-directory", "--url", self.target.url.trim(), "--auth-bearer",
|
||||
"--passwords", passwords, DIRECTORY],
|
||||
"secrets": {"token": TARGET_TOKEN_VAR},
|
||||
})]
|
||||
let mut args = vec![
|
||||
"import-directory",
|
||||
"--url",
|
||||
self.target.url.trim(),
|
||||
"--auth-bearer",
|
||||
];
|
||||
args.extend(self.target.flags());
|
||||
args.extend(["--passwords", passwords, DIRECTORY]);
|
||||
vec![json!({"name": "accounts", "args": args, "secrets": {"token": TARGET_TOKEN_VAR}})]
|
||||
}
|
||||
|
||||
/// The plan for the move: set up the accounts if asked, then read each
|
||||
@@ -356,6 +395,7 @@ mod tests {
|
||||
target: Target {
|
||||
url: "https://mail.example.org".into(),
|
||||
token: "t0ken".into(),
|
||||
..Target::default()
|
||||
},
|
||||
parallel: 4,
|
||||
}
|
||||
@@ -398,6 +438,57 @@ mod tests {
|
||||
assert_eq!(plan["steps"][1]["secrets"]["token"], TARGET_TOKEN_VAR);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_server_on_its_own_loopback_stays_there_in_every_inbuxa_step() {
|
||||
let mut f = form(SourceKind::Imap);
|
||||
f.target.url = "https://127.0.0.1:443".into();
|
||||
f.target.stay_on_url = true;
|
||||
f.target.allow_invalid_certs = true;
|
||||
let plan = f.plan("p.csv");
|
||||
for args in [&plan["setup"][0]["args"], &plan["steps"][1]["args"]] {
|
||||
assert_eq!(args[4], "--stay-on-url", "{args}");
|
||||
assert_eq!(args[5], "--allow-invalid-certs", "{args}");
|
||||
}
|
||||
let import = plan["steps"][0]["args"].to_string();
|
||||
assert!(!import.contains("--stay-on-url"), "{import}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_google_key_is_a_file_or_the_secret() {
|
||||
let mut f = form(SourceKind::Google);
|
||||
f.source.key_file = "sa.json".into();
|
||||
f.source.secret.clear();
|
||||
assert_eq!(f.check(), Ok(()));
|
||||
let step = &f.plan("p.csv")["steps"][0];
|
||||
assert_eq!(step["args"][4], "--google-key");
|
||||
assert_eq!(step["secrets"], json!({}));
|
||||
assert_eq!(f.env().len(), 1, "only the target's token");
|
||||
|
||||
f.source.key_file.clear();
|
||||
assert_eq!(f.check_answers(), Ok(()));
|
||||
assert_eq!(
|
||||
f.check(),
|
||||
Err("Give the service account's key file.".into())
|
||||
);
|
||||
f.source.secret = "{\"type\":\"service_account\"}".into();
|
||||
assert_eq!(f.check(), Ok(()));
|
||||
let step = &f.plan("p.csv")["steps"][0];
|
||||
assert_eq!(step["args"][4], "--google-key-from-env");
|
||||
assert_eq!(step["secrets"]["googleKey"], SOURCE_SECRET_VAR);
|
||||
assert!(f.env().iter().any(|(var, _)| *var == SOURCE_SECRET_VAR));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn answers_are_checked_without_their_secrets() {
|
||||
let mut f = form(SourceKind::Imap);
|
||||
f.source.secret.clear();
|
||||
f.target.token.clear();
|
||||
assert_eq!(f.check_answers(), Ok(()));
|
||||
assert!(f.check().is_err());
|
||||
f.target.url.clear();
|
||||
assert!(f.check_answers().is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_jmap_source_acts_for_each_account_by_basic_or_by_key() {
|
||||
let mut f = form(SourceKind::Jmap);
|
||||
|
||||
+92
-25
@@ -54,6 +54,14 @@ pub struct GuideConfig {
|
||||
pub program: PathBuf,
|
||||
}
|
||||
|
||||
/// `plan`: the plans the guide would write, from its answers in a file.
|
||||
pub struct PlanConfig {
|
||||
pub answers: PathBuf,
|
||||
/// What `export-directory` read from the source, if it was read.
|
||||
pub source_directory: Option<PathBuf>,
|
||||
pub dir: PathBuf,
|
||||
}
|
||||
|
||||
#[derive(Clone, Copy, PartialEq, Eq)]
|
||||
enum Kind {
|
||||
Directory,
|
||||
@@ -151,6 +159,86 @@ pub fn run(config: GuideConfig) -> Result<(), Error> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// `plan`: writes the plan, the preview's plan and the directory the setup
|
||||
/// reads, then says on stdout, as JSON, which files to run. The answers
|
||||
/// carry no secrets; the runs get them through the variables the plans
|
||||
/// name.
|
||||
pub fn write_plans(config: PlanConfig) -> Result<(), Error> {
|
||||
let usage = |e: String| Error::Usage(format!("{}: {e}", config.answers.display()));
|
||||
let text = fs::read_to_string(&config.answers).map_err(|e| usage(e.to_string()))?;
|
||||
let f: Form = serde_json::from_str(&text).map_err(|e| usage(e.to_string()))?;
|
||||
if !f.source.secret.is_empty() || !f.target.token.is_empty() {
|
||||
return Err(usage(format!(
|
||||
"the answers hold a secret; leave it out, and give it to migrate in \
|
||||
${} or ${}",
|
||||
form::SOURCE_SECRET_VAR,
|
||||
form::TARGET_TOKEN_VAR
|
||||
)));
|
||||
}
|
||||
f.check_answers().map_err(usage)?;
|
||||
let dir = std::path::absolute(&config.dir)?;
|
||||
private_dir(&dir)?;
|
||||
let written =
|
||||
write_plan_files(&dir, &f, config.source_directory.as_deref()).map_err(Error::Usage)?;
|
||||
let _ = writeln!(
|
||||
std::io::stdout(),
|
||||
"{}",
|
||||
json!({
|
||||
"plan": written.plan,
|
||||
"preview": written.preview,
|
||||
"passwords": written.passwords,
|
||||
"accounts": f.accounts.len(),
|
||||
"secrets": {
|
||||
"source": f.source.has_secret().then_some(form::SOURCE_SECRET_VAR),
|
||||
"target": form::TARGET_TOKEN_VAR,
|
||||
},
|
||||
})
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// What [`write_plan_files`] wrote, relative to the directory.
|
||||
struct Written {
|
||||
plan: &'static str,
|
||||
preview: &'static str,
|
||||
/// Where the move's setup puts new passwords, if it sets up accounts.
|
||||
passwords: Option<String>,
|
||||
}
|
||||
|
||||
/// The plan, the preview's plan and, when accounts are to be set up, the
|
||||
/// directory for it: the source's, if `source` is what `export-directory`
|
||||
/// read from it.
|
||||
fn write_plan_files(dir: &Path, f: &Form, source: Option<&Path>) -> Result<Written, String> {
|
||||
let write = |name: &str, value: &Value| {
|
||||
let path = dir.join(name);
|
||||
write_private(&path, &serde_json::to_vec_pretty(value).unwrap_or_default())
|
||||
.map_err(|e| format!("{} couldn't be written: {e}", path.display()))
|
||||
};
|
||||
if f.create {
|
||||
let source = match source {
|
||||
Some(d) => {
|
||||
let text = fs::read_to_string(d.join("directory.json"))
|
||||
.map_err(|e| format!("{}: {e}", d.display()))?;
|
||||
Some(serde_json::from_str::<Value>(&text).map_err(|e| e.to_string())?)
|
||||
}
|
||||
None => None,
|
||||
};
|
||||
private_dir(&dir.join(form::DIRECTORY)).map_err(|e| e.to_string())?;
|
||||
write(
|
||||
&format!("{}/directory.json", form::DIRECTORY),
|
||||
&form::directory(source.as_ref(), &f.accounts),
|
||||
)?;
|
||||
}
|
||||
let passwords = format!("passwords-{}.csv", stamp());
|
||||
write("preview-plan.json", &f.preview_plan())?;
|
||||
write("plan.json", &f.plan(&passwords))?;
|
||||
Ok(Written {
|
||||
plan: "plan.json",
|
||||
preview: "preview-plan.json",
|
||||
passwords: f.create.then_some(passwords),
|
||||
})
|
||||
}
|
||||
|
||||
fn open_browser(url: &str) -> bool {
|
||||
#[cfg(target_os = "macos")]
|
||||
let mut command = Command::new("open");
|
||||
@@ -500,33 +588,12 @@ impl Guide {
|
||||
/// Writes the plan, and the directory it sets up, for `migrate`.
|
||||
fn plan_run(&self, f: &Form, kind: Kind) -> Result<Prepared, String> {
|
||||
f.check()?;
|
||||
let write = |name: &str, value: &Value| {
|
||||
let path = self.dir.join(name);
|
||||
write_private(&path, &serde_json::to_vec_pretty(value).unwrap_or_default())
|
||||
.map_err(|e| format!("{} couldn't be written: {e}", path.display()))
|
||||
};
|
||||
if f.create {
|
||||
let source = lock(&self.state).directory.clone();
|
||||
let source = match source {
|
||||
Some(d) => {
|
||||
let text = fs::read_to_string(d.join("directory.json"))
|
||||
.map_err(|e| format!("{}: {e}", d.display()))?;
|
||||
Some(serde_json::from_str::<Value>(&text).map_err(|e| e.to_string())?)
|
||||
}
|
||||
None => None,
|
||||
};
|
||||
private_dir(&self.dir.join(form::DIRECTORY)).map_err(|e| e.to_string())?;
|
||||
write(
|
||||
&format!("{}/directory.json", form::DIRECTORY),
|
||||
&form::directory(source.as_ref(), &f.accounts),
|
||||
)?;
|
||||
}
|
||||
let source = lock(&self.state).directory.clone();
|
||||
let written = write_plan_files(&self.dir, f, source.as_deref())?;
|
||||
let plan = if kind == Kind::Preview {
|
||||
write("preview-plan.json", &f.preview_plan())?;
|
||||
"preview-plan.json"
|
||||
written.preview
|
||||
} else {
|
||||
write("plan.json", &f.plan(&format!("passwords-{}.csv", stamp())))?;
|
||||
"plan.json"
|
||||
written.plan
|
||||
};
|
||||
let mut args: Vec<String> = ["migrate", "--plan", plan, "--parallel"]
|
||||
.map(str::to_owned)
|
||||
|
||||
@@ -44,6 +44,8 @@ pub struct ImportDirectoryConfig {
|
||||
pub dry_run: bool,
|
||||
pub max_retries: u32,
|
||||
pub allow_invalid_certs: bool,
|
||||
/// `--stay-on-url`: every request goes to `url`'s host.
|
||||
pub stay_on_url: bool,
|
||||
pub logger: Logger,
|
||||
}
|
||||
|
||||
@@ -481,7 +483,10 @@ pub fn run(config: ImportDirectoryConfig) -> Result<Summary, Error> {
|
||||
CertOverride::for_url(config.allow_invalid_certs, &config.url),
|
||||
);
|
||||
client.set_logger(logger);
|
||||
let session = Session::discover(&client, &config.url)?;
|
||||
let mut session = Session::discover(&client, &config.url)?;
|
||||
if config.stay_on_url {
|
||||
session = session.kept_on(&config.url);
|
||||
}
|
||||
for mismatch in session.origin_mismatches(&config.url) {
|
||||
logger.warn(&mismatch.to_string());
|
||||
}
|
||||
|
||||
+5
-1
@@ -1,5 +1,6 @@
|
||||
/*
|
||||
* SPDX-FileCopyrightText: 2020 Stalwart Labs LLC <[email protected]>
|
||||
* SPDX-FileCopyrightText: 2026 Coffey Labs LLC <[email protected]>
|
||||
*
|
||||
* SPDX-License-Identifier: Apache-2.0 OR MIT
|
||||
*/
|
||||
@@ -30,7 +31,10 @@ impl Connected {
|
||||
pub fn prepare(ctx: &Context, connect: &ConnectConfig) -> Result<Connected, Error> {
|
||||
ctx.client.set_logger(ctx.common.logger);
|
||||
|
||||
let session = Session::discover(&ctx.client, &connect.url)?;
|
||||
let mut session = Session::discover(&ctx.client, &connect.url)?;
|
||||
if connect.stay_on_url {
|
||||
session = session.kept_on(&connect.url);
|
||||
}
|
||||
for mismatch in session.origin_mismatches(&connect.url) {
|
||||
ctx.common.logger.warn(&mismatch.to_string());
|
||||
}
|
||||
|
||||
@@ -183,6 +183,28 @@ impl Session {
|
||||
self
|
||||
}
|
||||
|
||||
/// The session with every URL moved onto `connect_url`'s origin: for
|
||||
/// `--stay-on-url`, when the address that reaches the server isn't the
|
||||
/// one it advertises, such as its own loopback.
|
||||
pub fn kept_on(mut self, connect_url: &str) -> Session {
|
||||
let Some(origin) = origin_of(connect_url) else {
|
||||
return self;
|
||||
};
|
||||
for url in [
|
||||
&mut self.api_url,
|
||||
&mut self.upload_url,
|
||||
&mut self.download_url,
|
||||
] {
|
||||
if let Some(after) = url.find("://").map(|at| at + 3) {
|
||||
let path = url[after..]
|
||||
.find('/')
|
||||
.map_or(url.len(), |slash| after + slash);
|
||||
*url = format!("{origin}{}", &url[path..]);
|
||||
}
|
||||
}
|
||||
self
|
||||
}
|
||||
|
||||
pub fn origin_mismatches(&self, connect_url: &str) -> Vec<OriginMismatch> {
|
||||
let Some(connect_origin) = origin_of(connect_url) else {
|
||||
return Vec::new();
|
||||
@@ -609,4 +631,29 @@ mod tests {
|
||||
"https://h.example/.well-known/jmap"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn kept_on_moves_every_url_to_the_connect_origin() {
|
||||
let raw = r#"{"apiUrl":"https://mail.example/jmap/",
|
||||
"uploadUrl":"https://mail.example/jmap/upload/{accountId}/",
|
||||
"downloadUrl":"https://mail.example:443/jmap/download/{accountId}/{blobId}/{name}?accept={type}",
|
||||
"capabilities":{},"accounts":{}}"#;
|
||||
let session = parse_session(raw)
|
||||
.unwrap()
|
||||
.kept_on("http://127.0.0.1:8080/");
|
||||
assert_eq!(session.api_url, "http://127.0.0.1:8080/jmap/");
|
||||
assert_eq!(
|
||||
session.upload_url,
|
||||
"http://127.0.0.1:8080/jmap/upload/{accountId}/"
|
||||
);
|
||||
assert_eq!(
|
||||
session.download_url,
|
||||
"http://127.0.0.1:8080/jmap/download/{accountId}/{blobId}/{name}?accept={type}"
|
||||
);
|
||||
assert!(
|
||||
session
|
||||
.origin_mismatches("http://127.0.0.1:8080")
|
||||
.is_empty()
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -156,6 +156,12 @@ fn run() -> i32 {
|
||||
Err(err) => fail(&err),
|
||||
};
|
||||
}
|
||||
Action::Plan(config) => {
|
||||
return match guide::write_plans(config) {
|
||||
Ok(()) => 0,
|
||||
Err(err) => fail(&err),
|
||||
};
|
||||
}
|
||||
Action::Inspect(config) => {
|
||||
return match inspect::run(config) {
|
||||
Ok(()) => 0,
|
||||
|
||||
@@ -30,6 +30,7 @@ pub const SECRET_VARS: &[&str] = &[
|
||||
"INBUXA_MIGRATE_EWS_CLIENT_SECRET",
|
||||
"INBUXA_MIGRATE_CLIENT_SECRET",
|
||||
"INBUXA_MIGRATE_GRAPH_TOKEN",
|
||||
crate::google::KEY_VAR,
|
||||
];
|
||||
|
||||
const ACCOUNT_COMMANDS: &[&str] = &[
|
||||
@@ -85,6 +86,8 @@ struct Secrets {
|
||||
token: Option<String>,
|
||||
client_secret: Option<String>,
|
||||
graph_token: Option<String>,
|
||||
/// A Google service account's JSON key, for `--google-key-from-env`.
|
||||
google_key: Option<String>,
|
||||
}
|
||||
|
||||
pub struct Plan {
|
||||
@@ -208,6 +211,7 @@ fn step_of(raw: RawStep, per_account: bool, names: &mut HashSet<String>) -> Resu
|
||||
("INBUXA_MIGRATE_API_KEY", secrets.token),
|
||||
("INBUXA_MIGRATE_EWS_CLIENT_SECRET", secrets.client_secret),
|
||||
("INBUXA_MIGRATE_GRAPH_TOKEN", secrets.graph_token),
|
||||
(crate::google::KEY_VAR, secrets.google_key),
|
||||
] {
|
||||
let Some(from) = from else { continue };
|
||||
match std::env::var(&from) {
|
||||
|
||||
@@ -63,6 +63,9 @@ pub struct ConnectConfig {
|
||||
pub url: String,
|
||||
pub auth: Auth,
|
||||
pub account: AccountSelector,
|
||||
/// `--stay-on-url`: every request goes to `url`'s host, whatever the
|
||||
/// session advertises.
|
||||
pub stay_on_url: bool,
|
||||
}
|
||||
|
||||
pub struct ImportConfig {
|
||||
|
||||
@@ -176,6 +176,7 @@ fn run_export(archive: &Path, base: &str, threads: usize) -> TypeCounts {
|
||||
},
|
||||
ExportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base.to_owned(),
|
||||
auth: Auth::Basic {
|
||||
user: "u".into(),
|
||||
|
||||
@@ -153,6 +153,7 @@ fn out_dir(name: &str) -> PathBuf {
|
||||
|
||||
fn config(registry: &Registry, dir: &std::path::Path) -> ExportDirectoryConfig {
|
||||
ExportDirectoryConfig {
|
||||
stay_on_url: false,
|
||||
url: format!("{}/jmap/session", registry.server.url()),
|
||||
auth: Auth::Basic {
|
||||
user: "admin".into(),
|
||||
|
||||
@@ -67,6 +67,7 @@ fn dry_run(archive: &Path, base: &str) -> Summary {
|
||||
},
|
||||
ExportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base.to_owned(),
|
||||
auth: Auth::Basic {
|
||||
user: "u".into(),
|
||||
|
||||
@@ -202,3 +202,100 @@ fn the_guide_checks_a_google_key_by_asking_for_a_token() {
|
||||
"{found}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn plan_writes_what_the_guide_would_from_answers_without_secrets() {
|
||||
let dir = TempDir::new().unwrap();
|
||||
let root = dir.path();
|
||||
let mail = root.join("mail/ann");
|
||||
for sub in ["cur", "new", "tmp"] {
|
||||
fs::create_dir_all(mail.join(sub)).unwrap();
|
||||
}
|
||||
fs::write(
|
||||
mail.join("cur/1:2,S"),
|
||||
"From: [email protected]\r\nSubject: Hi\r\nMessage-ID: <[email protected]>\r\n\r\nHi\r\n",
|
||||
)
|
||||
.unwrap();
|
||||
let plan = |answers: &Value| {
|
||||
fs::write(root.join("answers.json"), answers.to_string()).unwrap();
|
||||
std::process::Command::new(BIN)
|
||||
.args(["plan", "--answers"])
|
||||
.arg(root.join("answers.json"))
|
||||
.arg(root.join("run"))
|
||||
.output()
|
||||
.unwrap()
|
||||
};
|
||||
let mut answers = json!({
|
||||
"source": {"kind": "maildir", "path": "../mail/{local}"},
|
||||
"accounts": ["[email protected]"],
|
||||
"create": true,
|
||||
"target": {"url": "http://127.0.0.1:9", "stayOnUrl": true},
|
||||
});
|
||||
|
||||
let out = plan(&answers);
|
||||
assert!(
|
||||
out.status.success(),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&out.stderr)
|
||||
);
|
||||
let said: Value = serde_json::from_slice(&out.stdout).unwrap();
|
||||
assert_eq!(said["plan"], "plan.json");
|
||||
assert_eq!(said["preview"], "preview-plan.json");
|
||||
assert_eq!(said["accounts"], 1);
|
||||
assert_eq!(
|
||||
said["secrets"],
|
||||
json!({"source": null, "target": "MIGRATE_INBUXA_TOKEN"})
|
||||
);
|
||||
let passwords = said["passwords"].as_str().unwrap();
|
||||
let written: Value =
|
||||
serde_json::from_str(&fs::read_to_string(root.join("run/plan.json")).unwrap()).unwrap();
|
||||
assert_eq!(
|
||||
written["setup"][0]["args"],
|
||||
json!([
|
||||
"import-directory",
|
||||
"--url",
|
||||
"http://127.0.0.1:9",
|
||||
"--auth-bearer",
|
||||
"--stay-on-url",
|
||||
"--passwords",
|
||||
passwords,
|
||||
"directory"
|
||||
])
|
||||
);
|
||||
let directory = fs::read_to_string(root.join("run/directory/directory.json")).unwrap();
|
||||
assert!(directory.contains("[email protected]"), "{directory}");
|
||||
|
||||
// The preview it wrote runs as is.
|
||||
answers["create"] = json!(false);
|
||||
assert!(plan(&answers).status.success());
|
||||
let out = std::process::Command::new(BIN)
|
||||
.current_dir(root.join("run"))
|
||||
.args([
|
||||
"migrate",
|
||||
"--plan",
|
||||
"preview-plan.json",
|
||||
"--dry-run",
|
||||
"work",
|
||||
])
|
||||
.env("MIGRATE_INBUXA_TOKEN", "unused")
|
||||
.output()
|
||||
.unwrap();
|
||||
assert!(
|
||||
out.status.success(),
|
||||
"{}",
|
||||
String::from_utf8_lossy(&out.stderr)
|
||||
);
|
||||
let report = fs::read_to_string(root.join("run/work/dry-run-report.csv")).unwrap();
|
||||
assert!(report.contains("[email protected],import,done"), "{report}");
|
||||
|
||||
// A secret in the answers is refused, and isn't written anywhere.
|
||||
answers["target"]["token"] = json!("t0ken-for-the-test");
|
||||
let out = plan(&answers);
|
||||
assert!(!out.status.success());
|
||||
let err = String::from_utf8_lossy(&out.stderr);
|
||||
assert!(err.contains("hold a secret"), "{err}");
|
||||
answers["target"]["token"] = json!(null);
|
||||
answers["target"]["url"] = json!("");
|
||||
let out = plan(&answers);
|
||||
assert!(!out.status.success());
|
||||
}
|
||||
@@ -278,6 +278,7 @@ fn auth() -> Auth {
|
||||
|
||||
fn export(registry: &Registry, dir: &Path) {
|
||||
let summary = export_directory::run(ExportDirectoryConfig {
|
||||
stay_on_url: false,
|
||||
url: registry.url(),
|
||||
auth: auth(),
|
||||
dir: dir.to_path_buf(),
|
||||
@@ -293,6 +294,7 @@ fn export(registry: &Registry, dir: &Path) {
|
||||
|
||||
fn import(registry: &Registry, dir: &Path, passwords: Option<PathBuf>, dry_run: bool) -> Summary {
|
||||
import_directory::run(ImportDirectoryConfig {
|
||||
stay_on_url: false,
|
||||
url: registry.url(),
|
||||
auth: auth(),
|
||||
dir: dir.to_path_buf(),
|
||||
@@ -391,6 +393,7 @@ fn a_directory_imported_and_exported_again_comes_out_the_same() {
|
||||
|
||||
// Running again creates nothing and wants a new password file.
|
||||
let err = import_directory::run(ImportDirectoryConfig {
|
||||
stay_on_url: false,
|
||||
url: to.url(),
|
||||
auth: auth(),
|
||||
dir: exported.clone(),
|
||||
|
||||
@@ -1278,6 +1278,7 @@ fn import_aborts_with_exit_two_when_the_advertised_api_url_is_unreachable() {
|
||||
},
|
||||
inbuxa_migrate::sync::ImportConfig {
|
||||
connect: inbuxa_migrate::sync::ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: session_base.clone(),
|
||||
auth: Auth::Basic {
|
||||
user: "u".into(),
|
||||
@@ -1298,3 +1299,56 @@ fn import_aborts_with_exit_two_when_the_advertised_api_url_is_unreachable() {
|
||||
assert_eq!(err.exit_code(), 2, "must not report a partial failure");
|
||||
let _ = std::fs::remove_file(&archive);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stay_on_url_sends_the_api_calls_to_the_connect_host() {
|
||||
let mut api_host = mockito::Server::new();
|
||||
let mut session_host = mockito::Server::new();
|
||||
let session_base = session_host.url();
|
||||
|
||||
let _root = session_host.mock("GET", "/").with_status(404).create();
|
||||
let _wk = session_host
|
||||
.mock("GET", "/.well-known/jmap")
|
||||
.with_status(200)
|
||||
.with_header("content-type", "application/json")
|
||||
.with_body(session_json(&api_host.url()))
|
||||
.create();
|
||||
let advertised = api_host
|
||||
.mock("POST", "/jmap/api")
|
||||
.with_status(503)
|
||||
.expect(0)
|
||||
.create();
|
||||
let kept = session_host
|
||||
.mock("POST", "/jmap/api")
|
||||
.with_status(404)
|
||||
.expect_at_least(1)
|
||||
.create();
|
||||
|
||||
let archive = archive_path();
|
||||
let _ = inbuxa_migrate::sync::import_jmap::run(
|
||||
inbuxa_migrate::sync::CommonConfig {
|
||||
archive: archive.clone(),
|
||||
threads: 1,
|
||||
dry_run: false,
|
||||
max_retries: 0,
|
||||
allow_invalid_certs: false,
|
||||
logger: inbuxa_migrate::logging::Logger::from_flags(true, 0),
|
||||
},
|
||||
inbuxa_migrate::sync::ImportConfig {
|
||||
connect: inbuxa_migrate::sync::ConnectConfig {
|
||||
stay_on_url: true,
|
||||
url: session_base,
|
||||
auth: Auth::Basic {
|
||||
user: "u".into(),
|
||||
password: "p".into(),
|
||||
},
|
||||
account: AccountSelector::Id("w".into()),
|
||||
},
|
||||
objects: None,
|
||||
allow_source_change: false,
|
||||
},
|
||||
);
|
||||
advertised.assert();
|
||||
kept.assert();
|
||||
let _ = std::fs::remove_file(&archive);
|
||||
}
|
||||
@@ -146,6 +146,7 @@ fn export_email_already_exists_is_matched_not_failed() {
|
||||
},
|
||||
ExportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base.clone(),
|
||||
auth: Auth::Basic {
|
||||
user: "u".into(),
|
||||
@@ -287,6 +288,7 @@ fn export_mailbox_name_collision_merges_without_create() {
|
||||
},
|
||||
ExportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base.clone(),
|
||||
auth: Auth::Basic {
|
||||
user: "u".into(),
|
||||
@@ -445,6 +447,7 @@ fn export_mailbox_already_exists_maps_existing_id() {
|
||||
},
|
||||
ExportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base.clone(),
|
||||
auth: Auth::Basic {
|
||||
user: "u".into(),
|
||||
@@ -610,6 +613,7 @@ fn export_email_blob_not_found_reuploads_and_retries() {
|
||||
},
|
||||
ExportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base.clone(),
|
||||
auth: Auth::Basic {
|
||||
user: "u".into(),
|
||||
@@ -670,6 +674,7 @@ fn session_body_full(base: &str) -> String {
|
||||
fn import_cfg_objects(base: &str, objects: Vec<ObjectType>) -> ImportConfig {
|
||||
ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base.to_owned(),
|
||||
auth: Auth::Basic {
|
||||
user: "u".into(),
|
||||
@@ -685,6 +690,7 @@ fn import_cfg_objects(base: &str, objects: Vec<ObjectType>) -> ImportConfig {
|
||||
fn export_cfg_objects(base: &str, objects: Vec<ObjectType>) -> ExportConfig {
|
||||
ExportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base.to_owned(),
|
||||
auth: Auth::Basic {
|
||||
user: "u".into(),
|
||||
@@ -4500,6 +4506,7 @@ fn run_export(archive: &Path, base: &str, max_retries: u32) -> sync::Summary {
|
||||
},
|
||||
ExportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base.to_owned(),
|
||||
auth: Auth::Basic {
|
||||
user: "u".into(),
|
||||
|
||||
@@ -78,6 +78,7 @@ fn jmap_basic(localpart: &str, domain: &str) -> Auth {
|
||||
fn export_cfg(base_url: &str, domain: &str, localpart: &str, account_id: &str) -> ExportConfig {
|
||||
ExportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url.to_owned(),
|
||||
auth: jmap_basic(localpart, domain),
|
||||
account: AccountSelector::Id(account_id.to_owned()),
|
||||
@@ -96,6 +97,7 @@ fn jmap_import_cfg(
|
||||
) -> ImportConfig {
|
||||
ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url.to_owned(),
|
||||
auth: jmap_basic(localpart, domain),
|
||||
account: AccountSelector::Id(account_id.to_owned()),
|
||||
|
||||
@@ -78,6 +78,7 @@ fn jmap_basic(localpart: &str, domain: &str) -> Auth {
|
||||
fn export_cfg(base_url: &str, domain: &str, localpart: &str, account_id: &str) -> ExportConfig {
|
||||
ExportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url.to_owned(),
|
||||
auth: jmap_basic(localpart, domain),
|
||||
account: AccountSelector::Id(account_id.to_owned()),
|
||||
@@ -96,6 +97,7 @@ fn jmap_import_cfg(
|
||||
) -> ImportConfig {
|
||||
ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url.to_owned(),
|
||||
auth: jmap_basic(localpart, domain),
|
||||
account: AccountSelector::Id(account_id.to_owned()),
|
||||
|
||||
@@ -352,6 +352,7 @@ fn graph_import_survives_a_full_export_to_stalwart() {
|
||||
},
|
||||
ExportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url().to_owned(),
|
||||
auth: Auth::Basic {
|
||||
user: account.email.clone(),
|
||||
|
||||
@@ -236,6 +236,7 @@ fn imap_and_jmap_imports_share_blob_set() {
|
||||
|
||||
let jmap_cfg = ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: fx.base_url.clone(),
|
||||
auth: Auth::Basic {
|
||||
user: acc.email.clone(),
|
||||
@@ -376,6 +377,7 @@ fn imap_imported_archive_exports_via_jmap() {
|
||||
};
|
||||
let export_cfg = ExportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: fx.base_url.clone(),
|
||||
auth: Auth::Basic {
|
||||
user: dst.email.clone(),
|
||||
@@ -400,6 +402,7 @@ fn imap_imported_archive_exports_via_jmap() {
|
||||
};
|
||||
let export_cfg2 = ExportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: fx.base_url.clone(),
|
||||
auth: Auth::Basic {
|
||||
user: dst.email.clone(),
|
||||
@@ -494,6 +497,7 @@ fn dovecot_noselect_parents_keep_the_tree_through_export() {
|
||||
},
|
||||
ExportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: fx.base_url.clone(),
|
||||
auth: Auth::Basic {
|
||||
user: dst.email.clone(),
|
||||
|
||||
@@ -61,6 +61,7 @@ fn basic(localpart: &str) -> Auth {
|
||||
fn import_cfg(account: AccountSelector) -> ImportConfig {
|
||||
ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url().to_owned(),
|
||||
auth: basic("test1"),
|
||||
account,
|
||||
@@ -95,6 +96,7 @@ fn import_test1_full_matches_seed_stats() {
|
||||
|
||||
let cfg = ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: fx.base_url.clone(),
|
||||
auth: basic("test1"),
|
||||
account: AccountSelector::Id(acc.account_id.clone()),
|
||||
@@ -176,6 +178,7 @@ fn import_resolves_account_via_admin_principal_path() {
|
||||
|
||||
let cfg = ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: fx.base_url.clone(),
|
||||
auth: Auth::Basic {
|
||||
user: admin_fx.email.clone(),
|
||||
@@ -234,6 +237,7 @@ fn import_dry_run_writes_nothing() {
|
||||
fn export_cfg(localpart: &str, account_id: &str, prune: bool) -> ExportConfig {
|
||||
ExportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url().to_owned(),
|
||||
auth: Auth::Basic {
|
||||
user: format!("{localpart}@{}", seeder::DOMAIN),
|
||||
@@ -388,6 +392,7 @@ fn import_export_deeply_nested_mailbox_tree() {
|
||||
let archive = tmp_archive("deep_mbox");
|
||||
let imp = ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url().to_owned(),
|
||||
auth: basic("test4"),
|
||||
account: AccountSelector::Id(src.account_id.clone()),
|
||||
@@ -430,6 +435,7 @@ fn import_export_deeply_nested_mailbox_tree() {
|
||||
let again_archive = tmp_archive("deep_mbox_round");
|
||||
let imp_again = ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url().to_owned(),
|
||||
auth: basic("test5"),
|
||||
account: AccountSelector::Id(tgt.account_id.clone()),
|
||||
@@ -468,6 +474,7 @@ fn import_export_deeply_nested_filenode_tree() {
|
||||
let archive = tmp_archive("deep_fn");
|
||||
let imp = ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url().to_owned(),
|
||||
auth: basic("test4"),
|
||||
account: AccountSelector::Id(src.account_id.clone()),
|
||||
@@ -512,6 +519,7 @@ fn import_export_deeply_nested_filenode_tree() {
|
||||
common(&again_archive, false),
|
||||
ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url().to_owned(),
|
||||
auth: basic("test5"),
|
||||
account: AccountSelector::Id(tgt.account_id.clone()),
|
||||
@@ -552,6 +560,7 @@ fn prune_multilevel_mailbox_tree_destroys_leaf_first() {
|
||||
common(&arch_src, false),
|
||||
ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url().to_owned(),
|
||||
auth: basic("test4"),
|
||||
account: AccountSelector::Id(src.account_id.clone()),
|
||||
@@ -593,6 +602,7 @@ fn prune_multilevel_mailbox_tree_destroys_leaf_first() {
|
||||
common(&arch_empty, false),
|
||||
ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url().to_owned(),
|
||||
auth: basic("test6"),
|
||||
account: AccountSelector::Id(fx.account("test6").unwrap().account_id.clone()),
|
||||
@@ -641,6 +651,7 @@ fn prune_non_tree_type_contact_card() {
|
||||
common(&arch_full, false),
|
||||
ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url().to_owned(),
|
||||
auth: basic("test1"),
|
||||
account: AccountSelector::Id(src.account_id.clone()),
|
||||
@@ -893,6 +904,7 @@ fn export_round_trip_and_convergence() {
|
||||
|
||||
let imp = ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url().to_owned(),
|
||||
auth: basic("test1"),
|
||||
account: AccountSelector::Id(src.account_id.clone()),
|
||||
@@ -973,6 +985,7 @@ fn export_emails_in_multiple_import_batches() {
|
||||
|
||||
let imp = ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url().to_owned(),
|
||||
auth: basic("test1"),
|
||||
account: AccountSelector::Id(src.account_id.clone()),
|
||||
@@ -1053,6 +1066,7 @@ fn export_prune_destroys_unmatched() {
|
||||
common(&arch_big, false),
|
||||
ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url().to_owned(),
|
||||
auth: basic("test1"),
|
||||
account: AccountSelector::Id(big.account_id.clone()),
|
||||
@@ -1073,6 +1087,7 @@ fn export_prune_destroys_unmatched() {
|
||||
common(&arch_small, false),
|
||||
ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url().to_owned(),
|
||||
auth: basic("test3"),
|
||||
account: AccountSelector::Id(small.account_id.clone()),
|
||||
@@ -1108,6 +1123,7 @@ fn import_resume_converges_after_partial() {
|
||||
|
||||
let mk = || ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url().to_owned(),
|
||||
auth: basic("test2"),
|
||||
account: AccountSelector::Id(acc.account_id.clone()),
|
||||
|
||||
@@ -544,6 +544,7 @@ fn maildir_and_jmap_imports_share_blob_bytes() {
|
||||
|
||||
let jmap_cfg = ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: fx.base_url.clone(),
|
||||
auth: Auth::Basic {
|
||||
user: acc.email.clone(),
|
||||
@@ -620,6 +621,7 @@ fn maildir_message_count_matches_jmap_for_same_corpus() {
|
||||
let jmap_archive = tmp_archive("count-jmap");
|
||||
let jmap_cfg = ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: fx.base_url.clone(),
|
||||
auth: Auth::Basic {
|
||||
user: acc.email.clone(),
|
||||
|
||||
@@ -220,6 +220,7 @@ fn managesieve_and_jmap_imports_share_blob_bytes() {
|
||||
|
||||
let jmap_cfg = ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: fx.base_url.clone(),
|
||||
auth: Auth::Basic {
|
||||
user: acc.email.clone(),
|
||||
@@ -385,6 +386,7 @@ fn managesieve_round_trip_via_jmap_export_converges() {
|
||||
};
|
||||
let export_cfg = inbuxa_migrate::sync::ExportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: fx.base_url.clone(),
|
||||
auth: Auth::Basic {
|
||||
user: dst.email.clone(),
|
||||
@@ -419,6 +421,7 @@ fn managesieve_round_trip_via_jmap_export_converges() {
|
||||
};
|
||||
let export_cfg2 = inbuxa_migrate::sync::ExportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: fx.base_url.clone(),
|
||||
auth: Auth::Basic {
|
||||
user: dst.email.clone(),
|
||||
|
||||
@@ -78,6 +78,7 @@ fn jmap_basic(localpart: &str, domain: &str) -> Auth {
|
||||
fn export_cfg(base_url: &str, domain: &str, localpart: &str, account_id: &str) -> ExportConfig {
|
||||
ExportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url.to_owned(),
|
||||
auth: jmap_basic(localpart, domain),
|
||||
account: AccountSelector::Id(account_id.to_owned()),
|
||||
@@ -96,6 +97,7 @@ fn jmap_import_cfg(
|
||||
) -> ImportConfig {
|
||||
ImportConfig {
|
||||
connect: ConnectConfig {
|
||||
stay_on_url: false,
|
||||
url: base_url.to_owned(),
|
||||
auth: jmap_basic(localpart, domain),
|
||||
account: AccountSelector::Id(account_id.to_owned()),
|
||||
|
||||
Reference in new issue
Block a user