jcoffey-dev is traveling from Thursday 1 October through Sunday 4 October. Issues and pull requests are welcome, and will get an answer after that. Thanks for your patience.
Export wrote one message per request and said nothing while it did, so a
large mailbox took hours of silence.
Messages now go in batches of up to the target's maxObjectsInSet (at most
50), their blobs uploaded several at once, up to maxConcurrentUpload and no
more than --threads; each upload thread reads the archive through its own
read-only connection. Every message is still counted on its own: one the
target rejects fails alone, a request too large is split, and a method
error on the whole call is retried a message at a time.
A batch is sent once. If it ends without a clear answer -- a dropped
connection, a gateway timeout, a partial failure -- the target is read
again, the messages that arrived count as created, and only the rest are
imported again, so none is doubled.
A progress line (count, rate, time left) is printed every few seconds for
mail, contacts and events, and each type ends with a line of what was
created, updated, left unchanged and failed.
A POST that failed in transport -- including a timeout while waiting for the
answer -- is resent today, and so is one that got a 502 or 504. For a write
such as Email/import the server may already have applied the first copy,
so the resend can create a duplicate.
post_json_once and Request::send_once send it once: a transport failure or
a gateway error comes back to the caller, which can check the target before
trying again. 429 and 503, which mean the request was not processed, are
still retried.
jcoffey-dev