diff --git a/docs/spec/SPEC.md b/docs/spec/SPEC.md index a4d5ec75..a23f1bbf 100644 --- a/docs/spec/SPEC.md +++ b/docs/spec/SPEC.md @@ -21,15 +21,20 @@ setup and recovery. together. - No web interface on the mail host. Both front ends talk to the server over JMAP and OAuth, and can run beside it or on another machine. -- Stay close enough to upstream that its releases can be taken quickly. +- ~~Stay close enough to upstream that its releases can be taken quickly.~~ + Ended with the last import, 0.16.27 (2026-10-27): inbuxa diverges fully + from 27.1. **Non-goals** - Carrying, adapting or reading Stalwart Enterprise code (see §3). - Using Stalwart's name or logo anywhere in the product (see §2.4). -- Changing Stalwart's storage formats beyond what upstream changes. Existing - Stalwart 0.16 data, including data written by Enterprise features, must - open unchanged (see §7). +- ~~Changing Stalwart's storage formats beyond what upstream changes.~~ + Reversed 2026-10-29 (John): from 27.1 the store layout is inbuxa's own, + marked with its own store version, and Stalwart is a migration source. A + Stalwart 0.16.27 store stays readable as a frozen import format + (`import-0.16.md`). INBUXA's own Enterprise-era data was carried over at + its cutover (§7). ## 2. The fork itself diff --git a/docs/spec/import-0.16.md b/docs/spec/import-0.16.md new file mode 100644 index 00000000..1d1a99e0 --- /dev/null +++ b/docs/spec/import-0.16.md @@ -0,0 +1,120 @@ +# Importing a Stalwart 0.16.27 store + +Status: decided 2026-10-29 (John); checked against 27.1-rc.4 the same day. + +From 27.1 the store's layout is inbuxa's own, marked with its own store +version (`INBUXA_STORE_VERSION`, beside the 0.16 schema version). Stalwart is +a migration source like any other server. What inbuxa keeps is the ability to +read one Stalwart format, frozen at **0.16.27**, the last release it imported. +That format is defined here and checked on every release candidate. Stalwart +1.0 changed its own layout and isn't read by this route. It needs a converter +of its own (inbuxa-migrate, planned). + +This replaces "the store format is upstream's and unchanged" in `SPEC.md` §1 +and `migration.md`. inbuxa still opens a 0.16.27 store in place today, +because store version 1 is the 0.16 layout plus inbuxa's own subspace. That +holds until the first layout change bumps the version. The supported route +is the one below: export, then import into a new store. + +## The route + +1. Stop Stalwart 0.16.27 and export its store with its own binary: + `stalwart --config --export `. +2. Import into an empty inbuxa store: `inbuxa --config --import `. + The export holds the data, registry, blob, changelog, queue, report and + task subspaces. The full-text index isn't in it, so the import queues a + reindex. +3. Start inbuxa. The source isn't touched, so rolling back means starting + Stalwart again. + +An export goes from any backend to any other. A RocksDB 0.16 store can become +a PostgreSQL inbuxa one, or the other way round. + +## What carries over unchanged + +Checked by `tools/import-check` against the fixture in +`tests/resources/import-0.16.27/`: + +- accounts with their passwords, descriptions, aliases, quotas, group + membership and roles, including custom roles that inherit built-in ones; +- groups and their shared mailboxes; +- tenants, their domains and their accounts; +- domains and their DKIM keys (Ed25519 and RSA); +- every built-in role's permissions; +- mailboxes and their tree, mail with its keywords, threads, attachments, + sizes and received dates; +- mailbox sharing between accounts; +- calendars and events, with recurrence and time zones; +- address books and cards; +- Sieve scripts (active and inactive) and the vacation response. + +## What changes when inbuxa first starts + +Each of these is done in place on first start, and each is listed in +`tests/resources/import-0.16.27/known-differences.txt` or tested on its own: + +- **Store version:** written as 1 (`crates/migration`). +- **Roles:** inbuxa adds its Compliance Officer role. The built-in System + Administrator and Tenant Administrator roles gain inbuxa's own permissions + (account locks, audit, legal holds, journaling, DLP, mail rules, + deliverability, scheduled reports, security acceptance, AI explanations). + Nothing is taken away. +- **Spam filter blobs** move from their pre-rename keys (`crates/migration`, + `rename_spam_blobs`). +- **The web interface's OAuth client** `stalwart-webui` is retired, and an + application naming it moves to `inbuxa-webui` (`first_party.rs`, test + `renamed_identifiers`). +- **Sieve:** scripts that `require "vnd.stalwart.expressions"` or + `"vnd.stalwart.while"` keep running and can be edited and saved again. The + old names are aliases (#212). +- **Search:** the index is rebuilt. Each account's reindex is spread over the + first five minutes after start, so imported mail can be missing from + search results until then. New mail is indexed as it arrives. + +## What the operator has to change + +- **Clients signing in to JMAP with a password.** inbuxa takes tokens on JMAP + and the management API, never passwords (contract C-23). IMAP, POP3, + SMTP, CalDAV and CardDAV keep passwords. `INBUXA_HTTP_BASIC_AUTH=all` + restores passwords on JMAP for an operator who needs it while clients move + to OAuth. +- **The 0.16 web interface.** Its application record carries over and points + at upstream's `webui` release on GitHub. inbuxa keeps an imported + application and fetches it (SPEC.md §5.3), so it serves that interface at + `/admin` unless the operator removes it. Open: whether an import should + disable it (see below). + +## Not covered + +- **Enterprise data.** The fixture is a Community store, and a Community + server has no Enterprise objects to carry. inbuxa's own cutover from an + Enterprise store is recorded in `compat-tests.md`. +- **Stalwart releases other than 0.16.27.** An older 0.16 upgrades to 0.16.27 + with Stalwart first (stalwart-migrator does this). 1.0 and later aren't + read by this route. + +## The check + +`tools/import-check/check.sh ` runs on every release candidate. It +needs docker and about seven minutes, most of it waiting for the reindex. +It fails on any difference that isn't in `known-differences.txt`, and on +any of the after-import checks: delivery through the imported Sieve filter, +search over imported mail, and re-saving a pre-rename filter. A known +difference that stops happening is reported as `GONE`, so the list can't go +stale without anyone noticing. + +The CI workflows compile the test suites but don't run them (by design), and +they don't run this check either. + +`tools/import-check/make-fixture.sh` rebuilds the fixture from the official +0.16.27 image, pinned by digest. Run it only when the fixture needs more in +it. The server it builds has no internet access, so upstream's web +interface bundle and the data files it downloads at first start never reach +the fixture. The fixture holds only what `populate.py` creates, plus what +0.16.27 creates at setup. + +## Open + +- Whether importing a 0.16 store should disable the leftover web interface + application (keeping the record, so the operator can switch it back on), + rather than serving upstream's interface at `/admin`. diff --git a/docs/spec/migration.md b/docs/spec/migration.md index 4a4cdeb5..098f54d2 100644 --- a/docs/spec/migration.md +++ b/docs/spec/migration.md @@ -86,9 +86,11 @@ change a password, re-authorize a client, or re-point a DNS record. That is the promise the tool makes, and the verification below is what checks it rather than assuming it. -The store format is upstream's and unchanged (SPEC.md §2.6), which is what -makes this a copy rather than a conversion. A migration that also crosses an -upstream version is that upgrade first, then this. +~~The store format is upstream's and unchanged (SPEC.md §2.6), which is what +makes this a copy rather than a conversion.~~ Since 2026-10-29 the store +layout is inbuxa's own. A Stalwart 0.16.27 store is read as a frozen import +format, by export and import (`import-0.16.md`). A source on an older 0.16 +is upgraded to 0.16.27 first, then imported. ## Shape diff --git a/tests/resources/import-0.16.27/admin.json b/tests/resources/import-0.16.27/admin.json new file mode 100644 index 00000000..d23d2b4e --- /dev/null +++ b/tests/resources/import-0.16.27/admin.json @@ -0,0 +1 @@ +{"username": "admin@example.org", "secret": "wZoDr0W3jv30PGK3"} \ No newline at end of file diff --git a/tests/resources/import-0.16.27/expected.json b/tests/resources/import-0.16.27/expected.json new file mode 100644 index 00000000..e78340a7 --- /dev/null +++ b/tests/resources/import-0.16.27/expected.json @@ -0,0 +1,1445 @@ +{ + "admin": { + "accounts": { + "admin": { + "aliases": [], + "description": "System administrator", + "domain": "example.org", + "groups": [], + "quotas": {}, + "roles": [ + "Admin" + ], + "tenant": false, + "type": "User" + }, + "alice": { + "aliases": [ + "a.example@example.org" + ], + "description": "Alice Example", + "domain": "example.org", + "groups": [ + "sales" + ], + "quotas": {}, + "roles": [ + "Custom", + "Helpdesk", + "User" + ], + "tenant": false, + "type": "User" + }, + "bob": { + "aliases": [], + "description": "Bob Example", + "domain": "example.org", + "groups": [ + "sales" + ], + "quotas": { + "maxDiskQuota": 50000000 + }, + "roles": [ + "User" + ], + "tenant": false, + "type": "User" + }, + "carol": { + "aliases": [], + "description": "Carol Acme", + "domain": "acme.example.com", + "groups": [], + "quotas": {}, + "roles": [ + "User" + ], + "tenant": true, + "type": "User" + }, + "sales": { + "aliases": [], + "description": "Sales team", + "domain": "example.org", + "groups": [], + "quotas": {}, + "roles": [ + "Default" + ], + "tenant": false, + "type": "Group" + } + }, + "applications": [ + "Stalwart Web Interface enabled=True https://github.com/stalwartlabs/webui/releases/download/v1.0.11/webui.zip" + ], + "dkim": [ + "acme.example.com Dkim1Ed25519Sha256", + "acme.example.com Dkim1RsaSha256", + "example.org Dkim1Ed25519Sha256", + "example.org Dkim1RsaSha256" + ], + "domains": [ + "acme.example.com", + "example.org" + ], + "roles": { + "Group": { + "disabled": [], + "enabled": [ + "calendarAlarmsSend", + "calendarSchedulingReceive", + "calendarSchedulingSend", + "davCalAcl", + "davCalCopy", + "davCalDelete", + "davCalFreeBusyQuery", + "davCalGet", + "davCalLock", + "davCalMkCol", + "davCalMove", + "davCalMultiGet", + "davCalPropFind", + "davCalPropPatch", + "davCalPut", + "davCalQuery", + "davCardAcl", + "davCardCopy", + "davCardDelete", + "davCardGet", + "davCardLock", + "davCardMkCol", + "davCardMove", + "davCardMultiGet", + "davCardPropFind", + "davCardPropPatch", + "davCardPut", + "davCardQuery", + "davExpandProperty", + "davFileAcl", + "davFileCopy", + "davFileDelete", + "davFileGet", + "davFileLock", + "davFileMkCol", + "davFileMove", + "davFilePropFind", + "davFilePropPatch", + "davFilePut", + "davPrincipalAcl", + "davPrincipalList", + "davPrincipalMatch", + "davPrincipalSearch", + "davPrincipalSearchPropSet", + "davSyncCollection", + "emailReceive", + "emailSend", + "imapAclGet", + "imapAclSet", + "imapAppend", + "imapAuthenticate", + "imapCapability", + "imapCopy", + "imapCreate", + "imapDelete", + "imapEnable", + "imapExamine", + "imapExpunge", + "imapFetch", + "imapId", + "imapIdle", + "imapList", + "imapListRights", + "imapLsub", + "imapMove", + "imapMyRights", + "imapNamespace", + "imapRename", + "imapSearch", + "imapSelect", + "imapSort", + "imapStatus", + "imapStore", + "imapSubscribe", + "imapThread", + "jmapAddressBookChanges", + "jmapAddressBookCreate", + "jmapAddressBookDestroy", + "jmapAddressBookGet", + "jmapAddressBookUpdate", + "jmapBlobCopy", + "jmapBlobGet", + "jmapBlobLookup", + "jmapBlobUpload", + "jmapCalendarChanges", + "jmapCalendarCreate", + "jmapCalendarDestroy", + "jmapCalendarEventChanges", + "jmapCalendarEventCopy", + "jmapCalendarEventCreate", + "jmapCalendarEventDestroy", + "jmapCalendarEventGet", + "jmapCalendarEventNotificationChanges", + "jmapCalendarEventNotificationCreate", + "jmapCalendarEventNotificationDestroy", + "jmapCalendarEventNotificationGet", + "jmapCalendarEventNotificationQuery", + "jmapCalendarEventNotificationQueryChanges", + "jmapCalendarEventNotificationUpdate", + "jmapCalendarEventParse", + "jmapCalendarEventQuery", + "jmapCalendarEventQueryChanges", + "jmapCalendarEventUpdate", + "jmapCalendarGet", + "jmapCalendarUpdate", + "jmapContactCardChanges", + "jmapContactCardCopy", + "jmapContactCardCreate", + "jmapContactCardDestroy", + "jmapContactCardGet", + "jmapContactCardParse", + "jmapContactCardQuery", + "jmapContactCardQueryChanges", + "jmapContactCardUpdate", + "jmapCoreEcho", + "jmapEmailChanges", + "jmapEmailCopy", + "jmapEmailCreate", + "jmapEmailDestroy", + "jmapEmailGet", + "jmapEmailImport", + "jmapEmailParse", + "jmapEmailQuery", + "jmapEmailQueryChanges", + "jmapEmailSubmissionChanges", + "jmapEmailSubmissionCreate", + "jmapEmailSubmissionDestroy", + "jmapEmailSubmissionGet", + "jmapEmailSubmissionQuery", + "jmapEmailSubmissionQueryChanges", + "jmapEmailSubmissionUpdate", + "jmapEmailUpdate", + "jmapFileNodeChanges", + "jmapFileNodeCopy", + "jmapFileNodeCreate", + "jmapFileNodeDestroy", + "jmapFileNodeGet", + "jmapFileNodeQuery", + "jmapFileNodeQueryChanges", + "jmapFileNodeUpdate", + "jmapIdentityChanges", + "jmapIdentityCreate", + "jmapIdentityDestroy", + "jmapIdentityGet", + "jmapIdentityUpdate", + "jmapMailboxChanges", + "jmapMailboxCreate", + "jmapMailboxDestroy", + "jmapMailboxGet", + "jmapMailboxQuery", + "jmapMailboxQueryChanges", + "jmapMailboxUpdate", + "jmapParticipantIdentityChanges", + "jmapParticipantIdentityCreate", + "jmapParticipantIdentityDestroy", + "jmapParticipantIdentityGet", + "jmapParticipantIdentityUpdate", + "jmapPrincipalChanges", + "jmapPrincipalCreate", + "jmapPrincipalDestroy", + "jmapPrincipalGet", + "jmapPrincipalGetAvailability", + "jmapPrincipalQuery", + "jmapPrincipalQueryChanges", + "jmapPrincipalUpdate", + "jmapPushSubscriptionCreate", + "jmapPushSubscriptionDestroy", + "jmapPushSubscriptionGet", + "jmapPushSubscriptionUpdate", + "jmapQuotaChanges", + "jmapQuotaGet", + "jmapQuotaQuery", + "jmapQuotaQueryChanges", + "jmapSearchSnippetGet", + "jmapShareNotificationChanges", + "jmapShareNotificationCreate", + "jmapShareNotificationDestroy", + "jmapShareNotificationGet", + "jmapShareNotificationQuery", + "jmapShareNotificationQueryChanges", + "jmapShareNotificationUpdate", + "jmapSieveScriptCreate", + "jmapSieveScriptDestroy", + "jmapSieveScriptGet", + "jmapSieveScriptQuery", + "jmapSieveScriptUpdate", + "jmapSieveScriptValidate", + "jmapThreadChanges", + "jmapThreadGet", + "jmapVacationResponseCreate", + "jmapVacationResponseDestroy", + "jmapVacationResponseGet", + "jmapVacationResponseUpdate", + "pop3Authenticate", + "pop3Dele", + "pop3List", + "pop3Retr", + "pop3Stat", + "pop3Uidl", + "sieveAuthenticate", + "sieveCheckScript", + "sieveDeleteScript", + "sieveGetScript", + "sieveHaveSpace", + "sieveListScripts", + "sievePutScript", + "sieveRenameScript", + "sieveSetActive", + "sysAccountSettingsGet", + "sysAccountSettingsUpdate", + "sysArchivedItemCreate", + "sysArchivedItemDestroy", + "sysArchivedItemGet", + "sysArchivedItemQuery", + "sysArchivedItemUpdate", + "sysMaskedEmailCreate", + "sysMaskedEmailDestroy", + "sysMaskedEmailGet", + "sysMaskedEmailQuery", + "sysMaskedEmailUpdate", + "sysPublicKeyCreate", + "sysPublicKeyDestroy", + "sysPublicKeyGet", + "sysPublicKeyQuery", + "sysPublicKeyUpdate", + "sysSpamTrainingSampleDestroy", + "sysSpamTrainingSampleGet", + "sysSpamTrainingSampleQuery", + "sysSpamTrainingSampleUpdate" + ], + "inherits": [] + }, + "Helpdesk": { + "disabled": [], + "enabled": [ + "sysAccountGet", + "sysAccountQuery", + "sysQueuedMessageGet" + ], + "inherits": [] + }, + "System Administrator": { + "disabled": [], + "enabled": [ + "actionClassifySpam", + "actionInvalidateCaches", + "actionInvalidateNegativeCaches", + "actionPauseMtaQueue", + "actionReloadBlockedIps", + "actionReloadLookupStores", + "actionReloadSettings", + "actionReloadTlsCertificates", + "actionResumeMtaQueue", + "actionTroubleshootDmarc", + "actionUpdateApps", + "authenticate", + "authenticateWithAlias", + "fetchAnyBlob", + "impersonate", + "interactAi", + "liveDeliveryTest", + "liveMetrics", + "liveTracing", + "oAuthClientOverride", + "oAuthClientRegistration", + "scimAccess", + "sysAccountCreate", + "sysAccountDestroy", + "sysAccountGet", + "sysAccountPasswordGet", + "sysAccountPasswordUpdate", + "sysAccountQuery", + "sysAccountSettingsGet", + "sysAccountSettingsUpdate", + "sysAccountUpdate", + "sysAcmeProviderCreate", + "sysAcmeProviderDestroy", + "sysAcmeProviderGet", + "sysAcmeProviderQuery", + "sysAcmeProviderUpdate", + "sysActionCreate", + "sysActionDestroy", + "sysActionGet", + "sysActionQuery", + "sysActionUpdate", + "sysAddressBookGet", + "sysAddressBookUpdate", + "sysAiModelCreate", + "sysAiModelDestroy", + "sysAiModelGet", + "sysAiModelQuery", + "sysAiModelUpdate", + "sysAlertCreate", + "sysAlertDestroy", + "sysAlertGet", + "sysAlertQuery", + "sysAlertUpdate", + "sysAllowedIpCreate", + "sysAllowedIpDestroy", + "sysAllowedIpGet", + "sysAllowedIpQuery", + "sysAllowedIpUpdate", + "sysApiKeyCreate", + "sysApiKeyDestroy", + "sysApiKeyGet", + "sysApiKeyQuery", + "sysApiKeyUpdate", + "sysAppPasswordCreate", + "sysAppPasswordDestroy", + "sysAppPasswordGet", + "sysAppPasswordQuery", + "sysAppPasswordUpdate", + "sysApplicationCreate", + "sysApplicationDestroy", + "sysApplicationGet", + "sysApplicationQuery", + "sysApplicationUpdate", + "sysArchivedItemCreate", + "sysArchivedItemDestroy", + "sysArchivedItemGet", + "sysArchivedItemQuery", + "sysArchivedItemUpdate", + "sysArfExternalReportCreate", + "sysArfExternalReportDestroy", + "sysArfExternalReportGet", + "sysArfExternalReportQuery", + "sysArfExternalReportUpdate", + "sysAsnGet", + "sysAsnUpdate", + "sysAuthenticationGet", + "sysAuthenticationUpdate", + "sysBlobStoreGet", + "sysBlobStoreUpdate", + "sysBlockedIpCreate", + "sysBlockedIpDestroy", + "sysBlockedIpGet", + "sysBlockedIpQuery", + "sysBlockedIpUpdate", + "sysBootstrapGet", + "sysBootstrapUpdate", + "sysCacheGet", + "sysCacheUpdate", + "sysCalendarAlarmGet", + "sysCalendarAlarmUpdate", + "sysCalendarGet", + "sysCalendarSchedulingGet", + "sysCalendarSchedulingUpdate", + "sysCalendarUpdate", + "sysCertificateCreate", + "sysCertificateDestroy", + "sysCertificateGet", + "sysCertificateQuery", + "sysCertificateUpdate", + "sysClusterNodeCreate", + "sysClusterNodeDestroy", + "sysClusterNodeGet", + "sysClusterNodeQuery", + "sysClusterNodeUpdate", + "sysClusterRoleCreate", + "sysClusterRoleDestroy", + "sysClusterRoleGet", + "sysClusterRoleQuery", + "sysClusterRoleUpdate", + "sysCoordinatorGet", + "sysCoordinatorUpdate", + "sysDataRetentionGet", + "sysDataRetentionUpdate", + "sysDataStoreGet", + "sysDataStoreUpdate", + "sysDirectoryCreate", + "sysDirectoryDestroy", + "sysDirectoryGet", + "sysDirectoryQuery", + "sysDirectoryUpdate", + "sysDkimReportSettingsGet", + "sysDkimReportSettingsUpdate", + "sysDkimSignatureCreate", + "sysDkimSignatureDestroy", + "sysDkimSignatureGet", + "sysDkimSignatureQuery", + "sysDkimSignatureUpdate", + "sysDmarcExternalReportCreate", + "sysDmarcExternalReportDestroy", + "sysDmarcExternalReportGet", + "sysDmarcExternalReportQuery", + "sysDmarcExternalReportUpdate", + "sysDmarcInternalReportCreate", + "sysDmarcInternalReportDestroy", + "sysDmarcInternalReportGet", + "sysDmarcInternalReportQuery", + "sysDmarcInternalReportUpdate", + "sysDmarcReportSettingsGet", + "sysDmarcReportSettingsUpdate", + "sysDnsResolverGet", + "sysDnsResolverUpdate", + "sysDnsServerCreate", + "sysDnsServerDestroy", + "sysDnsServerGet", + "sysDnsServerQuery", + "sysDnsServerUpdate", + "sysDomainCreate", + "sysDomainDestroy", + "sysDomainGet", + "sysDomainQuery", + "sysDomainUpdate", + "sysDsnReportSettingsGet", + "sysDsnReportSettingsUpdate", + "sysEmailGet", + "sysEmailUpdate", + "sysEnterpriseGet", + "sysEnterpriseUpdate", + "sysEventTracingLevelCreate", + "sysEventTracingLevelDestroy", + "sysEventTracingLevelGet", + "sysEventTracingLevelQuery", + "sysEventTracingLevelUpdate", + "sysFileStorageGet", + "sysFileStorageUpdate", + "sysHttpFormGet", + "sysHttpFormUpdate", + "sysHttpGet", + "sysHttpLookupCreate", + "sysHttpLookupDestroy", + "sysHttpLookupGet", + "sysHttpLookupQuery", + "sysHttpLookupUpdate", + "sysHttpUpdate", + "sysImapGet", + "sysImapUpdate", + "sysInMemoryStoreGet", + "sysInMemoryStoreUpdate", + "sysJmapGet", + "sysJmapUpdate", + "sysLogCreate", + "sysLogDestroy", + "sysLogGet", + "sysLogQuery", + "sysLogUpdate", + "sysMailingListCreate", + "sysMailingListDestroy", + "sysMailingListGet", + "sysMailingListQuery", + "sysMailingListUpdate", + "sysMaskedEmailCreate", + "sysMaskedEmailDestroy", + "sysMaskedEmailGet", + "sysMaskedEmailQuery", + "sysMaskedEmailUpdate", + "sysMemoryLookupKeyCreate", + "sysMemoryLookupKeyDestroy", + "sysMemoryLookupKeyGet", + "sysMemoryLookupKeyQuery", + "sysMemoryLookupKeyUpdate", + "sysMemoryLookupKeyValueCreate", + "sysMemoryLookupKeyValueDestroy", + "sysMemoryLookupKeyValueGet", + "sysMemoryLookupKeyValueQuery", + "sysMemoryLookupKeyValueUpdate", + "sysMetricCreate", + "sysMetricDestroy", + "sysMetricGet", + "sysMetricQuery", + "sysMetricUpdate", + "sysMetricsGet", + "sysMetricsStoreGet", + "sysMetricsStoreUpdate", + "sysMetricsUpdate", + "sysMtaConnectionStrategyCreate", + "sysMtaConnectionStrategyDestroy", + "sysMtaConnectionStrategyGet", + "sysMtaConnectionStrategyQuery", + "sysMtaConnectionStrategyUpdate", + "sysMtaDeliveryScheduleCreate", + "sysMtaDeliveryScheduleDestroy", + "sysMtaDeliveryScheduleGet", + "sysMtaDeliveryScheduleQuery", + "sysMtaDeliveryScheduleUpdate", + "sysMtaExtensionsGet", + "sysMtaExtensionsUpdate", + "sysMtaHookCreate", + "sysMtaHookDestroy", + "sysMtaHookGet", + "sysMtaHookQuery", + "sysMtaHookUpdate", + "sysMtaInboundSessionGet", + "sysMtaInboundSessionUpdate", + "sysMtaInboundThrottleCreate", + "sysMtaInboundThrottleDestroy", + "sysMtaInboundThrottleGet", + "sysMtaInboundThrottleQuery", + "sysMtaInboundThrottleUpdate", + "sysMtaMilterCreate", + "sysMtaMilterDestroy", + "sysMtaMilterGet", + "sysMtaMilterQuery", + "sysMtaMilterUpdate", + "sysMtaOutboundStrategyGet", + "sysMtaOutboundStrategyUpdate", + "sysMtaOutboundThrottleCreate", + "sysMtaOutboundThrottleDestroy", + "sysMtaOutboundThrottleGet", + "sysMtaOutboundThrottleQuery", + "sysMtaOutboundThrottleUpdate", + "sysMtaQueueQuotaCreate", + "sysMtaQueueQuotaDestroy", + "sysMtaQueueQuotaGet", + "sysMtaQueueQuotaQuery", + "sysMtaQueueQuotaUpdate", + "sysMtaRouteCreate", + "sysMtaRouteDestroy", + "sysMtaRouteGet", + "sysMtaRouteQuery", + "sysMtaRouteUpdate", + "sysMtaStageAuthGet", + "sysMtaStageAuthUpdate", + "sysMtaStageConnectGet", + "sysMtaStageConnectUpdate", + "sysMtaStageDataGet", + "sysMtaStageDataUpdate", + "sysMtaStageEhloGet", + "sysMtaStageEhloUpdate", + "sysMtaStageMailGet", + "sysMtaStageMailUpdate", + "sysMtaStageRcptGet", + "sysMtaStageRcptUpdate", + "sysMtaStsGet", + "sysMtaStsUpdate", + "sysMtaTlsStrategyCreate", + "sysMtaTlsStrategyDestroy", + "sysMtaTlsStrategyGet", + "sysMtaTlsStrategyQuery", + "sysMtaTlsStrategyUpdate", + "sysMtaVirtualQueueCreate", + "sysMtaVirtualQueueDestroy", + "sysMtaVirtualQueueGet", + "sysMtaVirtualQueueQuery", + "sysMtaVirtualQueueUpdate", + "sysNetworkListenerCreate", + "sysNetworkListenerDestroy", + "sysNetworkListenerGet", + "sysNetworkListenerQuery", + "sysNetworkListenerUpdate", + "sysOAuthClientCreate", + "sysOAuthClientDestroy", + "sysOAuthClientGet", + "sysOAuthClientQuery", + "sysOAuthClientUpdate", + "sysOidcProviderGet", + "sysOidcProviderUpdate", + "sysPublicKeyCreate", + "sysPublicKeyDestroy", + "sysPublicKeyGet", + "sysPublicKeyQuery", + "sysPublicKeyUpdate", + "sysQueuedMessageCreate", + "sysQueuedMessageDestroy", + "sysQueuedMessageGet", + "sysQueuedMessageQuery", + "sysQueuedMessageUpdate", + "sysReportSettingsGet", + "sysReportSettingsUpdate", + "sysRoleCreate", + "sysRoleDestroy", + "sysRoleGet", + "sysRoleQuery", + "sysRoleUpdate", + "sysSearchGet", + "sysSearchStoreGet", + "sysSearchStoreUpdate", + "sysSearchUpdate", + "sysSecurityGet", + "sysSecurityUpdate", + "sysSenderAuthGet", + "sysSenderAuthUpdate", + "sysSharingGet", + "sysSharingUpdate", + "sysSieveSystemInterpreterGet", + "sysSieveSystemInterpreterUpdate", + "sysSieveSystemScriptCreate", + "sysSieveSystemScriptDestroy", + "sysSieveSystemScriptGet", + "sysSieveSystemScriptQuery", + "sysSieveSystemScriptUpdate", + "sysSieveUserInterpreterGet", + "sysSieveUserInterpreterUpdate", + "sysSieveUserScriptCreate", + "sysSieveUserScriptDestroy", + "sysSieveUserScriptGet", + "sysSieveUserScriptQuery", + "sysSieveUserScriptUpdate", + "sysSpamClassifierGet", + "sysSpamClassifierUpdate", + "sysSpamDnsblServerCreate", + "sysSpamDnsblServerDestroy", + "sysSpamDnsblServerGet", + "sysSpamDnsblServerQuery", + "sysSpamDnsblServerUpdate", + "sysSpamDnsblSettingsGet", + "sysSpamDnsblSettingsUpdate", + "sysSpamFileExtensionCreate", + "sysSpamFileExtensionDestroy", + "sysSpamFileExtensionGet", + "sysSpamFileExtensionQuery", + "sysSpamFileExtensionUpdate", + "sysSpamLlmGet", + "sysSpamLlmUpdate", + "sysSpamPyzorGet", + "sysSpamPyzorUpdate", + "sysSpamRuleCreate", + "sysSpamRuleDestroy", + "sysSpamRuleGet", + "sysSpamRuleQuery", + "sysSpamRuleUpdate", + "sysSpamSettingsGet", + "sysSpamSettingsUpdate", + "sysSpamTagCreate", + "sysSpamTagDestroy", + "sysSpamTagGet", + "sysSpamTagQuery", + "sysSpamTagUpdate", + "sysSpamTrainingSampleCreate", + "sysSpamTrainingSampleDestroy", + "sysSpamTrainingSampleGet", + "sysSpamTrainingSampleQuery", + "sysSpamTrainingSampleUpdate", + "sysSpfReportSettingsGet", + "sysSpfReportSettingsUpdate", + "sysStoreLookupCreate", + "sysStoreLookupDestroy", + "sysStoreLookupGet", + "sysStoreLookupQuery", + "sysStoreLookupUpdate", + "sysSystemSettingsGet", + "sysSystemSettingsUpdate", + "sysTaskCreate", + "sysTaskDestroy", + "sysTaskGet", + "sysTaskManagerGet", + "sysTaskManagerUpdate", + "sysTaskQuery", + "sysTaskUpdate", + "sysTenantCreate", + "sysTenantDestroy", + "sysTenantGet", + "sysTenantQuery", + "sysTenantUpdate", + "sysTlsExternalReportCreate", + "sysTlsExternalReportDestroy", + "sysTlsExternalReportGet", + "sysTlsExternalReportQuery", + "sysTlsExternalReportUpdate", + "sysTlsInternalReportCreate", + "sysTlsInternalReportDestroy", + "sysTlsInternalReportGet", + "sysTlsInternalReportQuery", + "sysTlsInternalReportUpdate", + "sysTlsReportSettingsGet", + "sysTlsReportSettingsUpdate", + "sysTraceCreate", + "sysTraceDestroy", + "sysTraceGet", + "sysTraceQuery", + "sysTraceUpdate", + "sysTracerCreate", + "sysTracerDestroy", + "sysTracerGet", + "sysTracerQuery", + "sysTracerUpdate", + "sysTracingStoreGet", + "sysTracingStoreUpdate", + "sysWebDavGet", + "sysWebDavUpdate", + "sysWebHookCreate", + "sysWebHookDestroy", + "sysWebHookGet", + "sysWebHookQuery", + "sysWebHookUpdate", + "taskAccountMaintenance", + "taskAcmeRenewal", + "taskCalendarAlarmEmail", + "taskCalendarAlarmNotification", + "taskCalendarItipMessage", + "taskDestroyAccount", + "taskDkimManagement", + "taskDmarcReport", + "taskDnsManagement", + "taskIndexDocument", + "taskIndexTrace", + "taskMergeThreads", + "taskRestoreArchivedItem", + "taskSpamFilterMaintenance", + "taskStoreMaintenance", + "taskTenantMaintenance", + "taskTlsReport", + "taskUnindexDocument", + "unlimitedRequests", + "unlimitedUploads" + ], + "inherits": [] + }, + "Tenant Administrator": { + "disabled": [], + "enabled": [ + "authenticate", + "authenticateWithAlias", + "fetchAnyBlob", + "interactAi", + "liveDeliveryTest", + "sysAccountCreate", + "sysAccountDestroy", + "sysAccountGet", + "sysAccountQuery", + "sysAccountUpdate", + "sysAcmeProviderCreate", + "sysAcmeProviderDestroy", + "sysAcmeProviderGet", + "sysAcmeProviderQuery", + "sysAcmeProviderUpdate", + "sysDkimSignatureCreate", + "sysDkimSignatureDestroy", + "sysDkimSignatureGet", + "sysDkimSignatureQuery", + "sysDkimSignatureUpdate", + "sysDnsServerCreate", + "sysDnsServerDestroy", + "sysDnsServerGet", + "sysDnsServerQuery", + "sysDnsServerUpdate", + "sysDomainCreate", + "sysDomainDestroy", + "sysDomainGet", + "sysDomainQuery", + "sysDomainUpdate", + "sysMailingListCreate", + "sysMailingListDestroy", + "sysMailingListGet", + "sysMailingListQuery", + "sysMailingListUpdate", + "sysOAuthClientCreate", + "sysOAuthClientDestroy", + "sysOAuthClientGet", + "sysOAuthClientQuery", + "sysOAuthClientUpdate", + "sysQueuedMessageCreate", + "sysQueuedMessageDestroy", + "sysQueuedMessageGet", + "sysQueuedMessageQuery", + "sysQueuedMessageUpdate", + "sysRoleCreate", + "sysRoleDestroy", + "sysRoleGet", + "sysRoleQuery", + "sysRoleUpdate" + ], + "inherits": [] + }, + "User": { + "disabled": [], + "enabled": [ + "authenticate", + "authenticateWithAlias", + "calendarAlarmsSend", + "calendarSchedulingReceive", + "calendarSchedulingSend", + "davCalAcl", + "davCalCopy", + "davCalDelete", + "davCalFreeBusyQuery", + "davCalGet", + "davCalLock", + "davCalMkCol", + "davCalMove", + "davCalMultiGet", + "davCalPropFind", + "davCalPropPatch", + "davCalPut", + "davCalQuery", + "davCardAcl", + "davCardCopy", + "davCardDelete", + "davCardGet", + "davCardLock", + "davCardMkCol", + "davCardMove", + "davCardMultiGet", + "davCardPropFind", + "davCardPropPatch", + "davCardPut", + "davCardQuery", + "davExpandProperty", + "davFileAcl", + "davFileCopy", + "davFileDelete", + "davFileGet", + "davFileLock", + "davFileMkCol", + "davFileMove", + "davFilePropFind", + "davFilePropPatch", + "davFilePut", + "davPrincipalAcl", + "davPrincipalList", + "davPrincipalMatch", + "davPrincipalSearch", + "davPrincipalSearchPropSet", + "davSyncCollection", + "emailReceive", + "emailSend", + "imapAclGet", + "imapAclSet", + "imapAppend", + "imapAuthenticate", + "imapCapability", + "imapCopy", + "imapCreate", + "imapDelete", + "imapEnable", + "imapExamine", + "imapExpunge", + "imapFetch", + "imapId", + "imapIdle", + "imapList", + "imapListRights", + "imapLsub", + "imapMove", + "imapMyRights", + "imapNamespace", + "imapRename", + "imapSearch", + "imapSelect", + "imapSort", + "imapStatus", + "imapStore", + "imapSubscribe", + "imapThread", + "interactAi", + "jmapAddressBookChanges", + "jmapAddressBookCreate", + "jmapAddressBookDestroy", + "jmapAddressBookGet", + "jmapAddressBookUpdate", + "jmapBlobCopy", + "jmapBlobGet", + "jmapBlobLookup", + "jmapBlobUpload", + "jmapCalendarChanges", + "jmapCalendarCreate", + "jmapCalendarDestroy", + "jmapCalendarEventChanges", + "jmapCalendarEventCopy", + "jmapCalendarEventCreate", + "jmapCalendarEventDestroy", + "jmapCalendarEventGet", + "jmapCalendarEventNotificationChanges", + "jmapCalendarEventNotificationCreate", + "jmapCalendarEventNotificationDestroy", + "jmapCalendarEventNotificationGet", + "jmapCalendarEventNotificationQuery", + "jmapCalendarEventNotificationQueryChanges", + "jmapCalendarEventNotificationUpdate", + "jmapCalendarEventParse", + "jmapCalendarEventQuery", + "jmapCalendarEventQueryChanges", + "jmapCalendarEventUpdate", + "jmapCalendarGet", + "jmapCalendarUpdate", + "jmapContactCardChanges", + "jmapContactCardCopy", + "jmapContactCardCreate", + "jmapContactCardDestroy", + "jmapContactCardGet", + "jmapContactCardParse", + "jmapContactCardQuery", + "jmapContactCardQueryChanges", + "jmapContactCardUpdate", + "jmapCoreEcho", + "jmapEmailChanges", + "jmapEmailCopy", + "jmapEmailCreate", + "jmapEmailDestroy", + "jmapEmailGet", + "jmapEmailImport", + "jmapEmailParse", + "jmapEmailQuery", + "jmapEmailQueryChanges", + "jmapEmailSubmissionChanges", + "jmapEmailSubmissionCreate", + "jmapEmailSubmissionDestroy", + "jmapEmailSubmissionGet", + "jmapEmailSubmissionQuery", + "jmapEmailSubmissionQueryChanges", + "jmapEmailSubmissionUpdate", + "jmapEmailUpdate", + "jmapFileNodeChanges", + "jmapFileNodeCopy", + "jmapFileNodeCreate", + "jmapFileNodeDestroy", + "jmapFileNodeGet", + "jmapFileNodeQuery", + "jmapFileNodeQueryChanges", + "jmapFileNodeUpdate", + "jmapIdentityChanges", + "jmapIdentityCreate", + "jmapIdentityDestroy", + "jmapIdentityGet", + "jmapIdentityUpdate", + "jmapMailboxChanges", + "jmapMailboxCreate", + "jmapMailboxDestroy", + "jmapMailboxGet", + "jmapMailboxQuery", + "jmapMailboxQueryChanges", + "jmapMailboxUpdate", + "jmapParticipantIdentityChanges", + "jmapParticipantIdentityCreate", + "jmapParticipantIdentityDestroy", + "jmapParticipantIdentityGet", + "jmapParticipantIdentityUpdate", + "jmapPrincipalChanges", + "jmapPrincipalCreate", + "jmapPrincipalDestroy", + "jmapPrincipalGet", + "jmapPrincipalGetAvailability", + "jmapPrincipalQuery", + "jmapPrincipalQueryChanges", + "jmapPrincipalUpdate", + "jmapPushSubscriptionCreate", + "jmapPushSubscriptionDestroy", + "jmapPushSubscriptionGet", + "jmapPushSubscriptionUpdate", + "jmapQuotaChanges", + "jmapQuotaGet", + "jmapQuotaQuery", + "jmapQuotaQueryChanges", + "jmapSearchSnippetGet", + "jmapShareNotificationChanges", + "jmapShareNotificationCreate", + "jmapShareNotificationDestroy", + "jmapShareNotificationGet", + "jmapShareNotificationQuery", + "jmapShareNotificationQueryChanges", + "jmapShareNotificationUpdate", + "jmapSieveScriptCreate", + "jmapSieveScriptDestroy", + "jmapSieveScriptGet", + "jmapSieveScriptQuery", + "jmapSieveScriptUpdate", + "jmapSieveScriptValidate", + "jmapThreadChanges", + "jmapThreadGet", + "jmapVacationResponseCreate", + "jmapVacationResponseDestroy", + "jmapVacationResponseGet", + "jmapVacationResponseUpdate", + "pop3Authenticate", + "pop3Dele", + "pop3List", + "pop3Retr", + "pop3Stat", + "pop3Uidl", + "sieveAuthenticate", + "sieveCheckScript", + "sieveDeleteScript", + "sieveGetScript", + "sieveHaveSpace", + "sieveListScripts", + "sievePutScript", + "sieveRenameScript", + "sieveSetActive", + "sysAccountPasswordGet", + "sysAccountPasswordUpdate", + "sysAccountSettingsGet", + "sysAccountSettingsUpdate", + "sysApiKeyCreate", + "sysApiKeyDestroy", + "sysApiKeyGet", + "sysApiKeyQuery", + "sysApiKeyUpdate", + "sysAppPasswordCreate", + "sysAppPasswordDestroy", + "sysAppPasswordGet", + "sysAppPasswordQuery", + "sysAppPasswordUpdate", + "sysArchivedItemCreate", + "sysArchivedItemDestroy", + "sysArchivedItemGet", + "sysArchivedItemQuery", + "sysArchivedItemUpdate", + "sysMaskedEmailCreate", + "sysMaskedEmailDestroy", + "sysMaskedEmailGet", + "sysMaskedEmailQuery", + "sysMaskedEmailUpdate", + "sysPublicKeyCreate", + "sysPublicKeyDestroy", + "sysPublicKeyGet", + "sysPublicKeyQuery", + "sysPublicKeyUpdate", + "sysSpamTrainingSampleDestroy", + "sysSpamTrainingSampleGet", + "sysSpamTrainingSampleQuery", + "sysSpamTrainingSampleUpdate" + ], + "inherits": [] + } + }, + "tenants": [ + "acme" + ] + }, + "alice": { + "contacts": [ + { + "emails": [ + "carol@example.net" + ], + "name": "Carol Client" + } + ], + "emails": [ + { + "hasAttachment": true, + "keywords": [], + "mailboxes": [ + "Inbox" + ], + "messageId": [ + "i1@example.net" + ], + "receivedAt": "2026-10-26T09:03:00Z", + "size": 525, + "subject": "Invoice attached", + "thread": [ + "Invoice attached" + ] + }, + { + "hasAttachment": false, + "keywords": [ + "$flagged" + ], + "mailboxes": [ + "Projects" + ], + "messageId": [ + "k1@example.net" + ], + "receivedAt": "2026-10-26T09:02:00Z", + "size": 230, + "subject": "Project kickoff", + "thread": [ + "Project kickoff" + ] + }, + { + "hasAttachment": false, + "keywords": [ + "$seen" + ], + "mailboxes": [ + "Inbox" + ], + "messageId": [ + "q1@example.net" + ], + "receivedAt": "2026-10-26T09:00:00Z", + "size": 246, + "subject": "Quarterly numbers", + "thread": [ + "Quarterly numbers", + "Re: Quarterly numbers" + ] + }, + { + "hasAttachment": false, + "keywords": [ + "$answered", + "$seen" + ], + "mailboxes": [ + "Inbox" + ], + "messageId": [ + "q2@example.org" + ], + "receivedAt": "2026-10-26T09:01:00Z", + "size": 293, + "subject": "Re: Quarterly numbers", + "thread": [ + "Quarterly numbers", + "Re: Quarterly numbers" + ] + } + ], + "events": [ + { + "duration": "PT15M", + "rrule": "weekly", + "start": "2026-11-02T09:30:00", + "timeZone": "America/Phoenix", + "title": "Team standup" + } + ], + "mailboxes": { + "Deleted Items": { + "role": "trash", + "total": 0, + "unread": 0 + }, + "Drafts": { + "role": "drafts", + "total": 0, + "unread": 0 + }, + "Inbox": { + "role": "inbox", + "total": 3, + "unread": 1 + }, + "Junk Mail": { + "role": "junk", + "total": 0, + "unread": 0 + }, + "Projects": { + "role": null, + "total": 1, + "unread": 1 + }, + "Projects/2026": { + "role": null, + "total": 0, + "unread": 0 + }, + "Sent Items": { + "role": "sent", + "total": 0, + "unread": 0 + } + }, + "sharedAccounts": [ + "bob@example.org", + "sales@example.org" + ], + "sieve": [ + { + "content": "require [\"fileinto\", \"vnd.stalwart.expressions\"];\r\nlet \"who\" \"envelope.from\";\r\nif header :contains \"subject\" \"kickoff\" { fileinto \"Projects\"; }\r\n", + "isActive": true, + "name": "filters" + }, + { + "content": "require [\"vacation\", \"relational\", \"date\"];\r\n\r\nvacation :mime :subject \"Away\" \"Content-Type: text/plain; charset=\\\"utf-8\\\"\r\nContent-Transfer-Encoding: 7bit\r\n\r\nBack on Monday.\";\r\n", + "isActive": false, + "name": "vacation" + } + ], + "vacation": { + "isEnabled": false, + "subject": "Away", + "textBody": "Back on Monday." + } + }, + "bob": { + "contacts": [ + { + "emails": [ + "carol@example.net" + ], + "name": "Carol Client" + } + ], + "emails": [ + { + "hasAttachment": true, + "keywords": [], + "mailboxes": [ + "Inbox" + ], + "messageId": [ + "i1@example.net" + ], + "receivedAt": "2026-10-26T09:03:00Z", + "size": 525, + "subject": "Invoice attached", + "thread": [ + "Invoice attached" + ] + }, + { + "hasAttachment": false, + "keywords": [ + "$flagged" + ], + "mailboxes": [ + "Projects" + ], + "messageId": [ + "k1@example.net" + ], + "receivedAt": "2026-10-26T09:02:00Z", + "size": 230, + "subject": "Project kickoff", + "thread": [ + "Project kickoff" + ] + }, + { + "hasAttachment": false, + "keywords": [ + "$seen" + ], + "mailboxes": [ + "Inbox" + ], + "messageId": [ + "q1@example.net" + ], + "receivedAt": "2026-10-26T09:00:00Z", + "size": 246, + "subject": "Quarterly numbers", + "thread": [ + "Quarterly numbers", + "Re: Quarterly numbers" + ] + }, + { + "hasAttachment": false, + "keywords": [ + "$answered", + "$seen" + ], + "mailboxes": [ + "Inbox" + ], + "messageId": [ + "q2@example.org" + ], + "receivedAt": "2026-10-26T09:01:00Z", + "size": 293, + "subject": "Re: Quarterly numbers", + "thread": [ + "Quarterly numbers", + "Re: Quarterly numbers" + ] + } + ], + "events": [ + { + "duration": "PT15M", + "rrule": "weekly", + "start": "2026-11-02T09:30:00", + "timeZone": "America/Phoenix", + "title": "Team standup" + } + ], + "mailboxes": { + "Deleted Items": { + "role": "trash", + "total": 0, + "unread": 0 + }, + "Drafts": { + "role": "drafts", + "total": 0, + "unread": 0 + }, + "Inbox": { + "role": "inbox", + "total": 3, + "unread": 1 + }, + "Junk Mail": { + "role": "junk", + "total": 0, + "unread": 0 + }, + "Projects": { + "role": null, + "total": 1, + "unread": 1 + }, + "Projects/2026": { + "role": null, + "total": 0, + "unread": 0 + }, + "Sent Items": { + "role": "sent", + "total": 0, + "unread": 0 + } + }, + "sharedAccounts": [ + "sales@example.org" + ], + "sieve": [ + { + "content": "require [\"fileinto\", \"vnd.stalwart.expressions\"];\r\nlet \"who\" \"envelope.from\";\r\nif header :contains \"subject\" \"kickoff\" { fileinto \"Projects\"; }\r\n", + "isActive": true, + "name": "filters" + }, + { + "content": "require [\"vacation\", \"relational\", \"date\"];\r\n\r\nvacation :mime :subject \"Away\" \"Content-Type: text/plain; charset=\\\"utf-8\\\"\r\nContent-Transfer-Encoding: 7bit\r\n\r\nBack on Monday.\";\r\n", + "isActive": false, + "name": "vacation" + } + ], + "vacation": { + "isEnabled": false, + "subject": "Away", + "textBody": "Back on Monday." + } + }, + "carol": { + "contacts": [], + "emails": [], + "events": [], + "mailboxes": { + "Deleted Items": { + "role": "trash", + "total": 0, + "unread": 0 + }, + "Drafts": { + "role": "drafts", + "total": 0, + "unread": 0 + }, + "Inbox": { + "role": "inbox", + "total": 0, + "unread": 0 + }, + "Junk Mail": { + "role": "junk", + "total": 0, + "unread": 0 + }, + "Sent Items": { + "role": "sent", + "total": 0, + "unread": 0 + } + }, + "sharedAccounts": [], + "sieve": [], + "vacation": { + "isEnabled": false, + "subject": null, + "textBody": null + } + } +} diff --git a/tests/resources/import-0.16.27/export/subspace_a b/tests/resources/import-0.16.27/export/subspace_a new file mode 100644 index 00000000..42cf6a54 Binary files /dev/null and b/tests/resources/import-0.16.27/export/subspace_a differ diff --git a/tests/resources/import-0.16.27/export/subspace_b b/tests/resources/import-0.16.27/export/subspace_b new file mode 100644 index 00000000..d65983d1 Binary files /dev/null and b/tests/resources/import-0.16.27/export/subspace_b differ diff --git a/tests/resources/import-0.16.27/export/subspace_d b/tests/resources/import-0.16.27/export/subspace_d new file mode 100644 index 00000000..7c78715d Binary files /dev/null and b/tests/resources/import-0.16.27/export/subspace_d differ diff --git a/tests/resources/import-0.16.27/export/subspace_e b/tests/resources/import-0.16.27/export/subspace_e new file mode 100644 index 00000000..07923465 Binary files /dev/null and b/tests/resources/import-0.16.27/export/subspace_e differ diff --git a/tests/resources/import-0.16.27/export/subspace_f b/tests/resources/import-0.16.27/export/subspace_f new file mode 100644 index 00000000..389fd113 Binary files /dev/null and b/tests/resources/import-0.16.27/export/subspace_f differ diff --git a/tests/resources/import-0.16.27/export/subspace_g b/tests/resources/import-0.16.27/export/subspace_g new file mode 100644 index 00000000..8acf24f9 Binary files /dev/null and b/tests/resources/import-0.16.27/export/subspace_g differ diff --git a/tests/resources/import-0.16.27/export/subspace_h b/tests/resources/import-0.16.27/export/subspace_h new file mode 100644 index 00000000..bb9a8efd Binary files /dev/null and b/tests/resources/import-0.16.27/export/subspace_h differ diff --git a/tests/resources/import-0.16.27/export/subspace_i b/tests/resources/import-0.16.27/export/subspace_i new file mode 100644 index 00000000..9321a4d9 Binary files /dev/null and b/tests/resources/import-0.16.27/export/subspace_i differ diff --git a/tests/resources/import-0.16.27/export/subspace_k b/tests/resources/import-0.16.27/export/subspace_k new file mode 100644 index 00000000..05e637e9 Binary files /dev/null and b/tests/resources/import-0.16.27/export/subspace_k differ diff --git a/tests/resources/import-0.16.27/export/subspace_l b/tests/resources/import-0.16.27/export/subspace_l new file mode 100644 index 00000000..2c6bd868 Binary files /dev/null and b/tests/resources/import-0.16.27/export/subspace_l differ diff --git a/tests/resources/import-0.16.27/export/subspace_n b/tests/resources/import-0.16.27/export/subspace_n new file mode 100644 index 00000000..45ccff85 Binary files /dev/null and b/tests/resources/import-0.16.27/export/subspace_n differ diff --git a/tests/resources/import-0.16.27/export/subspace_o b/tests/resources/import-0.16.27/export/subspace_o new file mode 100644 index 00000000..11d862ca Binary files /dev/null and b/tests/resources/import-0.16.27/export/subspace_o differ diff --git a/tests/resources/import-0.16.27/export/subspace_p b/tests/resources/import-0.16.27/export/subspace_p new file mode 100644 index 00000000..25566ff1 Binary files /dev/null and b/tests/resources/import-0.16.27/export/subspace_p differ diff --git a/tests/resources/import-0.16.27/export/subspace_q b/tests/resources/import-0.16.27/export/subspace_q new file mode 100644 index 00000000..ec9dc540 Binary files /dev/null and b/tests/resources/import-0.16.27/export/subspace_q differ diff --git a/tests/resources/import-0.16.27/export/subspace_r b/tests/resources/import-0.16.27/export/subspace_r new file mode 100644 index 00000000..a1f1a72f Binary files /dev/null and b/tests/resources/import-0.16.27/export/subspace_r differ diff --git a/tests/resources/import-0.16.27/export/subspace_s b/tests/resources/import-0.16.27/export/subspace_s new file mode 100644 index 00000000..992a790d Binary files /dev/null and b/tests/resources/import-0.16.27/export/subspace_s differ diff --git a/tests/resources/import-0.16.27/export/subspace_t b/tests/resources/import-0.16.27/export/subspace_t new file mode 100644 index 00000000..ef42701a Binary files /dev/null and b/tests/resources/import-0.16.27/export/subspace_t differ diff --git a/tests/resources/import-0.16.27/export/subspace_u b/tests/resources/import-0.16.27/export/subspace_u new file mode 100644 index 00000000..1d92eb54 Binary files /dev/null and b/tests/resources/import-0.16.27/export/subspace_u differ diff --git a/tests/resources/import-0.16.27/export/subspace_x b/tests/resources/import-0.16.27/export/subspace_x new file mode 100644 index 00000000..f6728b5e Binary files /dev/null and b/tests/resources/import-0.16.27/export/subspace_x differ diff --git a/tests/resources/import-0.16.27/known-differences.txt b/tests/resources/import-0.16.27/known-differences.txt new file mode 100644 index 00000000..8a06b662 --- /dev/null +++ b/tests/resources/import-0.16.27/known-differences.txt @@ -0,0 +1,42 @@ +# What inbuxa adds when it first starts on a 0.16.27 store: its own +# Compliance Officer role, and its own permissions on the built-in +# administrator roles. Read by tools/import-check/compare.py; one +# difference per line, as it prints them. ++ /admin/roles/Compliance Officer ++ /admin/roles/System Administrator/enabled sysAccountLockCreate ++ /admin/roles/System Administrator/enabled sysAccountLockDestroy ++ /admin/roles/System Administrator/enabled sysAccountLockGet ++ /admin/roles/System Administrator/enabled sysAccountLockUpdate ++ /admin/roles/System Administrator/enabled sysAiExplain ++ /admin/roles/System Administrator/enabled sysAuditExport ++ /admin/roles/System Administrator/enabled sysAuditGet ++ /admin/roles/System Administrator/enabled sysAuditSettingsUpdate ++ /admin/roles/System Administrator/enabled sysComplianceGet ++ /admin/roles/System Administrator/enabled sysDeliverabilityCheck ++ /admin/roles/System Administrator/enabled sysDeliverabilityGet ++ /admin/roles/System Administrator/enabled sysDeliverabilityUpdate ++ /admin/roles/System Administrator/enabled sysDlpPolicyGet ++ /admin/roles/System Administrator/enabled sysDlpPolicyUpdate ++ /admin/roles/System Administrator/enabled sysDlpReviewGet ++ /admin/roles/System Administrator/enabled sysDlpReviewUpdate ++ /admin/roles/System Administrator/enabled sysJournalGet ++ /admin/roles/System Administrator/enabled sysJournalUpdate ++ /admin/roles/System Administrator/enabled sysLegalHoldCreate ++ /admin/roles/System Administrator/enabled sysLegalHoldExport ++ /admin/roles/System Administrator/enabled sysLegalHoldGet ++ /admin/roles/System Administrator/enabled sysLegalHoldUpdate ++ /admin/roles/System Administrator/enabled sysMailRuleGet ++ /admin/roles/System Administrator/enabled sysMailRuleUpdate ++ /admin/roles/System Administrator/enabled sysScheduledReportGet ++ /admin/roles/System Administrator/enabled sysScheduledReportUpdate ++ /admin/roles/System Administrator/enabled sysSecurityAccept ++ /admin/roles/Tenant Administrator/enabled sysAccountLockCreate ++ /admin/roles/Tenant Administrator/enabled sysAccountLockDestroy ++ /admin/roles/Tenant Administrator/enabled sysAccountLockGet ++ /admin/roles/Tenant Administrator/enabled sysAccountLockUpdate ++ /admin/roles/Tenant Administrator/enabled sysAuditExport ++ /admin/roles/Tenant Administrator/enabled sysAuditGet ++ /admin/roles/Tenant Administrator/enabled sysComplianceGet ++ /admin/roles/Tenant Administrator/enabled sysDeliverabilityGet ++ /admin/roles/Tenant Administrator/enabled sysScheduledReportGet ++ /admin/roles/Tenant Administrator/enabled sysScheduledReportUpdate diff --git a/tools/import-check/after.py b/tools/import-check/after.py new file mode 100755 index 00000000..39a4320c --- /dev/null +++ b/tools/import-check/after.py @@ -0,0 +1,66 @@ +#!/usr/bin/env python3 +# SPDX-FileCopyrightText: 2026 Coffey Labs LLC +# SPDX-License-Identifier: AGPL-3.0-only +"""What happens on the imported store after the move: delivery through the +imported Sieve filter, search over imported mail, and re-saving a filter that +uses a pre-rename extension name.""" +import json, smtplib, sys, time, os +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from jmap import Jmap + +BASE = sys.argv[1] +alice = Jmap(BASE, "alice@example.org", "alice-fixture-pass-1") +acct = alice.account_id() + +failed = [] + + +def ok(label, cond, detail=""): + print(("PASS " if cond else "FAIL ") + label + (f": {detail}" if detail and not cond else "")) + if not cond: + failed.append(label) + +# 1. Delivery: the imported filter files "kickoff" mail into Projects. +with smtplib.SMTP("127.0.0.1", 18025, timeout=300) as s: + s.sendmail("dave@example.net", ["alice@example.org"], + b"From: dave@example.net\r\nTo: alice@example.org\r\nSubject: Second kickoff\r\n" + b"Message-ID: \r\n\r\nAgain.\r\n") +found = [] +for _ in range(30): + r = alice.call(("Email/query", {"accountId": acct, "filter": {"subject": "Second kickoff"}}), + ("Email/get", {"accountId": acct, "#ids": {"resultOf": "0", "name": "Email/query", "path": "/ids"}, + "properties": ["mailboxIds"]}), + ("Mailbox/get", {"accountId": acct, "ids": None, "properties": ["name"]})) + found = r[1]["list"] + if found: + break + time.sleep(1) +names = {m["id"]: m["name"] for m in r[2]["list"]} +ok("delivery after import", bool(found)) +ok("imported filter ran", bool(found) and [names[b] for b in found[0]["mailboxIds"]] == ["Projects"], + found and [names[b] for b in found[0]["mailboxIds"]]) + +# 2. Search over imported mail, once the queued reindex has run. +hits = [] +# Each account's reindex is spread over the first five minutes after start. +for _ in range(400): + hits = alice.call(("Email/query", {"accountId": acct, "filter": {"text": "quarterly"}}))[0]["ids"] + if len(hits) >= 2: + break + time.sleep(1) +ok("search finds imported mail", len(hits) >= 2, f"{len(hits)} of 2 found") + +# 3. Re-saving the imported filter, which says vnd.stalwart.expressions. +s = alice.call(("SieveScript/get", {"accountId": acct, "ids": None}))[0]["list"] +script = next(x for x in s if x["name"] == "filters") +blob = alice.upload(b'require ["fileinto", "vnd.stalwart.expressions"];\r\nlet "who" "envelope.from";\r\n' + b'if header :contains "subject" "kickoff" { fileinto "Projects"; }\r\n', "application/sieve") +r = alice.call(("SieveScript/set", {"accountId": acct, "update": {script["id"]: {"blobId": blob}}}))[0] +ok("re-saving a pre-rename filter", not r.get("notUpdated"), r.get("notUpdated")) + +# 4. The 0.16 web interface left in the registry. +A = json.load(open(os.path.join(os.environ["FIXTURE"], "admin.json"))) +adm = Jmap(BASE, A["username"], A["secret"], ["urn:inbuxa:jmap:registry"]) +apps = adm.call(("x:Application/get", {"ids": None}))[0]["list"] +print("INFO applications:", [(a.get("description"), a.get("enabled"), a.get("resourceUrl")) for a in apps]) +sys.exit(1 if failed else 0) diff --git a/tools/import-check/bootstrap.py b/tools/import-check/bootstrap.py new file mode 100755 index 00000000..237423ed --- /dev/null +++ b/tools/import-check/bootstrap.py @@ -0,0 +1,14 @@ +#!/usr/bin/env python3 +# SPDX-FileCopyrightText: 2026 Coffey Labs LLC +# SPDX-License-Identifier: AGPL-3.0-only +"""Run the 0.16.27 setup wizard over JMAP and save the administrator it creates.""" +import sys, json, os +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from jmap import Jmap +j = Jmap(sys.argv[1], "admin", os.environ["BOOT_PASSWORD"], ["urn:stalwart:jmap"]) +r = j.call(("x:Bootstrap/set", {"update": {"singleton": { + "serverHostname": "mail.example.org", "defaultDomain": "example.org", + "requestTlsCertificate": False, "generateDkimKeys": True, + "tracer": {"@type": "Stdout", "enable": True, "level": "info", "ansi": False, + "multiline": False, "lossy": False, "events": {}, "eventsPolicy": "exclude"}}}})) +json.dump(j.set_ok(r[0], "updated")["singleton"], open(os.path.join(os.environ["FIXTURE"], "admin.json"), "w"), indent=2) diff --git a/tools/import-check/check.sh b/tools/import-check/check.sh new file mode 100755 index 00000000..e4864178 --- /dev/null +++ b/tools/import-check/check.sh @@ -0,0 +1,46 @@ +#!/usr/bin/env bash +# SPDX-FileCopyrightText: 2026 Coffey Labs LLC +# SPDX-License-Identifier: AGPL-3.0-only +# +# Check that an inbuxa image still reads a Stalwart 0.16.27 store: +# +# tools/import-check/check.sh registry.coffeylabs.org/inbuxa/inbuxa-server:27.1-rc.5 +# +# Imports tests/resources/import-0.16.27/export into a fresh store with +# `inbuxa --import`, starts the image on it, and compares what it serves with +# what 0.16.27 served (expected.json, less known-differences.txt). Then it +# checks what happens after the move: mail delivered through the imported +# Sieve filter, search over imported mail once the reindex has run (up to +# five minutes), and re-saving a filter that uses a pre-rename extension name. +# Run it on every release candidate; see docs/spec/import-0.16.md. +# +# Needs docker, python3, loopback ports 18091 and 18025, and DNS (delivery +# runs the SPF/DKIM/DMARC checks). +set -euo pipefail +HERE=$(cd "$(dirname "$0")" && pwd) +ROOT=$(cd "$HERE/../.." && pwd) +export FIXTURE="${FIXTURE:-$ROOT/tests/resources/import-0.16.27}" +IMAGE=${1:?usage: check.sh } +NAME=inbuxa-import-check URL=http://127.0.0.1:18091 +WORK=$(mktemp -d) +trap 'docker rm -f $NAME >/dev/null 2>&1 || true; rm -rf "$WORK"' EXIT +mkdir -p "$WORK/etc" "$WORK/data" && chmod 777 "$WORK/etc" "$WORK/data" +echo '{"@type":"RocksDb","path":"/var/lib/inbuxa/"}' > "$WORK/etc/config.json" && chmod 644 "$WORK/etc/config.json" + +docker run --rm -v "$WORK/etc:/etc/inbuxa" -v "$WORK/data:/var/lib/inbuxa" -v "$FIXTURE/export:/import:ro" \ + $IMAGE --config /etc/inbuxa/config.json --import /import | tail -1 +# Basic auth on JMAP is refused by default (contract C-23); the check signs in +# with the fixture's passwords. GitHub stays unreachable, so the 0.16 web +# interface the store still lists is never fetched. +docker run -d --name $NAME -e INBUXA_HTTP_BASIC_AUTH=all --add-host github.com:127.0.0.1 \ + -p 127.0.0.1:18091:8080 -p 127.0.0.1:18025:25 \ + -v "$WORK/etc:/etc/inbuxa" -v "$WORK/data:/var/lib/inbuxa" $IMAGE >/dev/null +for _ in $(seq 1 60); do curl -sf -o /dev/null $URL/healthz/live && break; sleep 1; done +sleep 5 + +python3 -I "$HERE/snapshot.py" $URL > "$WORK/actual.json" +status=0 +python3 -I "$HERE/compare.py" "$FIXTURE/expected.json" "$WORK/actual.json" "$FIXTURE/known-differences.txt" || status=1 +python3 -I "$HERE/after.py" $URL || status=1 +docker logs $NAME 2>&1 | grep -E " (ERROR|WARN) " | grep -vE "resource.error|tls.no-certificates|DnsResolver" | head -20 || true +exit $status diff --git a/tools/import-check/compare.py b/tools/import-check/compare.py new file mode 100755 index 00000000..3bd62228 --- /dev/null +++ b/tools/import-check/compare.py @@ -0,0 +1,44 @@ +#!/usr/bin/env python3 +# SPDX-FileCopyrightText: 2026 Coffey Labs LLC +# SPDX-License-Identifier: AGPL-3.0-only +"""Diff two fixture snapshots. Every difference is printed; the ones listed in +the known-differences file (what inbuxa adds on start) are marked and don't +fail the check, and a listed one that no longer happens is reported too. + + compare.py expected.json actual.json [known-differences.txt] +""" +import json, sys + + +def differences(x, y, path=""): + if isinstance(x, dict) and isinstance(y, dict): + for k in sorted(set(x) | set(y)): + p = f"{path}/{k}" + if k not in x: + yield f"+ {p}" + elif k not in y: + yield f"- {p}" + else: + yield from differences(x[k], y[k], p) + elif isinstance(x, list) and isinstance(y, list) and all(isinstance(v, str) for v in x + y): + for v in sorted(set(x) - set(y)): + yield f"- {path} {v}" + for v in sorted(set(y) - set(x)): + yield f"+ {path} {v}" + elif x != y: + yield f"~ {path} {json.dumps(x)} => {json.dumps(y)}" + + +expected, actual = (json.load(open(f)) for f in sys.argv[1:3]) +known = set() +if len(sys.argv) > 3: + known = {line.strip() for line in open(sys.argv[3]) if line.strip() and not line.startswith("#")} + +found = list(differences(expected, actual)) +unexpected = [d for d in found if d not in known] +for d in found: + print((" known " if d in known else " NEW ") + d) +for d in sorted(known - set(found)): + print(" GONE " + d) +print(f"{len(found)} difference(s), {len(unexpected)} not known, {len(known - set(found))} known but gone") +sys.exit(1 if unexpected else 0) diff --git a/tools/import-check/jmap.py b/tools/import-check/jmap.py new file mode 100755 index 00000000..de5a892f --- /dev/null +++ b/tools/import-check/jmap.py @@ -0,0 +1,69 @@ +#!/usr/bin/env python3 +# SPDX-FileCopyrightText: 2026 Coffey Labs LLC +# SPDX-License-Identifier: AGPL-3.0-only +"""Tiny stdlib JMAP client for the 0.16.27 import fixture rig.""" +import base64, json, urllib.request, urllib.error + +CAPS = [ + "urn:ietf:params:jmap:core", "urn:ietf:params:jmap:mail", + "urn:ietf:params:jmap:submission", "urn:ietf:params:jmap:sieve", + "urn:ietf:params:jmap:calendars", "urn:ietf:params:jmap:contacts", + "urn:ietf:params:jmap:principals", "urn:ietf:params:jmap:vacationresponse", +] + + +class Jmap: + def __init__(self, base, user, password, extra_caps=()): + self.base, self.user = base.rstrip("/"), user + tok = base64.b64encode(f"{user}:{password}".encode()).decode() + self.auth = {"Authorization": f"Basic {tok}"} + self.caps = list(CAPS) + list(extra_caps) + self._session = None + + def _req(self, method, path, body=None, ctype="application/json"): + url = path if path.startswith("http") else self.base + path + data = body if isinstance(body, (bytes, type(None))) else json.dumps(body).encode() + req = urllib.request.Request(url, data=data, method=method, + headers={**self.auth, "Content-Type": ctype}) + try: + with urllib.request.urlopen(req, timeout=60) as r: + raw = r.read() + except urllib.error.HTTPError as e: + raise RuntimeError(f"{method} {path}: HTTP {e.code}: {e.read()[:500]!r}") + return json.loads(raw) if raw else None + + @property + def session(self): + if self._session is None: + self._session = self._req("GET", "/jmap/session") + return self._session + + def account_id(self, cap="urn:ietf:params:jmap:mail"): + return self.session["primaryAccounts"].get(cap) or next(iter(self.session["accounts"])) + + def call(self, *calls): + out = self._req("POST", "/jmap/", {"using": self.caps, "methodCalls": [ + [name, args, str(i)] for i, (name, args) in enumerate(calls)]}) + res = [] + for name, args, _ in out["methodResponses"]: + if name == "error": + raise RuntimeError(f"JMAP error: {json.dumps(args)[:800]}") + res.append(args) + return res + + def set_ok(self, resp, key="created"): + bad = {k: v for k, v in resp.items() if k.startswith("not") and v} + if bad: + raise RuntimeError(f"set failed: {json.dumps(bad)[:1200]}") + return resp.get(key) or {} + + def upload(self, data, ctype="application/octet-stream", account=None): + account = account or self.account_id() + url = self.local(self.session["uploadUrl"].replace("{accountId}", account)) + return self._req("POST", url, data, ctype)["blobId"] + + def local(self, url): + # The session names the public hostname; the rig is on loopback. + import urllib.parse + u = urllib.parse.urlsplit(url) + return self.base + urllib.parse.urlunsplit(("", "", u.path, u.query, "")) diff --git a/tools/import-check/make-fixture.sh b/tools/import-check/make-fixture.sh new file mode 100755 index 00000000..5d76217e --- /dev/null +++ b/tools/import-check/make-fixture.sh @@ -0,0 +1,47 @@ +#!/usr/bin/env bash +# SPDX-FileCopyrightText: 2026 Coffey Labs LLC +# SPDX-License-Identifier: AGPL-3.0-only +# +# Build the Stalwart 0.16.27 import fixture in tests/resources/import-0.16.27/: +# a fresh 0.16.27 server from the official image, set up and filled over JMAP +# (populate.py), snapshotted (expected.json), then exported with its own +# --export. The export is what an operator moving from 0.16.27 hands to +# `inbuxa --import`. +# +# The server gets no internet: GitHub resolves to loopback and DNS goes +# nowhere, so the web interface and data files it fetches at first start +# never land in the fixture. Only needed when the fixture itself changes; +# check.sh is what runs against a release. +set -euo pipefail +HERE=$(cd "$(dirname "$0")" && pwd) +ROOT=$(cd "$HERE/../.." && pwd) +export FIXTURE="${FIXTURE:-$ROOT/tests/resources/import-0.16.27}" +IMAGE=stalwartlabs/stalwart@sha256:c429b311c93db5606598abc1c68225a85a6953183848ae01bee9c6a98885836c # v0.16.27 +NAME=inbuxa-import-fixture PORT=18090 URL=http://127.0.0.1:18090 +WORK=$(mktemp -d) +trap 'docker rm -f $NAME >/dev/null 2>&1 || true; rm -rf "$WORK"' EXIT +mkdir -p "$WORK/etc" "$WORK/data" "$WORK/export" && chmod 777 "$WORK"/* +export BOOT_PASSWORD=$(head -c 18 /dev/urandom | base64 | tr -d '/+=') + +wait_live() { + for _ in $(seq 1 60); do curl -sf -o /dev/null $URL/healthz/live && return; sleep 1; done + echo "server never came up" >&2; docker logs $NAME | tail -20 >&2; exit 1 +} +run() { + docker rm -f $NAME >/dev/null 2>&1 || true + docker run -d --name $NAME --dns 127.0.0.1 --add-host github.com:127.0.0.1 \ + --add-host objects.githubusercontent.com:127.0.0.1 -p 127.0.0.1:$PORT:8080 \ + -v "$WORK/etc:/etc/stalwart" -v "$WORK/data:/var/lib/stalwart" "$@" $IMAGE >/dev/null + wait_live +} + +run -e STALWART_RECOVERY_ADMIN="admin:$BOOT_PASSWORD" +python3 -I "$HERE/bootstrap.py" $URL +run +python3 -I "$HERE/populate.py" $URL +python3 -I "$HERE/snapshot.py" $URL > "$FIXTURE/expected.json" +docker rm -f $NAME >/dev/null +docker run --rm -v "$WORK/etc:/etc/stalwart" -v "$WORK/data:/var/lib/stalwart" -v "$WORK/export:/export" \ + $IMAGE --config /etc/stalwart/config.json --export /export >/dev/null +rm -rf "$FIXTURE/export" && cp -r "$WORK/export" "$FIXTURE/export" +echo "fixture written: $(du -sh "$FIXTURE/export" | cut -f1) export, expected.json, admin.json" diff --git a/tools/import-check/populate.py b/tools/import-check/populate.py new file mode 100755 index 00000000..098a6dc2 --- /dev/null +++ b/tools/import-check/populate.py @@ -0,0 +1,147 @@ +#!/usr/bin/env python3 +# SPDX-FileCopyrightText: 2026 Coffey Labs LLC +# SPDX-License-Identifier: AGPL-3.0-only +"""Fill a fresh 0.16.27 server with a small, varied data set for the import fixture.""" +import json, sys, os + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from jmap import Jmap + +BASE = sys.argv[1] if len(sys.argv) > 1 else "http://127.0.0.1:18090" +A = json.load(open(os.path.join(os.environ["FIXTURE"], "admin.json"))) +adm = Jmap(BASE, A["username"], A["secret"], ["urn:stalwart:jmap"]) +PW = {"alice": "alice-fixture-pass-1", "bob": "bob-fixture-pass-2"} + +domain = adm.call(("x:Domain/query", {}))[0]["ids"][0] + +# Bootstrap makes an Ed25519 key, then an RSA one a moment later. +import time +for _ in range(60): + if len(adm.call(("x:DkimSignature/query", {}))[0]["ids"]) >= 2: + break + time.sleep(1) +else: + raise SystemExit("DKIM keys never appeared") + +# A tenant with its own domain and one user (multi-tenancy is AGPL in 0.16). +r = adm.call(("x:Tenant/set", {"create": {"t": {"name": "acme"}}})) +tenant = adm.set_ok(r[0])["t"]["id"] +r = adm.call(("x:Domain/set", {"create": {"d": {"name": "acme.example.com", "memberTenantId": tenant}}})) +acme_domain = adm.set_ok(r[0])["d"]["id"] + + +def user(name, extra=None): + u = {"@type": "User", "name": name, "domainId": domain, "description": f"{name.title()} Example", + "credentials": {"0": {"@type": "Password", "secret": PW[name]}}, + "roles": {"@type": "User"}, "permissions": {"@type": "Inherit"}, "quotas": {}, + "aliases": {}, "memberGroupIds": {}, "encryptionAtRest": {"@type": "Disabled"}} + u.update(extra or {}) + return u + + +existing = [x["id"] for x in adm.call(("x:Role/get", {"ids": None}))[0]["list"] if x.get("description") == "Helpdesk"] +r = [{"created": {"helpdesk": {"id": existing[0]}}}] if existing else adm.call( + ("x:Role/set", {"create": {"helpdesk": { + "description": "Helpdesk", "roleIds": {}, + "enabledPermissions": {"sysAccountGet": True, "sysAccountQuery": True, "sysQueuedMessageGet": True}, + "disabledPermissions": {}}}}), +) +role = adm.set_ok(r[0])["helpdesk"]["id"] +user_role = next(x["id"] for x in adm.call(("x:Role/get", {"ids": None}))[0]["list"] if x.get("description") == "User") +r = adm.call(("x:Account/set", {"create": {"sales": { + "@type": "Group", "name": "sales", "domainId": domain, "description": "Sales team", + "roles": {"@type": "Default"}, "permissions": {"@type": "Inherit"}, "quotas": {}, "aliases": {}}}})) +group = adm.set_ok(r[0])["sales"]["id"] +r = adm.call(("x:Account/set", {"create": { + "alice": user("alice", {"memberGroupIds": {group: True}, + "aliases": {"0": {"name": "a.example", "domainId": domain, "enabled": True, + "description": None}}, + "roles": {"@type": "Custom", "roleIds": {user_role: True, role: True}}}), + "bob": user("bob", {"memberGroupIds": {group: True}, "quotas": {"maxDiskQuota": 50_000_000}}), +}})) +ids = adm.set_ok(r[0]) +r = adm.call(("x:Account/set", {"create": {"carol": { + "@type": "User", "name": "carol", "domainId": acme_domain, "description": "Carol Acme", + "memberTenantId": tenant, "credentials": {"0": {"@type": "Password", "secret": "carol-fixture-pass-3"}}, + "roles": {"@type": "User"}, "permissions": {"@type": "Inherit"}, "quotas": {}, "aliases": {}, + "memberGroupIds": {}, "encryptionAtRest": {"@type": "Disabled"}}}})) +adm.set_ok(r[0]) +print("accounts", {k: v["id"] for k, v in ids.items()}, "group", group, "role", role) + +MSGS = [ + ("Quarterly numbers", "carol@example.net", "alice@example.org", "", None, + "Here are the numbers for the quarter.", ["$seen"], "Inbox"), + ("Re: Quarterly numbers", "alice@example.org", "carol@example.net", "", "", + "Thanks, looks good.", ["$seen", "$answered"], "Inbox"), + ("Project kickoff", "dave@example.net", "alice@example.org", "", None, + "Kickoff is Monday at 10.", ["$flagged"], "Projects"), + ("Invoice attached", "billing@example.net", "alice@example.org", "", None, + None, [], "Inbox"), +] + + +def rfc822(subject, frm, to, mid, reply, body, attach): + head = (f"From: {frm}\r\nTo: {to}\r\nSubject: {subject}\r\nMessage-ID: {mid}\r\n" + f"Date: Mon, 26 Oct 2026 09:00:00 +0000\r\nMIME-Version: 1.0\r\n") + if reply: + head += f"In-Reply-To: {reply}\r\nReferences: {reply}\r\n" + if not attach: + return (head + "Content-Type: text/plain; charset=utf-8\r\n\r\n" + body + "\r\n").encode() + return (head + 'Content-Type: multipart/mixed; boundary="b1"\r\n\r\n--b1\r\n' + "Content-Type: text/plain; charset=utf-8\r\n\r\nInvoice for October.\r\n--b1\r\n" + 'Content-Type: application/pdf; name="invoice.pdf"\r\nContent-Disposition: attachment; ' + 'filename="invoice.pdf"\r\nContent-Transfer-Encoding: base64\r\n\r\n' + "JVBERi0xLjQKJcOkw7zDtsOfCjEgMCBvYmoKPDw+PgplbmRvYmoKdHJhaWxlcgo8PD4+CiUlRU9GCg==\r\n" + "--b1--\r\n").encode() + + +def fill(name): + j = Jmap(BASE, f"{name}@example.org", PW[name]) + acct = j.account_id() + boxes = {m["name"]: m["id"] for m in j.call(("Mailbox/get", {"accountId": acct, "ids": None}))[0]["list"]} + r = j.call(("Mailbox/set", {"accountId": acct, "create": {"p": {"name": "Projects"}}})) + boxes["Projects"] = j.set_ok(r[0])["p"]["id"] + r = j.call(("Mailbox/set", {"accountId": acct, "create": {"c": {"name": "2026", "parentId": boxes["Projects"]}}})) + boxes["2026"] = j.set_ok(r[0])["c"]["id"] + imports = {} + for i, (subj, frm, to, mid, reply, body, kw, box) in enumerate(MSGS): + blob = j.upload(rfc822(subj, frm, to, mid, reply, body, body is None), "message/rfc822") + imports[str(i)] = {"blobId": blob, "mailboxIds": {boxes[box]: True}, + "keywords": {k: True for k in kw}, "receivedAt": f"2026-10-26T09:0{i}:00Z"} + j.set_ok(j.call(("Email/import", {"accountId": acct, "emails": imports}))[0]) + + cals = j.call(("Calendar/get", {"accountId": acct, "ids": None}))[0]["list"] + r = j.call(("CalendarEvent/set", {"accountId": acct, "create": {"e": { + "calendarIds": {cals[0]["id"]: True}, "@type": "Event", "title": "Team standup", + "start": "2026-11-02T09:30:00", "timeZone": "America/Phoenix", "duration": "PT15M", + "recurrenceRule": {"@type": "RecurrenceRule", "frequency": "weekly", "byDay": [{"@type": "NDay", "day": "mo"}]}}}})) + j.set_ok(r[0]) + books = j.call(("AddressBook/get", {"accountId": acct, "ids": None}))[0]["list"] + r = j.call(("ContactCard/set", {"accountId": acct, "create": {"c": { + "addressBookIds": {books[0]["id"]: True}, "@type": "Card", "version": "1.0", "kind": "individual", + "name": {"@type": "Name", "full": "Carol Client"}, + "emails": {"e1": {"@type": "EmailAddress", "address": "carol@example.net"}}}}})) + j.set_ok(r[0]) + + script = ('require ["fileinto", "vnd.stalwart.expressions"];\r\n' + 'let "who" "envelope.from";\r\n' + 'if header :contains "subject" "kickoff" { fileinto "Projects"; }\r\n').encode() + blob = j.upload(script, "application/sieve") + r = j.call(("SieveScript/set", {"accountId": acct, "create": {"s": {"name": "filters", "blobId": blob}}, + "onSuccessActivateScript": "#s"})) + j.set_ok(r[0]) + r = j.call(("VacationResponse/set", {"accountId": acct, "update": {"singleton": { + "isEnabled": False, "subject": "Away", "textBody": "Back on Monday."}}})) + j.set_ok(r[0], "updated") + return j, acct, boxes + + +alice, a_acct, a_boxes = fill("alice") +bob, b_acct, b_boxes = fill("bob") + +# bob shares his Projects folder with alice, read-only +principals = bob.call(("Principal/query", {"accountId": b_acct, "filter": {"email": "alice@example.org"}}))[0]["ids"] +r = bob.call(("Mailbox/set", {"accountId": b_acct, "update": {b_boxes["Projects"]: { + "shareWith": {principals[0]: {"mayReadItems": True}}}}})) +bob.set_ok(r[0], "updated") +print("populated") diff --git a/tools/import-check/snapshot.py b/tools/import-check/snapshot.py new file mode 100755 index 00000000..4706910b --- /dev/null +++ b/tools/import-check/snapshot.py @@ -0,0 +1,106 @@ +#!/usr/bin/env python3 +# SPDX-FileCopyrightText: 2026 Coffey Labs LLC +# SPDX-License-Identifier: AGPL-3.0-only +"""Print a server's fixture data as id-free JSON, so a 0.16.27 source and an +inbuxa target can be compared field by field.""" +import json, sys, urllib.request, os + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from jmap import Jmap + +BASE = sys.argv[1] +A = json.load(open(os.path.join(os.environ["FIXTURE"], "admin.json"))) +PW = {"alice": "alice-fixture-pass-1", "bob": "bob-fixture-pass-2", "carol": "carol-fixture-pass-3"} +DOMAIN = {"carol": "acme.example.com"} +REGISTRY_CAPS = ["urn:stalwart:jmap", "urn:inbuxa:jmap:registry"] + + +def admin_client(): + # The registry capability changed name; ask the session which one it has. + probe = Jmap(BASE, A["username"], A["secret"]) + caps = [c for c in REGISTRY_CAPS if c in probe.session["capabilities"]] + return Jmap(BASE, A["username"], A["secret"], caps) + + +def admin_view(): + j = admin_client() + r = j.call(("x:Domain/get", {"ids": None}), ("x:Account/get", {"ids": None}), + ("x:Role/get", {"ids": None}), ("x:DkimSignature/get", {"ids": None})) + domains = {d["id"]: d["name"] for d in r[0]["list"]} + names = {a["id"]: a["name"] for a in r[1]["list"]} + roles = {x["id"]: x.get("description") for x in r[2]["list"]} + accounts = {} + for a in r[1]["list"]: + role = a.get("roles") or {} + accounts[a["name"]] = { + "type": a["@type"], "description": a.get("description"), + "domain": domains.get(a.get("domainId")), "tenant": a.get("memberTenantId") is not None, + "aliases": sorted(f'{x["name"]}@{domains.get(x["domainId"])}' for x in (a.get("aliases") or {}).values()), + "groups": sorted(names.get(g, g) for g in (a.get("memberGroupIds") or {})), + "quotas": a.get("quotas") or {}, + "roles": [role.get("@type")] + sorted(roles.get(i, i) for i in (role.get("roleIds") or {})), + } + role_perms = {x["description"]: {"enabled": sorted(k for k, v in (x.get("enabledPermissions") or {}).items() if v), + "disabled": sorted(k for k, v in (x.get("disabledPermissions") or {}).items() if v), + "inherits": sorted(roles.get(i, i) for i in (x.get("roleIds") or {}))} + for x in r[2]["list"]} + dkim = sorted(f'{domains.get(d.get("domainId"), "?")} {d["@type"]}' for d in r[3]["list"]) + r = j.call(("x:Tenant/get", {"ids": None}), ("x:Application/get", {"ids": None})) + tenants = {t["id"]: t["name"] for t in r[0]["list"]} + apps = sorted(f'{x.get("description")} enabled={x.get("enabled")} {x.get("resourceUrl")}' for x in r[1]["list"]) + return {"domains": sorted(domains.values()), "accounts": accounts, "roles": role_perms, "dkim": dkim, + "tenants": sorted(t["name"] for t in r[0]["list"]), "applications": apps} + + +def user_view(name): + j = Jmap(BASE, f"{name}@{DOMAIN.get(name, 'example.org')}", PW[name]) + acct = j.account_id() + r = j.call(("Mailbox/get", {"accountId": acct, "ids": None})) + boxes = {m["id"]: m for m in r[0]["list"]} + + def path(mid): + m = boxes[mid] + return (path(m["parentId"]) + "/" if m.get("parentId") else "") + m["name"] + + mailboxes = {path(m["id"]): {"role": m.get("role"), "total": m["totalEmails"], "unread": m["unreadEmails"]} + for m in boxes.values()} + r = j.call(("Email/query", {"accountId": acct}), + ("Email/get", {"accountId": acct, "#ids": {"resultOf": "0", "name": "Email/query", "path": "/ids"}, + "properties": ["subject", "mailboxIds", "keywords", "threadId", "hasAttachment", + "size", "receivedAt", "messageId"]})) + threads = {} + for e in r[1]["list"]: + threads.setdefault(e["threadId"], []).append(e["subject"]) + emails = sorted(({"subject": e["subject"], "mailboxes": sorted(path(b) for b in e["mailboxIds"]), + "keywords": sorted(k for k in e["keywords"] if not k.startswith("$has")), + "thread": sorted(threads[e["threadId"]]), "hasAttachment": e["hasAttachment"], + "size": e["size"], "receivedAt": e["receivedAt"], "messageId": e["messageId"]} + for e in r[1]["list"]), key=lambda e: e["subject"]) + r = j.call(("CalendarEvent/get", {"accountId": acct, "ids": None}), + ("ContactCard/get", {"accountId": acct, "ids": None}), + ("SieveScript/get", {"accountId": acct, "ids": None}), + ("VacationResponse/get", {"accountId": acct, "ids": ["singleton"]})) + events = sorted([{"title": e.get("title"), "start": e.get("start"), "timeZone": e.get("timeZone"), + "duration": e.get("duration"), "rrule": (e.get("recurrenceRule") or {}).get("frequency")} + for e in r[0]["list"]], key=lambda e: e["title"] or "") + cards = sorted([{"name": (c.get("name") or {}).get("full"), + "emails": sorted(x["address"] for x in (c.get("emails") or {}).values())} + for c in r[1]["list"]], key=lambda c: c["name"] or "") + sieve = [] + for s in r[2]["list"]: + url = j.session["downloadUrl"].replace("{accountId}", acct).replace("{blobId}", s["blobId"]) \ + .replace("{name}", "s").replace("{type}", "application/sieve") + url = j.local(url) + req = urllib.request.Request(url, headers=j.auth) + sieve.append({"name": s["name"], "isActive": s["isActive"], + "content": urllib.request.urlopen(req).read().decode()}) + v = r[3]["list"][0] + vacation = {k: v.get(k) for k in ("isEnabled", "subject", "textBody")} + shared = sorted(a["name"] for k, a in j.session["accounts"].items() if k != acct) + return {"mailboxes": mailboxes, "emails": emails, "events": events, "contacts": cards, + "sieve": sorted(sieve, key=lambda s: s["name"]), "vacation": vacation, "sharedAccounts": shared} + + +print(json.dumps({"admin": admin_view(), "alice": user_view("alice"), "bob": user_view("bob"), + "carol": user_view("carol")}, + indent=2, sort_keys=True))