diff --git a/README.md b/README.md index b29bd30..b7656c3 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@

Licence: AGPL-3.0-or-later - Tested against Stalwart 0.16.19 and 0.15.5 + Requires Stalwart 0.16 or newer; tested against 0.16.19 by LINUXexpert.org

@@ -15,17 +15,28 @@ ihasmail is a JMAP-first web client: mail, calendars, contacts, files, filters and every other modern feature Stalwart exposes, in a responsive single-page app that works equally well on a desktop monitor and a phone. It talks only JMAP (plus Stalwart's blob/upload/EventSource endpoints) — no IMAP, no SMTP, no database. > Status: 2.0 rewrite, in QA against a live Stalwart server — **0.16.19** -> since 2026-08-25, 0.15.5 before that. The previous FastAPI/HTMX prototype +> since 2026-08-25. The previous FastAPI/HTMX prototype > has been removed entirely (only the logo survived, and it has since lost > the `.com` wordmark it used to carry — ihasmail is the software, not the > hosted instance). -ihasmail supports both generations of Stalwart, which are less alike than the -version numbers suggest: 0.16 replaced the REST management API with JMAP -registry objects, changed the shape of `FileNode`, split its rights up, and -moved configuration into the store. Where the two differ, ihasmail detects -which it is talking to rather than assuming — see [Known issues / pending -QA](#known-issues--pending-qa) for what is verified on which. +**ihasmail requires Stalwart 0.16 or newer.** Sign-in refuses anything older, +by name, rather than letting Files and credentials fail separately with +nothing to connect them. + +It used to support 0.15 as well. The two are less alike than the version +numbers suggest — 0.16 replaced the REST management API with JMAP registry +objects, changed the shape of `FileNode`, split its rights up, and moved +configuration into the store — and carrying both meant every call site had to +ask which one it was talking to. The cost was not the branches. It was that a +wrong answer had somewhere to fall back to, so it failed *quietly*: one +capability looked for in the wrong place downgraded every real 0.16 server to +the 0.15 path, and that reached production. With one supported generation a +wrong guess is a loud error on the first call instead. + +The last release that runs on 0.15 is tagged +[`stalwart-0.15-support`](https://github.com/LINUXexpert-org/ihasmail/releases/tag/stalwart-0.15-support), +if you are on 0.15 and cannot upgrade yet. The live instance was moved from 0.15.5 to 0.16.19 with [stalwart-migrator](https://github.com/LINUXexpert-org/stalwart-migrator), a @@ -80,10 +91,10 @@ seconds of downtime with nothing lost. **Settings** - **Dates & times**: language/region (every one of the ~620 locales CLDR has data for, each named in its own language and script), date order (locale default, `22.11.2025`, `22/11/2025`, `11/22/2025` or ISO `2025-11-22`) and 12h/24h clock, applied everywhere — message list and headers, calendar, contacts, files, sessions. The default comes from the locale configured for the account in Stalwart (`x:AccountSettings/get`, falling back to `x:Account/get`), and from the browser where the server will not say; POSIX forms are normalised (`de_DE.UTF-8` → `de-DE`) and script modifiers preserved (`sr_RS@latin` → `sr-Latn-RS`). Numerals follow the locale (`٢٢.١١.٢٠٢٥` for `ar-EG`), except under ISO 8601, which pins date *and* clock to Latin digits. Dates are **entered** through custom pickers in the same format (browsers render `` in their own locale and ignore the page's), with a calendar popover, a time list, keyboard navigation, and lenient typing — `22.11.`, `221125`, `6:23pm` and bare ISO all parse -- **Self-service credentials** in Settings › Security: change your password, manage **app passwords** (a separate password per mail app or device, revocable on its own), and turn **two-factor authentication** on or off by scanning a QR code. Enrolment codes are verified before anything is stored, so a mistyped key cannot lock you out, and switching 2FA on moves this browser's session onto a dedicated app password instead of signing you straight back out. Works against both Stalwart generations: the `x:AccountPassword` / `x:AppPassword` registry objects on 0.16+, and the `/api/account/auth` REST endpoint on 0.15.x (the latter confirmed live) +- **Self-service credentials** in Settings › Security: change your password, manage **app passwords** (a separate password per mail app or device, revocable on its own), and turn **two-factor authentication** on or off by scanning a QR code. Enrolment codes are verified before anything is stored, so a mistyped key cannot lock you out, and switching 2FA on moves this browser's session onto a dedicated app password instead of signing you straight back out. Built on the `x:AccountPassword` / `x:AppPassword` registry objects - **Light and dark** follow the system by default, with a toggle in the top bar for flipping between them and a three-way choice in Settings › Appearance - Identities & signatures, **Sieve filters** (visual rule builder that round-trips to a Sieve script, plus a raw script editor with server-side validation), out-of-office (`VacationResponse`), folders, labels, templates, notifications, calendar defaults, sessions (sign out other devices), keyboard shortcuts, import/export of settings -- **Settings follow the account, not the browser** (Stalwart 0.16+): they are kept in a `settings.json` in the account's own JMAP Files, so the default identity, locale, date and time formats, theme, labels, templates, folder colours and the rest are the same wherever you sign in — including a private window. ihasmail still stores nothing itself; the file lives in the mail store and is backed up with it. Settings that describe *this* screen or browser stay local, because syncing them would be wrong rather than helpful: list-pane sizes, density, font size, sidebar state, and the notification toggles (which track a permission the browser grants per-device). localStorage is kept as a cache so the first frame is already right, and the file corrects it a moment later. On Stalwart 0.15 nothing changes — settings stay local, as before +- **Settings follow the account, not the browser**: they are kept in a `settings.json` in the account's own JMAP Files, so the default identity, locale, date and time formats, theme, labels, templates, folder colours and the rest are the same wherever you sign in — including a private window. ihasmail still stores nothing itself; the file lives in the mail store and is backed up with it. Settings that describe *this* screen or browser stay local, because syncing them would be wrong rather than helpful: list-pane sizes, density, font size, sidebar state, and the notification toggles (which track a permission the browser grants per-device). localStorage is kept as a cache so the first frame is already right, and the file corrects it a moment later **Platform** - Installable PWA (manifest + service worker), mobile layout with bottom tab bar, drawer navigation, full-screen composer, FAB @@ -128,9 +139,6 @@ npm run dev # server on :8080 (tsx watch) + Vite dev server on :5173 # against the built-in mock Stalwart (demo@example.com / demo) — no real mailbox needed npm run dev:mock # mock on :8788, server on :8080, Vite on :5173 -# the same, with the mock impersonating Stalwart 0.15 instead of 0.16 -npm run dev:mock:legacy - # the same, with the mock advertising FUTURERELEASE but dropping every hold — # the shape of a real server whose `futureRelease` setting was never turned on npm run dev:mock:no-future-release @@ -143,31 +151,29 @@ npm start # serve the production build Open http://localhost:5173 in dev (or http://localhost:8080 for the production build). -### The mock, and which Stalwart it pretends to be +### The mock -`npm run mock` impersonates **0.16** by default; `MOCK_STALWART=0.15` (or -`npm run mock:legacy`) impersonates the generation before the registry. The -older mode is not a smaller mock — it reproduces the specific ways that -generation differs, none of which the server reports as an error: +`npm run mock` is an in-memory fake Stalwart 0.16 — enough of JMAP to develop +and demo against without a real mailbox. It models the things a real server +does that a naive fake would not, because each of these cost a live debugging +session to find: -- `urn:stalwart:jmap` is not a capability it knows, and naming one it cannot - parse fails the **whole request**, not the one call that wanted it. On 0.16 - it *is* known — but advertised per-account, in `primaryAccounts` and each - account's `accountCapabilities`, never in the session-level `capabilities`. - Stalwart validates `using` by parsing the urn rather than looking it up in - the session, so naming it works regardless; a client that tests for it in - the obvious place, though, mistakes every 0.16 server for an older one -- `x:` methods do not exist, so the registry — credentials, account settings — - is unreachable, and self-service credentials live at `POST /api/account/auth` -- `FileNode/query` masks its results to non-containers, so it returns files and - **never folders**, silently; `FileNode/get` has no such mask -- FileNode has no `nodeType` (a directory is a node with no file properties), - and rights are only `mayRead`/`mayWrite`/`mayShare` - -Both modes enforce the 2047-**byte** cap on identity signatures. Every one of -these cost a live debugging session against a real 0.15.5 server, because the -0.16-shaped mock could not express them; `server/src/account-legacy.test.ts` -now pins them. +- `urn:stalwart:jmap` is advertised **per-account**, in `primaryAccounts` and + each account's `accountCapabilities`, and never in the session-level + `capabilities`. That is where Stalwart actually puts it, and a client that + tests for it in the obvious place concludes it is talking to something far + older than it is. The mock used to advertise it in the wrong place, which is + exactly why nothing caught that bug +- identity signatures are capped at 2047 **bytes**, not characters +- `CalendarEvent/set` uses Stalwart's vocabulary, not RFC 8984's, and refuses + what the real server refuses — advertising the RFC spelling is how that one + reached a live server +- `MOCK_NO_FUTURE_RELEASE=1` (or `npm run mock:no-future-release`) advertises + FUTURERELEASE and then drops every hold, which is the shape of a real server + whose `futureRelease` setting was never turned on +- `MOCK_NO_REGISTRY=1` omits the Stalwart capability, so the sign-in refusal + for unsupported servers can be tested. That is all it does — the rest still + behaves like 0.16. Emulating 0.15 properly went with the support for it ## Configuration @@ -193,29 +199,22 @@ Press `?` anywhere. Highlights: `c` compose · `/` search · `j`/`k` navigate · ## Known issues / pending QA -The live instance ran **0.15.5** until 2026-08-25 and runs **0.16.19** now, -so both generations have been exercised against a real server. Everything -below says which. +The live instance runs **0.16.19**, and everything below says what has been +checked against it and what has not. -Verified against a live **0.15.5**: the mail flows, self-service credentials -over the REST path, Files, and signatures. - -The 0.16 registry path was previously recorded here as verified live. That -was wrong, and the entry below says why: ihasmail looked for -`urn:stalwart:jmap` in the session-level capabilities, where Stalwart has -never put it, so **every** real 0.16 server was taken for a pre-0.16 one. -Self-service credentials went to a REST endpoint 0.16 had removed, About -reported the wrong generation, and Files ran on the older code path. The mock -advertised the capability in the wrong place too, which is why nothing caught -it. Fixed, and the mock now advertises it where the real server does — but -the registry path is **awaiting live re-verification**. +Some entries record what a live **0.15.5** proved before that server was +upgraded on 2026-08-25. They are kept where the finding is about ihasmail +rather than about 0.15 — a byte cap that still applies, a flow that still +works the same way — and dropped where 0.15 was the whole subject. Support for +0.15 was removed on 2026-08-26; the last release that runs on it is tagged +[`stalwart-0.15-support`](https://github.com/LINUXexpert-org/ihasmail/releases/tag/stalwart-0.15-support). - **Read receipts are built here, not by the server** — JMAP has an extension for them, [RFC 9007](https://www.rfc-editor.org/rfc/rfc9007.html)'s `MDN/send`, and Stalwart does not implement it: `urn:ietf:params:jmap:mdn` is not among its capabilities. So ihasmail assembles the `multipart/report` itself and sends it the long way round — raw MIME uploaded as a blob, `Email/import`, then `EmailSubmission` — which is also why the receipt lands in Sent, where it honestly belongs. Non-ASCII parts are base64 rather than `8bit`, so nothing depends on 8BITMIME surviving every hop. There is deliberately no "always send" setting: a receipt confirms to whoever asked that the address is live and when it was read, to an address of the sender's choosing, so each one is a decision. Verified against the mock end to end (upload, import, submit, `$mdnsent`); **not yet exercised against the live server**. -- **Where 0.16 advertises `urn:stalwart:jmap`** — not where a JMAP client would look. Stalwart builds the session-level `capabilities` from a fixed list (`Session::new`, plus WebSocket) that has never contained this capability, in any 0.16.x from 0.16.0 to 0.16.19. It hands it out per-account instead, so it appears in `primaryAccounts` and in each account's `accountCapabilities`. ihasmail tested for it in `capabilities` alone, which made every real 0.16 server read as pre-0.16 — and that one check drove three things: self-service credentials fell back to `POST /api/account/auth`, which 0.16 removed, so password changes, 2FA and app passwords all failed with "this mail server does not offer self-service credential management"; About reported the wrong generation; and Files took the pre-0.16 code path. It now looks in all three places. Two related soft spots went with it: a transport error while probing the registry no longer downgrades a server to the legacy REST path (which would have posted the current password to an endpoint that is not there), and a locale request that is merely refused no longer discards a generation the capability had already settled. +- **Where 0.16 advertises `urn:stalwart:jmap`** — not where a JMAP client would look, and this now decides whether a sign-in is allowed at all. Stalwart builds the session-level `capabilities` from a fixed list (`Session::new`, plus WebSocket) that has never contained this capability, in any 0.16.x from 0.16.0 to 0.16.19. It hands it out per-account instead, so it appears in `primaryAccounts` and in each account's `accountCapabilities`. ihasmail tested for it in `capabilities` alone, which made every real 0.16 server read as older than 0.16 — and that one check drove three things: self-service credentials fell back to `POST /api/account/auth`, which 0.16 removed, so password changes, 2FA and app passwords all failed with "this mail server does not offer self-service credential management"; About reported the wrong generation; and Files took the older code path. It now looks in all three places, and is covered by tests on each. Worth restating plainly, because the stakes went up when 0.15 support was dropped: there is no longer a fallback path for this check to be wrong *into*. Getting it wrong now refuses every sign-in against a perfectly good server — a loud failure rather than a quiet misrouting, which is the trade the removal was making. - **HTML signatures** — Stalwart caps a signature at 2047 **bytes** (`value.len() < 2048` on a Rust string, so UTF-8 bytes, not characters). ihasmail compacts pasted HTML, moves images to Files and, if still too large, keeps the full signature in Files behind a short marker; other clients see a text fallback. Confirmed live on 0.15.5 (2026-08-24): oversized, non-ASCII and inline-image signatures all save, and a test message arrived intact at Gmail with the logo inline. -- **Settings live in the account's Files, not the browser** — every preference used to sit in `localStorage`, so none of them followed anyone between devices. The sharpest edge was the default identity: with none set the address that sorts first wins, so someone who set it at work found it unset at home and mail went out from an address the recipient might not recognise ([#54](https://github.com/LINUXexpert-org/ihasmail/issues/54)). They are now a `settings.json` in the `ihasmail` folder in JMAP Files, beside the signature images already kept there — which keeps ihasmail itself stateless: no volume, no database, nothing to back up separately, and the settings are covered by whatever backs up the mail store. `x:AccountSettings` was the other candidate and does not fit; its schema is `locale`/`timeZone`/`description` with no free-form field, and writing it needs `sysAccountSettingsSet`, where the built-in user role carries only the `…Get` half. `localStorage` stays on as a *cache* rather than the source of truth, so the first frame paints from it and the file corrects it a moment later; a browser with no cache shows defaults for that one frame, which is the trade for not gating the whole app on a round trip. Settings that describe *this* screen or browser deliberately stay local — list-pane sizes, density, font size, sidebar state, and the notification toggles, which track a permission the browser grants per-device and would be a claim about somewhere else it cannot make. That split is written as a list of exceptions, so a setting added later syncs by default. Writes are coalesced behind a three-second debounce, since `update()` fires on every frame of a splitter drag, and a tab going away or a sign-out flushes first. The `ihasmail` folder is now hidden from the Files view, contents and all: hiding the folder alone would be worse than showing it, because the tree attaches a node whose parent is missing to the root, so the signature images — visible there since signatures shipped — would have spilled into the top level. **Confirmed live on 0.16.19 (2026-08-26)**: settings set in Chrome came back on a fresh login in Firefox and in an incognito session, both of which start with an empty cache, so each read the account's file rather than anything local. Requires 0.16, since `FileNode/query` cannot see directories before that; on 0.15 settings stay local exactly as they were. Two limits worth knowing: conflicts are last-write-wins, and a change made on one device does not reach another that already has ihasmail open until it signs in again. -- **Files on Stalwart before 0.16** — three things differ there, none of which the server reports as an error. (Confirmed live on 0.15.5 before the upgrade. The live instance now runs 0.16.19, where folder creation, upload, rename, move and delete were also exercised — but under the capability-placement bug below, which means what ran there was this older path against a 0.16 server, not the 0.16 path. Files now takes the 0.16 path and wants checking again on its own terms. The older path is kept for anyone still on 0.15.x and covered by `npm run dev:mock:legacy`.) `FileNode/query` masks its results to non-containers, so it returns files and **never folders**; `nodeType` does not exist, and sending it fails the create outright (a directory is instead a node with no file properties at all); and rights are only `mayRead`/`mayWrite`/`mayShare`, so the finer-grained `mayDelete`/`mayRename` the UI gates on are absent. ihasmail detects the older server by the absence of `urn:stalwart:jmap` — looked for in `primaryAccounts` and `accountCapabilities` as well as the session capabilities, since that is where 0.16 actually advertises it — lists the tree through `FileNode/get` instead of query, shapes creates accordingly, and widens the old rights. Upload, folder creation, listing, rename, move and delete are all confirmed live on 0.15.5 (2026-08-24). Part of the 0.16 path has since been exercised live as a side effect of the settings file above, which on 0.16.19 (2026-08-26) found and created the folder, created a node with `nodeType`, uploaded and downloaded its blob, and pointed an existing node at a new one. That leaves rename, move, delete and the Files view itself still to be checked on the 0.16 path on their own terms. -- **Self-service credentials** — the **0.15.x REST path was confirmed live** against Stalwart 0.15.5 (2026-08-24): password change, app passwords, and enabling and disabling 2FA, on a real mailbox. The **0.16 registry path is confirmed live** against Stalwart 0.16.19 (2026-08-25): app passwords created and revoked, password changed, 2FA enabled and disabled, with the browser session surviving the switch to an app password. The mock enforces the same rules either way (current password required, password policy, a TOTP code on every request once 2FA is on, app passwords exempt from it). Password changes are refused by Stalwart for accounts backed by an external directory (LDAP/SQL/OIDC); the server's own message is shown when that happens. +- **Settings live in the account's Files, not the browser** — every preference used to sit in `localStorage`, so none of them followed anyone between devices. The sharpest edge was the default identity: with none set the address that sorts first wins, so someone who set it at work found it unset at home and mail went out from an address the recipient might not recognise ([#54](https://github.com/LINUXexpert-org/ihasmail/issues/54)). They are now a `settings.json` in the `ihasmail` folder in JMAP Files, beside the signature images already kept there — which keeps ihasmail itself stateless: no volume, no database, nothing to back up separately, and the settings are covered by whatever backs up the mail store. `x:AccountSettings` was the other candidate and does not fit; its schema is `locale`/`timeZone`/`description` with no free-form field, and writing it needs `sysAccountSettingsSet`, where the built-in user role carries only the `…Get` half. `localStorage` stays on as a *cache* rather than the source of truth, so the first frame paints from it and the file corrects it a moment later; a browser with no cache shows defaults for that one frame, which is the trade for not gating the whole app on a round trip. Settings that describe *this* screen or browser deliberately stay local — list-pane sizes, density, font size, sidebar state, and the notification toggles, which track a permission the browser grants per-device and would be a claim about somewhere else it cannot make. That split is written as a list of exceptions, so a setting added later syncs by default. Writes are coalesced behind a three-second debounce, since `update()` fires on every frame of a splitter drag, and a tab going away or a sign-out flushes first. The `ihasmail` folder is now hidden from the Files view, contents and all: hiding the folder alone would be worse than showing it, because the tree attaches a node whose parent is missing to the root, so the signature images — visible there since signatures shipped — would have spilled into the top level. **Confirmed live on 0.16.19 (2026-08-26)**: settings set in Chrome came back on a fresh login in Firefox and in an incognito session, both of which start with an empty cache, so each read the account's file rather than anything local. Requires 0.16, which ihasmail now requires everywhere — `FileNode/query` cannot see directories before that, and sign-in refuses an older server outright. Two limits worth knowing: conflicts are last-write-wins, and a change made on one device does not reach another that already has ihasmail open until it signs in again. +- **Files on 0.16** — the pre-0.16 quirks this entry used to describe are gone with the support for them: `FileNode/query` masking directories out of its own results, `nodeType` not existing, and rights being a single `mayWrite`. What is left is what has actually been exercised on 0.16.19. Finding and creating a folder, creating a node with `nodeType`, uploading and downloading its blob, and pointing an existing node at a new one all ran live on 2026-08-26, as a side effect of the settings file. **Rename, move, delete and the Files view itself are still unchecked on 0.16** — what was confirmed live on 0.15.5 (2026-08-24) was the older code path, which no longer exists. Two fallbacks went with the removal and are worth knowing about: `ensureFolder` and `findInFolder` now filter on `parentId`/`isTopLevel` alone and match names client-side, since `name` is not a filter Stalwart is known to implement and one it does not know fails the whole query; and a refused filter or sort no longer drops the view into fetching every node in the account, which would have hidden a real fault behind a performance cliff nobody would notice. +- **Self-service credentials** — the registry path is **confirmed live** against Stalwart 0.16.19 (2026-08-25): app passwords created and revoked, password changed, 2FA enabled and disabled, with the browser session surviving the switch to an app password. The 0.15 REST path was confirmed live too, on 0.15.5 (2026-08-24), and has since been removed along with the rest of 0.15 support. The mock enforces the same rules the real server does (current password required, password policy, a TOTP code on every request once 2FA is on, app passwords exempt from it). Password changes are refused by Stalwart for accounts backed by an external directory (LDAP/SQL/OIDC); the server's own message is shown when that happens. - **Scheduled send needs one setting turned on, and says nothing when it is off.** Stalwart advertises the delay in the account's `urn:ietf:params:jmap:submission` capability — `maxDelayedSend: 2592000` (30 days) and `FUTURERELEASE` among its `submissionExtensions`, and note it is the *account* capability, not the session-level one, which is empty. But the MTA only honours a hold when `futureRelease` is set under the session's MTA extensions, and [that setting defaults to `false`](https://stalw.art/docs/ref/object/mta-extensions/). With it off, Stalwart takes the `HOLDUNTIL` parameter, skips the hold and sends the message immediately **without an error** — the capability still says thirty days. So set `futureRelease` (to the longest hold you want to allow) before relying on this; a value shorter than 30 days is fine, and a request past it is refused honestly, with a `forbiddenMailFrom` naming the limit. `npm run dev:mock:no-future-release` reproduces the silent-drop case. ihasmail asks for the delay the way JMAP requires — a `HOLDUNTIL` parameter on the envelope's `mailFrom`, since RFC 8621 makes `sendAt` read-only and server-derived — and files the held message in a **Scheduled** folder, because `onSuccessUpdateEmail` would otherwise drop it in Sent the moment the submission is created. Nothing moves it out when the hold expires, so ihasmail reconciles the folder on the way in: released messages to Sent, cancelled ones back to Drafts. Three fixes this depends on landed in **0.16.17**, below the live instance's 0.16.19: `HOLDUNTIL` taking RFC 3339 date-times again (0.16.16 had it wanting Unix timestamps), `EmailSubmission/query` on `undoStatus` agreeing with `/get` about held submissions, and `EmailSubmission/get` without `ids` iterating the right index. The hold itself is now **confirmed against the live 0.16.19** (2026-08-25), once `futureRelease` was set to `30d` there: a submission carrying a `HOLDUNTIL` ten minutes out came back `pending`, with `sendAt` equal to the time asked for and a `250 2.1.5 Queued` from the MTA, rather than going out at once. Worth repeating that the capability is no evidence either way — it advertised `maxDelayedSend: 2592000` and `FUTURERELEASE` while the setting was still off. Only a submission tells you. What is still mock-only is the rest of the journey: the **Scheduled** folder reconciling on the way in, and a hold actually expiring and being delivered. - **Stalwart 0.16 and RFC 8984 disagree about the calendar vocabulary, and the server only says so half the time.** A participant's address lives in `calendarAddress`, not RFC 8984's `sendTo`/`email`; the organizer is `organizerCalendarAddress`, not `replyTo`; and a recurrence is a single `recurrenceRule`, not a `recurrenceRules` array. Addressed the RFC's way, `CalendarEvent/set` **keeps the event and discards the whole participant map without an error** — guests disappeared on save and no invitation was ever sent, which is what [#26](https://github.com/LINUXexpert-org/ihasmail/issues/26) reported. The array form of the rule is refused honestly, with `invalidProperties`, so recurring events could not be created at all and existing ones showed no repeat ([#30](https://github.com/LINUXexpert-org/ihasmail/issues/30)). ihasmail now writes Stalwart's names and reads either, and the mock refuses what the real server refuses, since advertising the RFC spelling is precisely how this got as far as a live server. Verified against 0.16.19 on 2026-08-25, end to end: participants, organizer and rule all survive a create, an update and a re-read; an invitation to an external Gmail address arrived as an invite card, and the decline came back and was applied to the event (`needs-action` → `declined`, sequence 1). Cancelling the event notified the guest too. Adding guests to an event that had none, and clearing them again with `null`, both work on the update path, as does RSVP — which patches `participants/{key}/participationStatus` (and `participationComment`) rather than sending the whole map. That patch has to be aimed at the base event: `CalendarEvent/set` refuses a synthetic id with *"Updating synthetic ids is not yet supported"*, which is why RSVP resolves `baseEventId` first. Adding a *new* participant by patch is refused as well (`Patch operation failed`), so a changed guest list is written as the whole `participants` property. One more thing to know when reading this code: an expanded occurrence carries a `recurrenceId` but *no* rule of its own, and `baseEventId` is set on everything an expanded query returns — a one-off included, whose own id differs from its base — so neither is a test for recurrence. - Recurring events: colour/category/edit/delete apply to the whole series (per-occurrence overrides aren't supported by the server yet). diff --git a/package.json b/package.json index cd41ff4..f1f2cfb 100644 --- a/package.json +++ b/package.json @@ -21,7 +21,6 @@ "lint": "npm run typecheck", "mock": "npm run mock -w server", "dev:mock": "concurrently -n mock,server,web -c yellow,blue,magenta \"npm run mock -w server\" \"STALWART_URL=http://127.0.0.1:8788 npm run dev -w server\" \"npm run dev -w web\"", - "dev:mock:legacy": "concurrently -n mock,server,web -c yellow,blue,magenta \"npm run mock:legacy -w server\" \"STALWART_URL=http://127.0.0.1:8788 npm run dev -w server\" \"npm run dev -w web\"", "dev:mock:no-future-release": "concurrently -n mock,server,web -c yellow,blue,magenta \"npm run mock:no-future-release -w server\" \"STALWART_URL=http://127.0.0.1:8788 npm run dev -w server\" \"npm run dev -w web\"" }, "devDependencies": { diff --git a/server/package.json b/server/package.json index c988ceb..30a4b37 100644 --- a/server/package.json +++ b/server/package.json @@ -12,7 +12,6 @@ "typecheck": "tsc -p tsconfig.json --noEmit", "test": "tsx --test src/*.test.ts src/**/*.test.ts", "mock": "tsx src/mock/index.ts", - "mock:legacy": "MOCK_STALWART=0.15 tsx src/mock/index.ts", "mock:no-future-release": "MOCK_NO_FUTURE_RELEASE=1 tsx src/mock/index.ts" }, "dependencies": { diff --git a/server/src/account-legacy.test.ts b/server/src/account-legacy.test.ts deleted file mode 100644 index 4f90dea..0000000 --- a/server/src/account-legacy.test.ts +++ /dev/null @@ -1,183 +0,0 @@ -import { test, before, after } from "node:test"; -import assert from "node:assert/strict"; - -/** - * The same self-service flows, against a mock impersonating Stalwart 0.15. - * - * That generation has no registry: credentials live behind a REST endpoint, - * `urn:stalwart:jmap` is not a capability it knows, and naming one it cannot - * parse fails the whole request. Until now this adapter had no coverage at all - * — it was the least-tested code in the project, verified only by hand. - */ - -const PORT = 18799; -process.env.MOCK_PORT = String(PORT); -process.env.MOCK_STALWART = "0.15"; -process.env.MOCK_USER = "demo@example.com"; -process.env.MOCK_PASS = "demo-password"; -process.env.STALWART_URL = `http://127.0.0.1:${PORT}`; -process.env.APP_SECRET = "test-secret-for-legacy-flows"; - -const mock = await import("./mock/index.js"); -const { createApp } = await import("./app.js"); - -const app = createApp(); -let cookie = ""; -const HEADERS = { "content-type": "application/json", "x-requested-with": "ihasmail" }; - -async function call(path: string, init: RequestInit = {}): Promise<{ status: number; body: any }> { - const res = await app.request(path, { - ...init, - headers: { ...HEADERS, ...(init.headers as Record), ...(cookie ? { cookie } : {}) }, - }); - const setCookie = res.headers.get("set-cookie"); - if (setCookie) cookie = setCookie.split(";")[0]!; - const text = await res.text(); - return { status: res.status, body: text ? JSON.parse(text) : null }; -} - -const post = (path: string, body: unknown) => call(path, { method: "POST", body: JSON.stringify(body) }); - -before(async () => { - const res = await post("/api/auth/login", { username: "demo@example.com", password: "demo-password" }); - assert.equal(res.status, 200, "login should succeed against the legacy mock"); -}); - -after(() => { - (mock as { server?: { close(): void } }).server?.close(); -}); - -test("the older server is recognised, and reported as such", async () => { - const res = await call("/api/auth/session"); - assert.equal(res.status, 200); - assert.equal(res.body.ihasmail.server.generation, "pre-0.16"); - assert.equal(res.body.ihasmail.server.edition, null, "no edition is reported before 0.16"); - assert.equal(res.body.capabilities["urn:stalwart:jmap"], undefined, "the capability does not exist here"); -}); - -test("credentials fall back to the REST endpoint", async () => { - const res = await call("/api/account/security"); - assert.equal(res.status, 200); - assert.equal(res.body.backend, "legacy"); - assert.equal(res.body.otpEnabled, false); - assert.equal(res.body.appPasswordsKeyedByName, true, "this generation has only names to go on"); -}); - -test("app passwords round-trip, keyed by their name", async () => { - const created = await post("/api/account/app-passwords", { description: "Thunderbird" }); - assert.equal(created.status, 200); - assert.ok(created.body.secret, "a secret is generated for the user to copy"); - assert.equal(created.body.id, "Thunderbird", "the name is the identifier here"); - - const listed = await call("/api/account/security"); - assert.deepEqual(listed.body.appPasswords.map((a: { description: string }) => a.description), ["Thunderbird"]); - - await post("/api/account/app-passwords/revoke", { id: "Thunderbird" }); - assert.deepEqual((await call("/api/account/security")).body.appPasswords, []); -}); - -test("the current password is verified before it is changed", async () => { - // The REST endpoint would take our word for it, so ihasmail proves it first. - const wrong = await post("/api/account/password", { current: "not-my-password", next: "a-much-longer-password" }); - assert.equal(wrong.status, 403); - assert.match(wrong.body.message, /incorrect/i); - assert.equal((mock as { account: { password: string } }).account.password, "demo-password", "nothing was changed"); -}); - -test("changing the password keeps this session working", async () => { - const res = await post("/api/account/password", { current: "demo-password", next: "a-brand-new-password" }); - assert.equal(res.status, 200); - assert.equal((mock as { account: { password: string } }).account.password, "a-brand-new-password"); - assert.equal((await call("/api/auth/session")).status, 200, "the session was re-sealed"); -}); - -test("2FA is enabled with a code proved against the new secret", async () => { - const { parseOtpauthUrl, totpCode } = await import("./totp.js"); - const begin = await post("/api/account/2fa/begin", {}); - const params = parseOtpauthUrl(begin.body.url); - assert.ok(params); - - const bad = await post("/api/account/2fa/enable", { url: begin.body.url, code: "000000", current: "a-brand-new-password" }); - assert.equal(bad.status, 400); - assert.equal((mock as { account: { otpUrl: string | null } }).account.otpUrl, null, "nothing was stored"); - - const good = await post("/api/account/2fa/enable", { url: begin.body.url, code: totpCode(params), current: "a-brand-new-password" }); - assert.equal(good.status, 200); - assert.equal(good.body.sessionKept, true, "the session moved onto an app password"); - assert.equal((await call("/api/account/security")).body.otpEnabled, true); -}); - -test("2FA is switched off again", async () => { - const { parseOtpauthUrl, totpCode } = await import("./totp.js"); - const stored = (mock as { account: { otpUrl: string | null } }).account.otpUrl; - const params = parseOtpauthUrl(stored!); - assert.ok(params); - const res = await post("/api/account/2fa/disable", { current: "a-brand-new-password", code: totpCode(params) }); - assert.equal(res.status, 200); - assert.equal((await call("/api/account/security")).body.otpEnabled, false); -}); - -/** - * The mock is only worth having if it is faithful, so these pin the specific - * behaviours that cost us a live debugging session each. Every one of them was - * invisible to the 0.16 mock, which is how the bugs shipped. - */ - -const jmap = (using: string[], methodCalls: unknown[]) => post("/api/jmap", { using, methodCalls }); -const CORE = "urn:ietf:params:jmap:core"; -const MAIL = "urn:ietf:params:jmap:mail"; -const FILES = "urn:ietf:params:jmap:filenode"; - -test("naming a capability it cannot parse fails the whole request", async () => { - const res = await jmap([CORE, "urn:stalwart:jmap"], [["Mailbox/get", { accountId: "a1", ids: null }, "c0"]]); - assert.notEqual(res.status, 200, "not one failed call - the entire request"); -}); - -test("x: methods do not exist, so they come back unknownMethod", async () => { - const res = await jmap([CORE], [["x:AccountPassword/get", { accountId: "a1", ids: ["singleton"] }, "c0"]]); - assert.equal(res.status, 200); - assert.equal(res.body.methodResponses[0][0], "error"); - assert.equal(res.body.methodResponses[0][1].type, "unknownMethod"); -}); - -test("FileNode/set refuses nodeType by name", async () => { - const res = await jmap([CORE, FILES], [["FileNode/set", { accountId: "a1", create: { d: { parentId: null, name: "New", nodeType: "directory" } } }, "c0"]]); - const set = res.body.methodResponses[0][1]; - assert.equal(set.notCreated.d.type, "invalidProperties"); - assert.deepEqual(set.notCreated.d.properties, ["nodeType"]); -}); - -test("a directory is a node with no file properties, and query cannot see it", async () => { - const made = await jmap([CORE, FILES], [["FileNode/set", { accountId: "a1", create: { d: { parentId: null, name: "Reports" } } }, "c0"]]); - const id = made.body.methodResponses[0][1].created.d.id; - assert.ok(id); - - const queried = await jmap([CORE, FILES], [["FileNode/query", { accountId: "a1" }, "c0"]]); - assert.equal(queried.body.methodResponses[0][1].ids.includes(id), false, "query masks out containers"); - - // get carries no such mask, which is the only way to find a folder here. - const got = await jmap([CORE, FILES], [["FileNode/get", { accountId: "a1", ids: null }, "c0"]]); - const list = got.body.methodResponses[0][1].list as { id: string; nodeType?: string; myRights: Record }[]; - const dir = list.find((n) => n.id === id); - assert.ok(dir, "get returns the directory"); - assert.equal(dir!.nodeType, undefined, "nodeType is not a property here"); - assert.deepEqual(Object.keys(dir!.myRights).sort(), ["mayRead", "mayShare", "mayWrite"], "the coarser rights"); -}); - -test("FileNode/query refuses the filters and sorts this generation lacks", async () => { - const filtered = await jmap([CORE, FILES], [["FileNode/query", { accountId: "a1", filter: { isTopLevel: true } }, "c0"]]); - assert.equal(filtered.body.methodResponses[0][1].type, "unsupportedFilter"); - const sorted = await jmap([CORE, FILES], [["FileNode/query", { accountId: "a1", sort: [{ property: "nodeType" }] }, "c0"]]); - assert.equal(sorted.body.methodResponses[0][1].type, "unsupportedSort"); -}); - -test("an identity signature is capped in bytes, not characters", async () => { - // 1200 CJK characters: comfortably under 2047 counted as characters, and - // 3600 bytes once encoded. - const tooBig = "日".repeat(1200); - assert.ok(tooBig.length < 2047 && Buffer.byteLength(tooBig, "utf8") > 2047); - const res = await jmap([CORE, MAIL], [["Identity/set", { accountId: "a1", update: { i1: { htmlSignature: tooBig } } }, "c0"]]); - const set = res.body.methodResponses[0][1]; - assert.equal(set.notUpdated.i1.type, "invalidProperties"); - assert.deepEqual(set.notUpdated.i1.properties, ["htmlSignature"]); -}); diff --git a/server/src/account.test.ts b/server/src/account.test.ts index bb56db5..c5dbd84 100644 --- a/server/src/account.test.ts +++ b/server/src/account.test.ts @@ -47,27 +47,25 @@ after(() => { }); /** - * What the About page reads. Stalwart advertises `urn:stalwart:jmap` only - * per-account, so a session that looks for it at the top level reports a real - * 0.16 server as older than 0.16 — the same mistake that sent credentials to - * the removed REST endpoint. + * Stalwart advertises `urn:stalwart:jmap` only per-account, never in the + * session-level capabilities. Looking for it at the top level alone reported + * every real 0.16 server as older than 0.16 — and now that the same check + * decides whether a sign-in is allowed at all, that mistake would lock + * everyone out rather than merely misroute credentials. */ -test("the session reports the 0.16 generation the server actually is", async () => { +test("the session is accepted on a server that advertises the registry per-account", async () => { const res = await call("/api/auth/session"); assert.equal(res.status, 200); - assert.equal(res.body.ihasmail.server.generation, "0.16+"); assert.equal(res.body.ihasmail.server.edition, "oss"); assert.equal(res.body.capabilities["urn:stalwart:jmap"], undefined, "not where a client would first look"); assert.ok("urn:stalwart:jmap" in res.body.primaryAccounts, "but here, as on a real server"); }); -test("the 0.16 registry backend is detected and reported empty", async () => { +test("the registry reports an account with nothing set up yet", async () => { const res = await call("/api/account/security"); assert.equal(res.status, 200); - assert.equal(res.body.backend, "registry"); assert.equal(res.body.otpEnabled, false); assert.deepEqual(res.body.appPasswords, []); - assert.equal(res.body.appPasswordsKeyedByName, false); }); test("app passwords are created, listed once with their secret, and revoked", async () => { diff --git a/server/src/account.ts b/server/src/account.ts index ce412b2..6969ad8 100644 --- a/server/src/account.ts +++ b/server/src/account.ts @@ -1,17 +1,15 @@ import { config } from "./config.js"; -import { absoluteUpstream, hasStalwartRegistry, UpstreamError, type UpstreamSession } from "./upstream.js"; +import { absoluteUpstream, UpstreamError, type UpstreamSession } from "./upstream.js"; import { generateSecret, otpauthUrl, parseOtpauthUrl, verifyTotp } from "./totp.js"; -import { randomBytes } from "node:crypto"; /** - * Self-service credential management, across two incompatible Stalwart APIs. + * Self-service credential management, over Stalwart's JMAP registry: + * `x:AccountPassword` (a singleton holding the password and the otpauth URL) + * and `x:AppPassword`. * - * 0.16+ JMAP registry objects: x:AccountPassword (a singleton holding the - * password and the otpauth URL) and x:AppPassword. - * 0.15.x a REST endpoint, POST /api/account/auth, taking a list of actions. - * - * The registry crate does not exist before 0.16 and the REST endpoint is gone - * after it, so which one answers is the only reliable way to tell them apart. + * The registry crate arrived in 0.16, which is the oldest Stalwart ihasmail + * supports. Sign-in refuses anything older, so by the time any of this runs + * the registry is known to be there. */ const STALWART_CAP = "urn:stalwart:jmap"; @@ -21,10 +19,7 @@ const SINGLETON = "singleton"; /** Returned in place of a stored secret; echo it back to leave one unchanged. */ const MASKED = "[********]"; -export type Backend = "registry" | "legacy"; - export interface AppPasswordRow { - /** Registry object id, or the name itself on legacy servers. */ id: string; description: string; createdAt: string | null; @@ -32,14 +27,8 @@ export interface AppPasswordRow { } export interface SecurityState { - backend: Backend; otpEnabled: boolean; appPasswords: AppPasswordRow[]; - /** - * Legacy servers key app passwords by name and hand back nothing else, so - * the UI must keep names unique and cannot show when one was created. - */ - appPasswordsKeyedByName: boolean; } /** An error with a message meant for the person using the app. */ @@ -61,49 +50,7 @@ interface Ctx { } /* ------------------------------------------------------------------ */ -/* Backend detection */ -/* ------------------------------------------------------------------ */ - -const backendCache = new Map(); -const BACKEND_CACHE_MS = 30 * 60_000; - -export function forgetBackend(sessionId: string): void { - backendCache.delete(sessionId); -} - -export async function detectBackend(sessionId: string, ctx: Ctx): Promise { - const cached = backendCache.get(sessionId); - if (cached && Date.now() - cached.at < BACKEND_CACHE_MS) return cached.backend; - const backend = await probeBackend(ctx); - backendCache.set(sessionId, { backend, at: Date.now() }); - return backend; -} - -async function probeBackend(ctx: Ctx): Promise { - // A server with the registry answers x:AccountPassword/get; one without it - // fails to parse the method name at all and returns unknownMethod. - if (hasStalwartRegistry(ctx.session)) { - try { - const res = await jmap(ctx, [["x:AccountPassword/get", { accountId: accountId(ctx), ids: [SINGLETON] }, "p"]]); - const [name, args] = res.methodResponses?.[0] ?? []; - if (name && name !== "error") return "registry"; - const type = (args as { type?: string } | undefined)?.type; - if (type && type !== "unknownMethod") return "registry"; // present, but refused us - } catch { - // The capability already told us this server has the registry, so a - // request we could not read is a fault to surface, not evidence of an - // older server. Falling back here would post the user's password to a - // REST endpoint 0.16 removed and report the feature as unsupported. - return "registry"; - } - // It named the capability and then disowned the method: nothing else to try. - return "registry"; - } - return "legacy"; -} - -/* ------------------------------------------------------------------ */ -/* Transports */ +/* Transport */ /* ------------------------------------------------------------------ */ function accountId(ctx: Ctx): string { @@ -129,29 +76,6 @@ async function jmap(ctx: Ctx, methodCalls: Invocation[]): Promise<{ methodRespon return (await res.json()) as { methodResponses?: [string, unknown, string][] }; } -async function legacy(ctx: Ctx, init: RequestInit): Promise { - const res = await fetch(`${config.stalwartUrl}/api/account/auth`, { - ...init, - headers: { authorization: ctx.authorization, "content-type": "application/json", accept: "application/json" }, - signal: AbortSignal.timeout(config.upstreamTimeout), - }); - if (res.status === 401 || res.status === 403) throw new UpstreamError("Invalid credentials", 401); - if (res.status === 404) { - throw new AccountError("This mail server does not offer self-service credential management.", 501, "unsupported"); - } - if (!res.ok) { - let detail = ""; - try { - const body = (await res.json()) as { error?: string; details?: string; reason?: string }; - detail = body.details ?? body.reason ?? body.error ?? ""; - } catch { - /* fall through to the generic message */ - } - throw new AccountError(detail || `The mail server rejected the change (${res.status}).`, 502, "upstream"); - } - return ((await res.json()) as { data: T }).data; -} - /** * Pull the single result out of a /set, turning JMAP's several failure shapes * into one error carrying whatever the server was willing to explain. @@ -192,17 +116,7 @@ function describeSetError(err: { type?: string; description?: string; properties /* Operations */ /* ------------------------------------------------------------------ */ -export async function getState(sessionId: string, ctx: Ctx): Promise { - const backend = await detectBackend(sessionId, ctx); - if (backend === "legacy") { - const data = await legacy<{ otpEnabled?: boolean; appPasswords?: string[] }>(ctx, { method: "GET" }); - return { - backend, - otpEnabled: Boolean(data.otpEnabled), - appPasswords: (data.appPasswords ?? []).map((name) => ({ id: name, description: name, createdAt: null, expiresAt: null })), - appPasswordsKeyedByName: true, - }; - } +export async function getState(ctx: Ctx): Promise { const id = accountId(ctx); const res = await jmap(ctx, [ ["x:AccountPassword/get", { accountId: id, ids: [SINGLETON] }, "p"], @@ -211,7 +125,6 @@ export async function getState(sessionId: string, ctx: Ctx): Promise ({ @@ -220,7 +133,6 @@ export async function getState(sessionId: string, ctx: Ctx): Promise { - const backend = await detectBackend(sessionId, ctx); - if (backend === "registry") { - const update: Record = { currentSecret: opts.current, secret: opts.next }; - if (opts.otpCode) update["otpAuth/otpCode"] = opts.otpCode; - const res = await jmap(ctx, [["x:AccountPassword/set", { accountId: accountId(ctx), update: { [SINGLETON]: update } }, "s"]]); - setResult(res, "updated"); - return; - } - // The legacy endpoint changes the password without asking for the old one, - // so anyone holding a live session could set it. Prove it ourselves first. - await assertCurrentPassword(ctx, opts.current, opts.otpCode); - await legacy(ctx, { method: "POST", body: JSON.stringify([{ type: "setPassword", password: opts.next }]) }); +export async function changePassword(ctx: Ctx, opts: { current: string; next: string; otpCode?: string }): Promise { + const update: Record = { currentSecret: opts.current, secret: opts.next }; + if (opts.otpCode) update["otpAuth/otpCode"] = opts.otpCode; + const res = await jmap(ctx, [["x:AccountPassword/set", { accountId: accountId(ctx), update: { [SINGLETON]: update } }, "s"]]); + setResult(res, "updated"); } -export async function createAppPassword( - sessionId: string, - ctx: Ctx, - opts: { description: string }, -): Promise<{ id: string; secret: string }> { - const backend = await detectBackend(sessionId, ctx); +export async function createAppPassword(ctx: Ctx, opts: { description: string }): Promise<{ id: string; secret: string }> { const description = opts.description.trim() || "App password"; - if (backend === "registry") { - const res = await jmap(ctx, [["x:AppPassword/set", { accountId: accountId(ctx), create: { n: { description } } }, "s"]]); - const created = setResult(res, "created"); - const secret = created && typeof created.secret === "string" ? created.secret : ""; - if (!secret) throw new AccountError("The mail server created the app password but did not return it.", 502, "upstream"); - return { id: String(created?.id ?? description), secret }; - } - // Legacy servers take a secret of our choosing and key it by name. - const secret = readableSecret(); - await legacy(ctx, { - method: "POST", - body: JSON.stringify([{ type: "addAppPassword", name: description, password: secret }]), - }); - return { id: description, secret }; + const res = await jmap(ctx, [["x:AppPassword/set", { accountId: accountId(ctx), create: { n: { description } } }, "s"]]); + const created = setResult(res, "created"); + const secret = created && typeof created.secret === "string" ? created.secret : ""; + if (!secret) throw new AccountError("The mail server created the app password but did not return it.", 502, "upstream"); + return { id: String(created?.id ?? description), secret }; } -export async function revokeAppPassword(sessionId: string, ctx: Ctx, id: string): Promise { - const backend = await detectBackend(sessionId, ctx); - if (backend === "registry") { - const res = await jmap(ctx, [["x:AppPassword/set", { accountId: accountId(ctx), destroy: [id] }, "s"]]); - setResult(res, "destroyed"); - return; - } - await legacy(ctx, { method: "POST", body: JSON.stringify([{ type: "removeAppPassword", name: id }]) }); +export async function revokeAppPassword(ctx: Ctx, id: string): Promise { + const res = await jmap(ctx, [["x:AppPassword/set", { accountId: accountId(ctx), destroy: [id] }, "s"]]); + setResult(res, "destroyed"); } /** @@ -311,89 +192,30 @@ export function assertEnrolmentCode(url: string, code: string): void { } } -export async function enableOtp( - sessionId: string, - ctx: Ctx, - opts: { url: string; code: string; current: string }, -): Promise { +export async function enableOtp(ctx: Ctx, opts: { url: string; code: string; current: string }): Promise { assertEnrolmentCode(opts.url, opts.code); - const backend = await detectBackend(sessionId, ctx); - if (backend === "registry") { - const res = await jmap(ctx, [ - [ - "x:AccountPassword/set", - { accountId: accountId(ctx), update: { [SINGLETON]: { currentSecret: opts.current, "otpAuth/otpUrl": opts.url } } }, - "s", - ], - ]); - setResult(res, "updated"); - return; - } - await assertCurrentPassword(ctx, opts.current); - await legacy(ctx, { method: "POST", body: JSON.stringify([{ type: "enableOtpAuth", url: opts.url }]) }); + const res = await jmap(ctx, [ + [ + "x:AccountPassword/set", + { accountId: accountId(ctx), update: { [SINGLETON]: { currentSecret: opts.current, "otpAuth/otpUrl": opts.url } } }, + "s", + ], + ]); + setResult(res, "updated"); } -export async function disableOtp( - sessionId: string, - ctx: Ctx, - opts: { current: string; code: string }, -): Promise { - const backend = await detectBackend(sessionId, ctx); - if (backend === "registry") { - const res = await jmap(ctx, [ - [ - "x:AccountPassword/set", - { - accountId: accountId(ctx), - update: { [SINGLETON]: { currentSecret: opts.current, "otpAuth/otpCode": opts.code, "otpAuth/otpUrl": null } }, - }, - "s", - ], - ]); - setResult(res, "updated"); - return; - } - await assertCurrentPassword(ctx, opts.current, opts.code); - await legacy(ctx, { method: "POST", body: JSON.stringify([{ type: "disableOtpAuth", url: null }]) }); -} - -/** - * Confirm a password by authenticating with it, for the legacy endpoint that - * would otherwise take our word for it. - */ -async function assertCurrentPassword(ctx: Ctx, current: string, otpCode?: string): Promise { - const secret = otpCode ? `${current}$${otpCode}` : current; - const authorization = `Basic ${Buffer.from(`${ctx.username}:${secret}`, "utf8").toString("base64")}`; - const res = await fetch(`${config.stalwartUrl}/.well-known/jmap`, { - headers: { authorization, accept: "application/json" }, - redirect: "follow", - signal: AbortSignal.timeout(config.upstreamTimeout), - }); - if (res.status === 401 || res.status === 403) { - throw new AccountError("That password is incorrect.", 403, "bad_password"); - } - if (!res.ok) throw new UpstreamError(`Could not verify the current password (${res.status})`, 502); -} - -/** - * A legacy app password a person can read off a screen and type. - * - * Drawn by rejection sampling. Plain `% alphabet.length` would favour the - * first 25 characters, because 256 is not a multiple of 33: each of those - * would come up on 8 byte values and the remaining 8 on only 7. - */ -export function readableSecret(): string { - const alphabet = "abcdefghijkmnopqrstuvwxyz23456789"; // no l/1/0 lookalikes - const limit = 256 - (256 % alphabet.length); - const chars: string[] = []; - while (chars.length < 20) { - for (const b of randomBytes(32)) { - if (b >= limit) continue; // the tail that would skew the alphabet - chars.push(alphabet[b % alphabet.length]!); - if (chars.length === 20) break; - } - } - return (chars.join("").match(/.{5}/g) ?? []).join("-"); +export async function disableOtp(ctx: Ctx, opts: { current: string; code: string }): Promise { + const res = await jmap(ctx, [ + [ + "x:AccountPassword/set", + { + accountId: accountId(ctx), + update: { [SINGLETON]: { currentSecret: opts.current, "otpAuth/otpCode": opts.code, "otpAuth/otpUrl": null } }, + }, + "s", + ], + ]); + setResult(res, "updated"); } export { MASKED }; diff --git a/server/src/accountinfo.test.ts b/server/src/accountinfo.test.ts index 07dc4d9..11d23ea 100644 --- a/server/src/accountinfo.test.ts +++ b/server/src/accountinfo.test.ts @@ -7,7 +7,8 @@ import { getAccountInfo, hasStalwartRegistry, interpretAccountInfo } from "./ups * the `sysAccountGet` permission — one the built-in `user` role is not given. * Ordinary users therefore silently fell back to the browser locale. Stalwart * 0.16 exposes the same field on `x:AccountSettings`, which users *can* read, - * so both are asked for and whichever answers wins. + * so both are asked for and whichever answers wins. Both are 0.16 methods: + * this is a permissions fallback, not a version one. */ type Responses = [string, Record, string][]; @@ -19,7 +20,6 @@ const failed = (id: string, type: string): Responses[number] => ["error", { type test("prefers the locale a regular user is allowed to read", () => { const info = interpretAccountInfo([settingsOk("de_DE.UTF-8"), accountOk("fr_FR")]); assert.equal(info.locale, "de-DE"); - assert.equal(info.generation, "0.16+"); }); test("falls back to x:Account when the settings object is forbidden", () => { @@ -27,22 +27,14 @@ test("falls back to x:Account when the settings object is forbidden", () => { assert.equal(info.locale, "sr-Latn-RS"); }); -test("an older server is recognised by its unknownMethod, and still yields a locale", () => { - const info = interpretAccountInfo([failed("s", "unknownMethod"), accountOk("en_GB")]); - assert.equal(info.generation, "pre-0.16"); - assert.equal(info.locale, "en-GB"); -}); - -test("a server answering the new method is 0.16+ even with no locale set", () => { +test("an account with no locale set yields none, rather than a guess", () => { const info = interpretAccountInfo([["x:AccountSettings/get", { list: [] }, "s"], failed("a", "forbidden")]); - assert.equal(info.generation, "0.16+"); assert.equal(info.locale, null); }); -test("neither answering leaves everything unknown rather than guessing", () => { - const info = interpretAccountInfo([failed("s", "forbidden"), failed("a", "forbidden")]); - assert.deepEqual(info, { locale: null, generation: null, edition: null }); - assert.deepEqual(interpretAccountInfo([]), { locale: null, generation: null, edition: null }); +test("neither answering leaves the locale unknown", () => { + assert.deepEqual(interpretAccountInfo([failed("s", "forbidden"), failed("a", "forbidden")]), { locale: null, edition: null }); + assert.deepEqual(interpretAccountInfo([]), { locale: null, edition: null }); }); test("locales that carry no language are dropped, not passed through", () => { @@ -50,20 +42,18 @@ test("locales that carry no language are dropped, not passed through", () => { assert.equal(interpretAccountInfo([settingsOk("POSIX")]).locale, null); }); -test("a server that never heard of the Stalwart capability is reported as pre-0.16", async () => { - // 0.16 always advertises urn:stalwart:jmap and nothing older knows it at all, - // so its absence is the answer - and asking anyway would fail the whole - // request on those servers. This is what the live 0.15.5 box hits. +test("a server without the registry is not asked for anything", async () => { + // Sign-in refuses these, so getAccountInfo should never reach the wire for + // one - and must not, since a server that cannot parse `urn:stalwart:jmap` + // fails the whole request rather than the one call. const session = { capabilities: { "urn:ietf:params:jmap:core": {}, "urn:ietf:params:jmap:mail": {} }, accounts: {}, primaryAccounts: {} }; - const info = await getAccountInfo("session-pre-016", "Basic x", session as never); - assert.equal(info.generation, "pre-0.16"); - assert.equal(info.locale, null); - assert.equal(info.edition, null); + const info = await getAccountInfo("session-unsupported", "Basic x", session as never); + assert.deepEqual(info, { locale: null, edition: null }); }); -test("no capabilities at all leaves the generation unknown", async () => { +test("no capabilities at all is treated the same way", async () => { const info = await getAccountInfo("session-no-caps", "Basic x", { accounts: {}, primaryAccounts: {} } as never); - assert.equal(info.generation, null); + assert.equal(info.locale, null); }); /** @@ -73,9 +63,12 @@ test("no capabilities at all leaves the generation unknown", async () => { * fixed list that has never carried this capability, in any 0.16.x. It is * handed out per-account instead, so it lands in `primaryAccounts` and in each * account's `accountCapabilities`. Looking only at the session level called - * every real 0.16 server pre-0.16, which sent self-service credentials to a + * every real 0.16 server too old, which sent self-service credentials to a * REST endpoint 0.16 had removed and made the About page report the wrong - * generation. + * thing. + * + * This check now decides whether a sign-in is allowed at all, so getting it + * wrong would lock every user out of a perfectly good server. */ const STALWART = "urn:stalwart:jmap"; const baseCaps = { "urn:ietf:params:jmap:core": {}, "urn:ietf:params:jmap:mail": {} }; @@ -102,7 +95,7 @@ test("the session level still counts, for a server that ever advertises it there assert.equal(hasStalwartRegistry({ capabilities: { ...baseCaps, [STALWART]: {} }, accounts: {}, primaryAccounts: {} }), true); }); -test("a server that advertises it nowhere is pre-0.16", () => { +test("a server that advertises it nowhere is one we do not support", () => { assert.equal(hasStalwartRegistry({ capabilities: baseCaps, accounts: { a1: { accountCapabilities: baseCaps } }, primaryAccounts: { "urn:ietf:params:jmap:mail": "a1" } }), false); assert.equal(hasStalwartRegistry(undefined), false); }); @@ -117,15 +110,3 @@ test("a shared account carrying the capability is enough to recognise the server true, ); }); - -test("a locale request that fails does not talk us out of a generation we proved", () => { - // The capability settled it. A forbidden reply costs the locale, nothing more. - const info = interpretAccountInfo([failed("s", "forbidden"), failed("a", "forbidden")], "0.16+"); - assert.equal(info.generation, "0.16+"); - assert.equal(info.locale, null); -}); - -test("a server that disowns the method is still older, whatever we came in believing", () => { - const info = interpretAccountInfo([failed("s", "unknownMethod")], "0.16+"); - assert.equal(info.generation, "pre-0.16"); -}); diff --git a/server/src/app.ts b/server/src/app.ts index 026dd9b..a2dc660 100644 --- a/server/src/app.ts +++ b/server/src/app.ts @@ -12,6 +12,7 @@ import { absoluteUpstream, expandTemplate, fetchUpstreamSession, + hasStalwartRegistry, forgetUpstreamSession, getAccountInfo, getUpstreamSession, @@ -25,7 +26,6 @@ import { createAppPassword, disableOtp, enableOtp, - forgetBackend, getState, revokeAppPassword, } from "./account.js"; @@ -180,6 +180,20 @@ export function createApp(): Hono { const authorization = `Basic ${Buffer.from(`${username}:${effectivePassword}`, "utf8").toString("base64")}`; try { const upstream = await fetchUpstreamSession(authorization); + // ihasmail requires Stalwart 0.16 or newer. Refuse here, once and + // clearly, rather than signing someone in and letting Files, the account + // locale and self-service credentials each fail in their own way with + // nothing to connect them. The credentials were good, so say so. + if (!hasStalwartRegistry(upstream)) { + return c.json( + { + error: "unsupported_server", + message: + "Your credentials are fine, but this mail server is older than Stalwart 0.16, which ihasmail needs. Upgrade the server, or run the release tagged stalwart-0.15-support.", + }, + 501, + ); + } loginLimiter.reset(limitKey); const { cookie, session } = sessions.create({ username, @@ -236,9 +250,8 @@ export function createApp(): Hono { // ---------- Self-service credentials ---------- /** * Password, app passwords and 2FA. These live on the server rather than in - * the browser because the pre-0.16 API is REST rather than JMAP (the browser - * only ever sees /api/jmap), and because changing a credential means - * re-sealing the session cookie that holds it. + * the browser because changing a credential means re-sealing the session + * cookie that holds it, and because the browser only ever sees /api/jmap. */ const accountCtx = async (c: Context) => { const session = c.get("session"); @@ -264,7 +277,7 @@ export function createApp(): Hono { api.get("/account/security", requireSession, async (c) => { const session = c.get("session"); try { - return c.json(await getState(session.id, await accountCtx(c))); + return c.json(await getState(await accountCtx(c))); } catch (err) { return accountFailure(c, err); } @@ -284,7 +297,7 @@ export function createApp(): Hono { return c.json({ error: "unchanged", message: "The new password matches the old one." }, 400); } try { - await changePassword(session.id, await accountCtx(c), { current, next, otpCode: body.otpCode?.trim() || undefined }); + await changePassword(await accountCtx(c), { current, next, otpCode: body.otpCode?.trim() || undefined }); } catch (err) { return accountFailure(c, err); } @@ -300,8 +313,8 @@ export function createApp(): Hono { api.get("/account/app-passwords", requireSession, async (c) => { const session = c.get("session"); try { - const state = await getState(session.id, await accountCtx(c)); - return c.json({ appPasswords: state.appPasswords, keyedByName: state.appPasswordsKeyedByName }); + const state = await getState(await accountCtx(c)); + return c.json({ appPasswords: state.appPasswords }); } catch (err) { return accountFailure(c, err); } @@ -314,7 +327,7 @@ export function createApp(): Hono { const description = (body.description ?? "").trim().slice(0, 120); if (!description) return c.json({ error: "missing_fields", message: "Give the app password a name." }, 400); try { - return c.json(await createAppPassword(session.id, await accountCtx(c), { description })); + return c.json(await createAppPassword(await accountCtx(c), { description })); } catch (err) { return accountFailure(c, err); } @@ -325,7 +338,7 @@ export function createApp(): Hono { const body = await readJson<{ id?: string }>(c); if (!body?.id) return c.json({ error: "bad_request" }, 400); try { - await revokeAppPassword(session.id, await accountCtx(c), body.id); + await revokeAppPassword(await accountCtx(c), body.id); return c.json({ ok: true }); } catch (err) { return accountFailure(c, err); @@ -366,18 +379,18 @@ export function createApp(): Hono { } let app: { id: string; secret: string } | null = null; try { - app = await createAppPassword(session.id, ctx, { description: appPasswordName(c) }); + app = await createAppPassword(ctx, { description: appPasswordName(c) }); } catch (err) { // Out of app-password quota, say. 2FA is still worth having; the user // just has to sign in again afterwards. console.warn("[ihasmail] could not mint a session app password:", (err as Error).message); } try { - await enableOtp(session.id, ctx, { url: body.url, code, current: body.current }); + await enableOtp(ctx, { url: body.url, code, current: body.current }); } catch (err) { if (app) { // Don't leave a credential behind for a change that never happened. - await revokeAppPassword(session.id, ctx, app.id).catch(() => {}); + await revokeAppPassword(ctx, app.id).catch(() => {}); } return accountFailure(c, err); } @@ -398,7 +411,7 @@ export function createApp(): Hono { const body = await readJson<{ current?: string; code?: string }>(c); if (!body?.current || !body.code) return c.json({ error: "bad_request" }, 400); try { - await disableOtp(session.id, await accountCtx(c), { current: body.current, code: body.code.trim() }); + await disableOtp(await accountCtx(c), { current: body.current, code: body.code.trim() }); } catch (err) { return accountFailure(c, err); } @@ -406,7 +419,6 @@ export function createApp(): Hono { // the plain password works again now, so put it back. sessions.reseal(getCookie(c, config.cookieName), body.current); forgetUpstreamSession(session.id); - forgetBackend(session.id); return c.json({ ok: true }); }); @@ -578,7 +590,7 @@ function appPasswordName(c: Context): string { return `${config.appName} (${browser})`; } -function sessionExtras(session: LiveSession, info: AccountInfo = { locale: null, generation: null, edition: null }) { +function sessionExtras(session: LiveSession, info: AccountInfo = { locale: null, edition: null }) { return { ihasmail: { appName: config.appName, @@ -591,7 +603,7 @@ function sessionExtras(session: LiveSession, info: AccountInfo = { locale: null, /** Locale configured for the account in Stalwart's directory, if readable. */ userLocale: info.locale, /** What the upstream server would tell us about itself. */ - server: { generation: info.generation, edition: info.edition }, + server: { edition: info.edition }, }, }; } diff --git a/server/src/login-guard.test.ts b/server/src/login-guard.test.ts new file mode 100644 index 0000000..5ad7bac --- /dev/null +++ b/server/src/login-guard.test.ts @@ -0,0 +1,73 @@ +import { test, before, after } from "node:test"; +import assert from "node:assert/strict"; + +/** + * ihasmail requires Stalwart 0.16 or newer. Sign-in is where that is enforced, + * and it matters that it is enforced *there*: the alternative is signing + * someone in and letting Files, the account locale and self-service + * credentials each fail in their own way, with nothing to connect the three or + * to say what the real problem is. + * + * The refusal also has to keep two things apart that look the same from the + * outside. Bad credentials are a 401 the user can fix by typing again; an + * unsupported server is not, and telling someone their password is wrong when + * it is not would send them round in circles. + */ + +const PORT = 18799; +process.env.MOCK_PORT = String(PORT); +process.env.MOCK_USER = "demo@example.com"; +process.env.MOCK_PASS = "demo-password"; +process.env.MOCK_NO_REGISTRY = "1"; // a server without urn:stalwart:jmap +process.env.STALWART_URL = `http://127.0.0.1:${PORT}`; +process.env.APP_SECRET = "test-secret-for-login-guard"; + +const mock = await import("./mock/index.js"); +const { createApp } = await import("./app.js"); + +const app = createApp(); +const HEADERS = { "content-type": "application/json", "x-requested-with": "ihasmail" }; + +async function login(body: unknown): Promise<{ status: number; body: any; setCookie: string | null }> { + const res = await app.request("/api/auth/login", { method: "POST", headers: HEADERS, body: JSON.stringify(body) }); + const text = await res.text(); + return { status: res.status, body: text ? JSON.parse(text) : null, setCookie: res.headers.get("set-cookie") }; +} + +before(() => { + assert.equal(process.env.MOCK_NO_REGISTRY, "1"); +}); + +after(() => { + (mock as { server?: { close(): void } }).server?.close(); +}); + +test("a server without the registry is refused, with good credentials", async () => { + const res = await login({ username: "demo@example.com", password: "demo-password" }); + assert.equal(res.status, 501); + assert.equal(res.body.error, "unsupported_server"); +}); + +test("the message says the credentials were fine, and names the way out", async () => { + const { body } = await login({ username: "demo@example.com", password: "demo-password" }); + // Someone hitting this has typed a correct password. Saying so is the + // difference between "upgrade your server" and "try your password again". + assert.match(body.message, /credentials are fine/i); + assert.match(body.message, /0\.16/); + assert.match(body.message, /stalwart-0\.15-support/, "the tag to build from if they cannot upgrade"); +}); + +test("no session is minted for a server we cannot talk to", async () => { + // A cookie here would leave a signed-in session against a server every + // other request is going to fail on. + const res = await login({ username: "demo@example.com", password: "demo-password" }); + assert.equal(res.setCookie, null); +}); + +test("bad credentials on such a server are still a 401, not the server error", async () => { + // The upstream session request fails first, and that answer is the honest + // one: we never got far enough to learn what the server supports. + const res = await login({ username: "demo@example.com", password: "wrong-password" }); + assert.equal(res.status, 401); + assert.notEqual(res.body.error, "unsupported_server"); +}); diff --git a/server/src/mock/index.ts b/server/src/mock/index.ts index f6c7266..f90bfcf 100644 --- a/server/src/mock/index.ts +++ b/server/src/mock/index.ts @@ -10,14 +10,12 @@ import { holdUntilOf, undoStatusOf } from "./futurerelease.js"; const PORT = Number(process.env.MOCK_PORT ?? 8788); /** - * Which Stalwart generation to impersonate. "0.16" (the default) has the - * registry — the `x:` methods, `nodeType` on FileNode, the finer-grained - * rights. "0.15" is the older shape, and differs in ways that mostly do not - * announce themselves: its FileNode/query cannot see directories at all, it - * refuses a `using` naming a capability it does not know, and self-service - * credentials live behind a REST endpoint instead. + * Omit `urn:stalwart:jmap` from the session, so a sign-in can be tested + * against a server ihasmail does not support. This is only that: the rest of + * the mock still behaves like 0.16. Emulating 0.15 properly went with the + * support for it. */ -const LEGACY = process.env.MOCK_STALWART === "0.15"; +const NO_REGISTRY = process.env.MOCK_NO_REGISTRY === "1"; /** * Stalwart advertises FUTURERELEASE in the session but only honours it when * the MTA's own `futureRelease` setting is on -- and that setting defaults to @@ -169,10 +167,7 @@ const fileNodes: Obj[] = [ { id: "f3", parentId: null, nodeType: "file", blobId: putBlob("%PDF-1.4 mock", "application/pdf"), size: 14, name: "report.pdf", type: "application/pdf", created: new Date().toISOString(), modified: new Date().toISOString(), myRights: fr() }, ]; function fr() { - // 0.16 split what used to be a single mayWrite into four. - return LEGACY - ? { mayRead: true, mayWrite: true, mayShare: true } - : { mayRead: true, mayAddChildren: true, mayRename: true, mayDelete: true, mayModifyContent: true, mayShare: true }; + return { mayRead: true, mayAddChildren: true, mayRename: true, mayDelete: true, mayModifyContent: true, mayShare: true }; } function recount() { @@ -616,32 +611,11 @@ const handlers: Record = { "ContactCard/parse": (a) => { const parsed: Obj = {}; for (const b of a.blobIds as string[]) { const t = blobs.get(b)?.data.toString() ?? ""; const fn = /^FN:(.*)$/m.exec(t)?.[1]?.trim() ?? "Imported"; const em = /^EMAIL[^:]*:(.*)$/m.exec(t)?.[1]?.trim(); parsed[b] = [{ "@type": "Card", version: "1.0", uid: randomUUID(), kind: "individual", name: { full: fn }, emails: em ? { e1: { address: em } } : undefined }]; } return { accountId: ACCOUNT, parsed, notParsable: [] }; }, "FileNode/query": (a) => { const f = (a.filter as Obj) ?? {}; - if (LEGACY) { - // Sorting is refused outright, and isTopLevel / nodeType are not filters - // this generation knows. - if (a.sort) throw new MethodError("unsupportedSort", "Sorting is not supported on FileNode"); - if ("isTopLevel" in f || "nodeType" in f) throw new MethodError("unsupportedFilter", "Unsupported filter"); - } - let list = fileNodes.filter((n) => (f.isTopLevel ? n.parentId == null : f.parentId ? n.parentId === f.parentId : true)); - // The pre-0.16 query masks its results to non-containers, so a directory - // never comes back — with nothing to say it was left out. - if (LEGACY) list = list.filter((n) => n.nodeType !== "directory"); + const list = fileNodes.filter((n) => (f.isTopLevel ? n.parentId == null : f.parentId ? n.parentId === f.parentId : true)); return { accountId: ACCOUNT, queryState: "1", canCalculateChanges: false, position: 0, ids: list.map((n) => n.id), total: list.length }; }, - "FileNode/get": (a) => { - const res = genericGet(fileNodes)(a); - // nodeType does not exist before 0.16; the shape is all the client gets. - if (LEGACY) res.list = (res.list as Obj[]).map((n) => { const { nodeType: _drop, ...rest } = n; return rest; }); - return res; - }, + "FileNode/get": genericGet(fileNodes), "FileNode/set": (a) => { - if (LEGACY) { - for (const obj of [...Object.values((a.create as Obj) ?? {}), ...Object.values((a.update as Obj) ?? {})]) { - if (obj && typeof obj === "object" && "nodeType" in (obj as Obj)) { - return setResp({ notCreated: Object.fromEntries(Object.keys((a.create as Obj) ?? {}).map((k) => [k, { type: "invalidProperties", properties: ["nodeType"], description: "Invalid property." }])), notUpdated: Object.fromEntries(Object.keys((a.update as Obj) ?? {}).map((k) => [k, { type: "invalidProperties", properties: ["nodeType"], description: "Invalid property." }])) }); - } - } - } return genericSet(fileNodes, "f", (o) => { Object.assign(o, { created: new Date().toISOString(), modified: new Date().toISOString(), myRights: fr(), size: o.blobId ? (blobs.get(o.blobId as string)?.data.length ?? 0) : null, type: o.type ?? null, blobId: o.blobId ?? null, ...o }); // Without nodeType, a node is a directory precisely when it carries no @@ -685,8 +659,8 @@ function readBody(req: IncomingMessage): Promise { const session = () => ({ capabilities: { "urn:ietf:params:jmap:core": { maxSizeUpload: 50000000, maxConcurrentUpload: 4, maxSizeRequest: 10000000, maxConcurrentRequests: 4, maxCallsInRequest: 16, maxObjectsInGet: MAX_OBJECTS, maxObjectsInSet: MAX_OBJECTS, collationAlgorithms: ["i;ascii-casemap"] }, "urn:ietf:params:jmap:mail": {}, "urn:ietf:params:jmap:submission": {}, "urn:ietf:params:jmap:vacationresponse": {}, "urn:ietf:params:jmap:sieve": { implementation: "mock" }, "urn:ietf:params:jmap:calendars": {}, "urn:ietf:params:jmap:calendars:parse": {}, "urn:ietf:params:jmap:contacts": {}, "urn:ietf:params:jmap:contacts:parse": {}, "urn:ietf:params:jmap:principals": {}, "urn:ietf:params:jmap:principals:availability": {}, "urn:ietf:params:jmap:quota": {}, "urn:ietf:params:jmap:blob": {}, "urn:ietf:params:jmap:filenode": {} }, - accounts: { [ACCOUNT]: { name: USER, isPersonal: true, isReadOnly: false, accountCapabilities: { "urn:ietf:params:jmap:mail": {}, "urn:ietf:params:jmap:submission": { maxDelayedSend: MAX_DELAYED_SEND, submissionExtensions: { FUTURERELEASE: [], SIZE: [], DSN: [], DELIVERYBY: [], "MT-PRIORITY": ["MIXER"], REQUIRETLS: [] } }, "urn:ietf:params:jmap:vacationresponse": {}, "urn:ietf:params:jmap:sieve": {}, "urn:ietf:params:jmap:calendars": {}, "urn:ietf:params:jmap:contacts": {}, "urn:ietf:params:jmap:principals": {}, "urn:ietf:params:jmap:quota": {}, "urn:ietf:params:jmap:filenode": {}, ...(LEGACY ? {} : { "urn:stalwart:jmap": {} }) } } }, - primaryAccounts: { ...Object.fromEntries(["mail", "submission", "vacationresponse", "sieve", "calendars", "contacts", "principals", "quota", "filenode", "blob"].map((c) => [`urn:ietf:params:jmap:${c}`, ACCOUNT])), ...(LEGACY ? {} : { "urn:stalwart:jmap": ACCOUNT }) }, + accounts: { [ACCOUNT]: { name: USER, isPersonal: true, isReadOnly: false, accountCapabilities: { "urn:ietf:params:jmap:mail": {}, "urn:ietf:params:jmap:submission": { maxDelayedSend: MAX_DELAYED_SEND, submissionExtensions: { FUTURERELEASE: [], SIZE: [], DSN: [], DELIVERYBY: [], "MT-PRIORITY": ["MIXER"], REQUIRETLS: [] } }, "urn:ietf:params:jmap:vacationresponse": {}, "urn:ietf:params:jmap:sieve": {}, "urn:ietf:params:jmap:calendars": {}, "urn:ietf:params:jmap:contacts": {}, "urn:ietf:params:jmap:principals": {}, "urn:ietf:params:jmap:quota": {}, "urn:ietf:params:jmap:filenode": {}, ...(NO_REGISTRY ? {} : { "urn:stalwart:jmap": {} }) } } }, + primaryAccounts: { ...Object.fromEntries(["mail", "submission", "vacationresponse", "sieve", "calendars", "contacts", "principals", "quota", "filenode", "blob"].map((c) => [`urn:ietf:params:jmap:${c}`, ACCOUNT])), ...(NO_REGISTRY ? {} : { "urn:stalwart:jmap": ACCOUNT }) }, username: USER, apiUrl: `http://127.0.0.1:${PORT}/jmap/`, downloadUrl: `http://127.0.0.1:${PORT}/jmap/download/{accountId}/{blobId}/{name}?accept={type}`, @@ -709,37 +683,8 @@ export const server = createServer(async (req, res) => { res.writeHead(200, { "content-type": "application/json" }); return res.end(JSON.stringify(session())); } - // Before 0.16, self-service credentials are a REST endpoint rather than - // registry objects: GET reports the state, POST takes a list of actions. - if (LEGACY && url.pathname === "/api/account/auth") { - if (req.method === "GET") { - res.writeHead(200, { "content-type": "application/json" }); - return res.end(JSON.stringify({ data: { otpEnabled: Boolean(account.otpUrl), appPasswords: account.appPasswords.map((a) => a.description) } })); - } - if (req.method === "POST") { - const actions = JSON.parse((await readBody(req)).toString()) as { type: string; password?: string; url?: string | null; name?: string }[]; - // Password and OTP changes are only accepted over Basic auth. - if (actions.some((a) => ["setPassword", "enableOtpAuth", "disableOtpAuth"].includes(a.type)) && !(req.headers.authorization ?? "").startsWith("Basic ")) { - res.writeHead(400, { "content-type": "application/json" }); - return res.end(JSON.stringify({ error: "unauthorized", details: "Password changes only allowed using Basic auth" })); - } - for (const a of actions) { - if (a.type === "setPassword") account.password = a.password ?? account.password; - else if (a.type === "enableOtpAuth") account.otpUrl = a.url ?? null; - else if (a.type === "disableOtpAuth") account.otpUrl = null; - else if (a.type === "addAppPassword") account.appPasswords.push({ id: `ap${randomUUID().slice(0, 6)}`, description: a.name ?? "App password", secret: a.password ?? "", createdAt: new Date().toISOString(), expiresAt: null }); - else if (a.type === "removeAppPassword") { - const i = account.appPasswords.findIndex((p) => p.description === a.name); - if (i >= 0) account.appPasswords.splice(i, 1); - } - } - res.writeHead(200, { "content-type": "application/json" }); - return res.end(JSON.stringify({ data: null })); - } - } - - // 0.16's account info endpoint; the only place a server reports its edition. - if (!LEGACY && url.pathname === "/api/account" && req.method === "GET") { + // The account info endpoint; the only place a server reports its edition. + if (url.pathname === "/api/account" && req.method === "GET") { res.writeHead(200, { "content-type": "application/json" }); return res.end(JSON.stringify({ permissions: ["jmapEmailGet", "sysAccountSettingsGet"], edition: "oss", locale: MOCK_LOCALE })); } @@ -763,7 +708,7 @@ export const server = createServer(async (req, res) => { for (const [name, rawArgs, id] of body.methodCalls) { const h = handlers[name]; // The registry, and every x: method with it, arrived in 0.16. - if (!h || (LEGACY && name.startsWith("x:"))) { responses.push(["error", { type: "unknownMethod" }, id]); continue; } + if (!h) { responses.push(["error", { type: "unknownMethod" }, id]); continue; } try { const args = resolveRefs(rawArgs, responses, creations); enforceLimits(name, args); @@ -810,7 +755,6 @@ export const server = createServer(async (req, res) => { res.end(JSON.stringify({ error: "not found" })); }).listen(PORT, "127.0.0.1", () => { console.log(`[mock-stalwart] listening on http://127.0.0.1:${PORT} (login: ${USER} / ${PASS})`); - console.log(`[mock-stalwart] impersonating Stalwart ${LEGACY ? "0.15 (pre-registry)" : "0.16+"}`); console.log(`[mock-stalwart] run the app with: STALWART_URL=http://127.0.0.1:${PORT} npm run dev`); }); diff --git a/server/src/sessions.test.ts b/server/src/sessions.test.ts index 9290584..91ae914 100644 --- a/server/src/sessions.test.ts +++ b/server/src/sessions.test.ts @@ -63,37 +63,3 @@ test("normalizes Stalwart account locales to BCP-47 tags", () => { assert.equal(normalizeLocale({ locale: "de_DE" }), null); assert.equal(normalizeLocale("../etc/passwd"), null); }); - -test("generated app passwords are unbiased and long enough", async () => { - const { readableSecret } = await import("./account.js"); - const alphabet = "abcdefghijkmnopqrstuvwxyz23456789"; - const counts = new Map(); - let samples = 0; - for (let i = 0; i < 2000; i++) { - const secret = readableSecret(); - assert.match(secret, /^[a-z2-9]{5}-[a-z2-9]{5}-[a-z2-9]{5}-[a-z2-9]{5}$/, secret); - for (const ch of secret.replace(/-/g, "")) { - counts.set(ch, (counts.get(ch) ?? 0) + 1); - samples++; - } - } - assert.equal(samples, 2000 * 20); - - /* - * `% 33` over a byte maps 25 characters onto 8 values each and the last 8 - * onto 7, so the digits — the tail of the alphabet — would come up about - * 7/8 as often as they should. Testing each character on its own cannot see - * a skew that size against the noise, so weigh the whole tail at once: - * uniform puts 8/33 of the draw there, the biased version 7/8 of that, and - * over 40,000 draws the two are more than four standard deviations apart. - */ - const tail = alphabet.slice(25); // "23456789" - const tailSeen = [...tail].reduce((n, ch) => n + (counts.get(ch) ?? 0), 0); - const p = tail.length / alphabet.length; - const expected = samples * p; - const sigma = Math.sqrt(samples * p * (1 - p)); - assert.ok( - Math.abs(tailSeen - expected) < 4 * sigma, - `digits appeared ${tailSeen} times, expected ~${Math.round(expected)} (sigma ${sigma.toFixed(1)}) - modulo bias?`, - ); -}); diff --git a/server/src/upstream.ts b/server/src/upstream.ts index 0741447..e77fa6c 100644 --- a/server/src/upstream.ts +++ b/server/src/upstream.ts @@ -78,10 +78,14 @@ const JMAP_CORE = "urn:ietf:params:jmap:core"; * builds that list from a fixed set that has never included this capability; * it hands it out per-account instead, so it turns up in `primaryAccounts` and * in each account's `accountCapabilities`. Checking only the session level - * therefore reports every real 0.16 server as pre-0.16 — which routed + * therefore reported every real 0.16 server as older than 0.16 — which routed * self-service credentials to a REST endpoint 0.16 had removed, and told the * About page the wrong thing. The session level is still checked last, in case * a later release advertises it there as well. + * + * This is now what sign-in tests to decide whether a server is supported at + * all, so the same mistake would lock every user out of a working server + * rather than merely misroute them. */ export function hasStalwartRegistry(session: Pick | undefined): boolean { if (!session) return false; @@ -96,22 +100,13 @@ export function hasStalwartRegistry(session: Pick(); const INFO_CACHE_MS = 30 * 60_000; -const EMPTY_INFO: AccountInfo = { locale: null, generation: null, edition: null }; -/** A server that has never heard of the registry: nothing to read, but dated. */ -const PRE_REGISTRY_INFO: AccountInfo = { locale: null, generation: "pre-0.16", edition: null }; -const REGISTRY_INFO: AccountInfo = { locale: null, generation: "0.16+", edition: null }; +const EMPTY_INFO: AccountInfo = { locale: null, edition: null }; /** * glibc modifiers that name a script rather than a dialect or a currency: @@ -165,13 +160,9 @@ export function normalizeLocale(raw: unknown): string | null { * tells us which generation we are talking to. */ async function fetchAccountInfo(authorization: string, session: UpstreamSession): Promise { - // Every 0.16 build advertises urn:stalwart:jmap, and no earlier one knows it - // at all, so its absence already answers the question — and asking anyway - // would fail the whole request, since those servers reject a `using` naming - // a capability they cannot parse. - // A session with no capabilities at all is not one we can read anything from. - if (!session.capabilities) return EMPTY_INFO; - if (!hasStalwartRegistry(session)) return PRE_REGISTRY_INFO; + // Sign-in refuses a server without the registry, so this should not happen — + // but a session we cannot read capabilities from is not one to ask. + if (!session.capabilities || !hasStalwartRegistry(session)) return EMPTY_INFO; const accountId = session.primaryAccounts?.[STALWART_CAP] ?? session.primaryAccounts?.["urn:ietf:params:jmap:mail"] ?? @@ -189,35 +180,23 @@ async function fetchAccountInfo(authorization: string, session: UpstreamSession) }), signal: AbortSignal.timeout(config.upstreamTimeout), }); - // The registry capability already settled the generation. A locale request - // that fails — a permission we lack, a hiccup upstream — can only cost us the - // locale; it must not talk us out of what we know. - if (!res.ok) return REGISTRY_INFO; + // A locale request that fails — a permission we lack, a hiccup upstream — + // costs us the locale and nothing else. + if (!res.ok) return EMPTY_INFO; const body = (await res.json()) as { methodResponses?: [string, Record, string][] }; - return interpretAccountInfo(body.methodResponses ?? [], "0.16+"); + return interpretAccountInfo(body.methodResponses ?? []); } /** - * Read the pair of replies: prefer the locale from `x:AccountSettings`, fall - * back to `x:Account` for servers (or permissions) where only that one works, - * and note which generation answered. + * Read the pair of replies: prefer the locale from `x:AccountSettings`, whose + * permission the built-in user role has, and fall back to `x:Account` for the + * accounts allowed the admin-only `sysAccountGet` instead. Both are 0.16 + * methods; this is a permissions fallback, not a version one. */ -export function interpretAccountInfo( - responses: [string, Record, string][], - known: AccountInfo["generation"] = null, -): AccountInfo { +export function interpretAccountInfo(responses: [string, Record, string][]): AccountInfo { const settings = responses.find((r) => r[2] === "s"); const account = responses.find((r) => r[2] === "a"); - // Only 0.16+ knows the method at all; older builds cannot even parse the name. - // `known` is what the session capability already proved, and outranks a reply - // that merely refused us. - const generation: AccountInfo["generation"] = - settings && settings[0] !== "error" - ? "0.16+" - : (settings?.[1] as { type?: string } | undefined)?.type === "unknownMethod" - ? "pre-0.16" - : known; - return { locale: localeOf(settings) ?? localeOf(account), generation, edition: null }; + return { locale: localeOf(settings) ?? localeOf(account), edition: null }; } function localeOf(call: [string, Record, string] | undefined): string | null { @@ -251,7 +230,7 @@ export async function getAccountInfo(sessionId: string, authorization: string, s let info = EMPTY_INFO; try { info = await fetchAccountInfo(authorization, session); - if (info.generation === "0.16+") info = { ...info, edition: await fetchEdition(authorization) }; + info = { ...info, edition: await fetchEdition(authorization) }; } catch { /* all of this is a nicety - never fail the session over it */ } diff --git a/web/src/jmap/types.ts b/web/src/jmap/types.ts index 4055c39..cdbad6b 100644 --- a/web/src/jmap/types.ts +++ b/web/src/jmap/types.ts @@ -36,8 +36,7 @@ export interface JmapSession { userLocale?: string | null; /** What the upstream server was willing to say about itself. */ server?: { - /** Which API generation answered: Stalwart publishes no version number. */ - generation?: "0.16+" | "pre-0.16" | null; + /** "oss" | "community" | "enterprise". Stalwart publishes no version. */ edition?: string | null; }; }; diff --git a/web/src/lib/__tests__/filenode.test.ts b/web/src/lib/__tests__/filenode.test.ts deleted file mode 100644 index 80175c7..0000000 --- a/web/src/lib/__tests__/filenode.test.ts +++ /dev/null @@ -1,161 +0,0 @@ -import { afterEach, describe, expect, it } from "vitest"; -import { client } from "@/jmap/client"; -import { directoryCreate, fileCreate, fileNodeProps, normalizeFileNodes, queryOmitsDirectories, supportsNodeType } from "../filenode"; -import type { FileNode, JmapSession } from "@/jmap/types"; - -/** - * `nodeType` arrived in Stalwart 0.16. Sending it to an older server fails the - * whole create with `invalidProperties (nodeType)` — which is what uploading a - * file or making a folder hit on the live 0.15.5 box. Those servers tell a file - * from a directory by whether it carries file properties at all. - */ - -function session(caps: string[]): JmapSession { - return { capabilities: Object.fromEntries(caps.map((c) => [c, {}])), accounts: {}, primaryAccounts: {}, state: "s" } as unknown as JmapSession; -} - -const NEW_SERVER = ["urn:ietf:params:jmap:core", "urn:ietf:params:jmap:filenode", "urn:stalwart:jmap"]; -const OLD_SERVER = ["urn:ietf:params:jmap:core", "urn:ietf:params:jmap:filenode"]; - -/** - * The session a real Stalwart 0.16 sends: `urn:stalwart:jmap` is handed out - * per-account and never appears in the session-level capabilities, so a client - * that only checks there drops every 0.16 server onto the older code path. - */ -function realStalwartSession(): JmapSession { - return { - capabilities: Object.fromEntries(OLD_SERVER.map((c) => [c, {}])), - accounts: { a1: { accountCapabilities: { "urn:ietf:params:jmap:filenode": {}, "urn:stalwart:jmap": {} } } }, - primaryAccounts: { "urn:stalwart:jmap": "a1" }, - state: "s", - } as unknown as JmapSession; -} - -afterEach(() => { - client.session = null; -}); - -describe("on Stalwart 0.16 and newer", () => { - it("uses nodeType everywhere", () => { - client.session = session(NEW_SERVER); - expect(supportsNodeType()).toBe(true); - expect(fileNodeProps()).toContain("nodeType"); - expect(directoryCreate(null, "ihasmail")).toEqual({ parentId: null, name: "ihasmail", nodeType: "directory" }); - expect(fileCreate("d1", "logo.png", "b1", "image/png")).toEqual({ parentId: "d1", name: "logo.png", blobId: "b1", type: "image/png", nodeType: "file" }); - }); - - it("leaves what the server reported alone", () => { - client.session = session(NEW_SERVER); - const nodes = [{ id: "1", name: "x", nodeType: "directory" }] as Partial[]; - expect(normalizeFileNodes(nodes)).toEqual(nodes); - }); -}); - -describe("on a real 0.16 session, which advertises per-account only", () => { - it("is recognised as 0.16 even though the session capabilities do not say so", () => { - client.session = realStalwartSession(); - expect(client.hasCapability("urn:stalwart:jmap")).toBe(false); - expect(supportsNodeType()).toBe(true); - expect(queryOmitsDirectories()).toBe(false); - expect(directoryCreate(null, "ihasmail")).toEqual({ parentId: null, name: "ihasmail", nodeType: "directory" }); - }); -}); - -describe("on Stalwart before 0.16", () => { - it("never mentions nodeType, in creates or in requested properties", () => { - client.session = session(OLD_SERVER); - expect(supportsNodeType()).toBe(false); - expect(fileNodeProps()).not.toContain("nodeType"); - expect(directoryCreate(null, "ihasmail")).toEqual({ parentId: null, name: "ihasmail" }); - expect(JSON.stringify(fileCreate("d1", "logo.png", "b1", "image/png"))).not.toContain("nodeType"); - }); - - it("keeps a directory free of file properties, which is what makes it one", () => { - client.session = session(OLD_SERVER); - const dir = directoryCreate(null, "ihasmail"); - // Setting blobId, size or type — even to null — would make this a file. - expect(dir).not.toHaveProperty("blobId"); - expect(dir).not.toHaveProperty("size"); - expect(dir).not.toHaveProperty("type"); - }); - - it("still sends what a file needs", () => { - client.session = session(OLD_SERVER); - expect(fileCreate("d1", "logo.png", "b1", "image/png")).toEqual({ parentId: "d1", name: "logo.png", blobId: "b1", type: "image/png" }); - }); - - it("works out nodeType from the file properties, so folders stay folders", () => { - client.session = session(OLD_SERVER); - const out = normalizeFileNodes([ - { id: "1", name: "Documents", blobId: null, size: null, type: null }, - { id: "2", name: "notes.txt", blobId: "b1", size: 11, type: "text/plain" }, - { id: "3", name: "empty.txt", blobId: "b2", size: 0, type: null }, - ] as Partial[]); - expect(out.map((n) => n.nodeType)).toEqual(["directory", "file", "file"]); - }); - - it("does not overwrite a nodeType that did come back", () => { - client.session = session(OLD_SERVER); - const out = normalizeFileNodes([{ id: "1", name: "x", nodeType: "symlink", blobId: "b1" }] as Partial[]); - expect(out[0]!.nodeType).toBe("symlink"); - }); -}); - -it("assumes the older shape when there is no session yet", () => { - client.session = null; - expect(supportsNodeType()).toBe(false); -}); - -/** - * Rights were split up in 0.16. Before that a node carried mayRead / mayWrite / - * mayShare, with mayWrite covering everything the newer release names - * separately — so Rename and Delete sat permanently greyed out, doing nothing - * and saying nothing. - */ -describe("rights on a pre-0.16 server", () => { - const oldRights = (mayWrite: boolean) => ({ mayRead: true, mayWrite, mayShare: false }); - - it("widens mayWrite into the rights the UI gates on", () => { - client.session = session(OLD_SERVER); - const [node] = normalizeFileNodes([{ id: "1", name: "x", myRights: oldRights(true) }] as unknown as Partial[]); - expect(node!.myRights).toMatchObject({ mayRead: true, mayAddChildren: true, mayRename: true, mayDelete: true, mayModifyContent: true, mayShare: false }); - }); - - it("does not hand out rights the server withheld", () => { - client.session = session(OLD_SERVER); - const [node] = normalizeFileNodes([{ id: "1", name: "x", myRights: oldRights(false) }] as unknown as Partial[]); - expect(node!.myRights).toMatchObject({ mayRename: false, mayDelete: false, mayModifyContent: false }); - }); - - it("leaves rights that already use the newer names untouched", () => { - client.session = session(OLD_SERVER); - const newer = { mayRead: true, mayAddChildren: true, mayRename: true, mayDelete: false, mayModifyContent: true, mayShare: true }; - const [node] = normalizeFileNodes([{ id: "1", name: "x", myRights: newer }] as unknown as Partial[]); - expect(node!.myRights).toEqual(newer); - }); - - it("copes with a node that reported no rights at all", () => { - client.session = session(OLD_SERVER); - const [node] = normalizeFileNodes([{ id: "1", name: "x" }] as Partial[]); - expect(node!.myRights).toBeUndefined(); - expect(node!.nodeType).toBe("directory"); - }); -}); - -/** - * Before 0.16, FileNode/query masks its results with `document_ids(false)` — - * only resources that are *not* containers. It therefore returns files and - * never folders, with no error to explain the omission: a folder created there - * exists but never comes back in a listing. FileNode/get carries no such mask. - */ -describe("directory-blind query", () => { - it("is worked around on older servers", () => { - client.session = session(OLD_SERVER); - expect(queryOmitsDirectories()).toBe(true); - }); - - it("is not worked around where query can see folders", () => { - client.session = session(NEW_SERVER); - expect(queryOmitsDirectories()).toBe(false); - }); -}); diff --git a/web/src/lib/appFolder.ts b/web/src/lib/appFolder.ts index bcfbac4..b95c443 100644 --- a/web/src/lib/appFolder.ts +++ b/web/src/lib/appFolder.ts @@ -7,53 +7,39 @@ * is what makes this state travel between devices without ihasmail storing * anything server-side of its own — but it is housekeeping rather than * something anyone filed there, so the Files view hides it. See `isAppFolder`. + * + * Both lookups below filter on `parentId`/`isTopLevel` alone and match the name + * here rather than asking the server to. Those are the filters Files itself + * relies on; `name` is not one Stalwart is known to implement, and a filter it + * does not know fails the whole query rather than being ignored. */ import { client, setErrorMessage } from "@/jmap/client"; import type { FileNode, GetResponse, Id, SetResponse } from "@/jmap/types"; -import { directoryCreate, normalizeFileNodes, queryOmitsDirectories, supportsNodeType } from "@/lib/filenode"; +import { directoryCreate } from "@/lib/filenode"; export const APP_FOLDER = "ihasmail"; -/** Just enough to find the folder, asking for nodeType only where it exists. */ -export const folderProps = (): string[] => - supportsNodeType() ? ["id", "name", "nodeType", "parentId"] : ["id", "name", "parentId", "blobId", "size", "type"]; +/** Just enough to find the folder. */ +export const folderProps = (): string[] => ["id", "name", "nodeType", "parentId"]; /** The client's own folder, which the Files view does not show. */ export function isAppFolder(n: Pick): boolean { return n.name === APP_FOLDER && !n.parentId && n.nodeType === "directory"; } -/** Every node in the account, for servers whose query cannot see directories. */ -async function allNodes(accountId: Id, properties: string[]): Promise { - const res = await client.call>("FileNode/get", { accountId, ids: null, properties }); - return normalizeFileNodes(res.list); +/** List one level of the tree: the top level, or the children of a folder. */ +async function children(accountId: Id, parentId: Id | null, properties: string[]): Promise { + const filter = parentId ? { parentId } : { isTopLevel: true }; + const res = await client.chain([ + ["FileNode/query", { accountId, filter, limit: 1000 }, "q"], + ["FileNode/get", { accountId, "#ids": { resultOf: "q", name: "FileNode/query", path: "/ids" }, properties }, "g"], + ]); + return (res.get("g")?.[0] as unknown as GetResponse).list; } /** Find the app folder, or make it. Returns its node id. */ export async function ensureFolder(accountId: Id): Promise { - const props = folderProps(); - let list: FileNode[] = []; - if (queryOmitsDirectories()) { - // Query cannot see a directory on these servers, so it would never find the - // folder and we would make a fresh one on every save. Ask get for the lot. - list = await allNodes(accountId, props); - } else { - try { - const res = await client.chain([ - ["FileNode/query", { accountId, filter: { isTopLevel: true, nodeType: "directory", name: APP_FOLDER }, limit: 5 }, "q"], - ["FileNode/get", { accountId, "#ids": { resultOf: "q", name: "FileNode/query", path: "/ids" }, properties: props }, "g"], - ]); - list = normalizeFileNodes((res.get("g")?.[0] as unknown as GetResponse).list); - } catch { - // Filters unsupported: scan everything and pick it out here. - const res = await client.chain([ - ["FileNode/query", { accountId, limit: 1000 }, "q"], - ["FileNode/get", { accountId, "#ids": { resultOf: "q", name: "FileNode/query", path: "/ids" }, properties: props }, "g"], - ]); - list = normalizeFileNodes((res.get("g")?.[0] as unknown as GetResponse).list); - } - } - const existing = list.find(isAppFolder); + const existing = (await children(accountId, null, folderProps())).find(isAppFolder); if (existing) return existing.id; const set = await client.call>("FileNode/set", { accountId, create: { d: directoryCreate(null, APP_FOLDER) } }); const err = set.notCreated?.d; @@ -61,7 +47,10 @@ export async function ensureFolder(accountId: Id): Promise { return set.created!.d!.id; } -/** A node's persistent blobId, for servers that do not return one on create. */ +/** + * A node's persistent blobId. `FileNode/set` does not return one on create, so + * anything that needs the blob straight after making the node has to ask. + */ export async function nodeBlobId(accountId: Id, id?: Id): Promise { if (!id) return undefined; try { @@ -74,18 +63,7 @@ export async function nodeBlobId(accountId: Id, id?: Id): Promise { - const props = ["id", "name", "parentId", "blobId", "size", "type", ...(supportsNodeType() ? ["nodeType"] : [])]; - try { - const res = await client.chain([ - ["FileNode/query", { accountId, filter: { parentId: folderId, name }, limit: 5 }, "q"], - ["FileNode/get", { accountId, "#ids": { resultOf: "q", name: "FileNode/query", path: "/ids" }, properties: props }, "g"], - ]); - const list = normalizeFileNodes((res.get("g")?.[0] as unknown as GetResponse).list); - const hit = list.find((n) => n.name === name && n.parentId === folderId); - if (hit) return hit; - } catch { - /* filters unsupported: fall through to the full scan */ - } - const list = await allNodes(accountId, props); + const props = ["id", "name", "parentId", "blobId", "size", "type", "nodeType"]; + const list = await children(accountId, folderId, props); return list.find((n) => n.name === name && n.parentId === folderId); } diff --git a/web/src/lib/filenode.ts b/web/src/lib/filenode.ts index dfe507e..8436207 100644 --- a/web/src/lib/filenode.ts +++ b/web/src/lib/filenode.ts @@ -1,92 +1,26 @@ /** - * FileNode compatibility across Stalwart releases. + * FileNode shapes, as Stalwart 0.16 defines them. * - * `nodeType` arrived in 0.16. Before that a FileNode had no such property at - * all, and the server rejects the whole create with - * `invalidProperties (nodeType)` — which is what uploading a file or making a - * folder used to hit. Older servers instead tell a file from a directory by - * whether it carries file properties at all: set `blobId`, `size` or `type` - * (even to null) and the node becomes a file, leave them off and it is a - * directory. - * - * 0.16 is also the first release to advertise `urn:stalwart:jmap`, and no - * earlier one knows that capability, so its presence is a reliable stand-in for - * "this server has the newer FileNode shape" — as long as it is looked for in - * `primaryAccounts` and `accountCapabilities`, which is where Stalwart puts it, - * and not only in the session-level `capabilities`, where it never appears. + * This used to be a compatibility layer spanning 0.15 and 0.16, which differ + * in ways the server does not report: `nodeType` did not exist and sending it + * failed the create outright, `FileNode/query` masked directories out of its + * own results, and rights were a single `mayWrite` rather than the four + * separate ones. ihasmail requires 0.16 now — sign-in refuses anything older — + * so a node has one shape and there is nothing left to detect. */ -import { client } from "@/jmap/client"; -import type { FileNode, Id } from "@/jmap/types"; +import type { Id } from "@/jmap/types"; -const STALWART_CAP = "urn:stalwart:jmap"; - -export function supportsNodeType(): boolean { - // Not `hasCapability`: Stalwart advertises this per-account, never in the - // session-level capabilities, so looking only there treats every real 0.16 - // server as pre-0.16 and drops Files onto the older code path. - return client.hasCapabilityAnywhere(STALWART_CAP); -} - -const BASE_PROPS = ["id", "parentId", "blobId", "size", "name", "type", "created", "modified", "myRights", "role", "executable"]; - -/** - * Whether `FileNode/query` is blind to directories. - * - * Before 0.16 the query masks its results with `document_ids(false)`, which - * keeps only resources that are *not* containers — so it returns files and - * never folders, with no error to say so. A folder created there is real, and - * simply never comes back in a listing. `FileNode/get` has no such mask, so - * asking it for every id is the only way to see the whole tree. - */ -export function queryOmitsDirectories(): boolean { - return !supportsNodeType(); -} - -/** Properties to request, asking for `nodeType` only where it exists. */ +/** Properties to request for a node. */ export function fileNodeProps(): string[] { - return supportsNodeType() ? [...BASE_PROPS, "nodeType"] : BASE_PROPS; + return ["id", "parentId", "blobId", "size", "name", "type", "created", "modified", "myRights", "role", "executable", "nodeType"]; } /** Create-arguments for a directory. */ export function directoryCreate(parentId: Id | null, name: string): Record { - // Any file property — blobId, size, type — would make this a file on an - // older server, so a directory there is exactly parentId plus name. - return supportsNodeType() ? { parentId, name, nodeType: "directory" } : { parentId, name }; + return { parentId, name, nodeType: "directory" }; } /** Create-arguments for a file with an already-uploaded blob. */ export function fileCreate(parentId: Id | null, name: string, blobId: Id, type: string): Record { - const base = { parentId, name, blobId, type }; - return supportsNodeType() ? { ...base, nodeType: "file" } : base; -} - -/** - * Fill in what an older server does not report, so everything downstream — - * icons, sorting, "may I delete this" — can read the 0.16 shape. - * - * Rights were split up in 0.16. Before that a node carried `mayRead`, - * `mayWrite` and `mayShare`, with the one `mayWrite` covering everything the - * newer release names separately. Without translating it, the Rename and - * Delete menu items sit permanently greyed out: no error, just nothing. - */ -export function normalizeFileNodes>(nodes: T[]): T[] { - if (supportsNodeType()) return nodes; - return nodes.map((n) => ({ - ...n, - nodeType: n.nodeType ?? (isFile(n) ? "file" : "directory"), - myRights: widenRights(n.myRights), - })); -} - -type Rights = FileNode["myRights"]; - -function widenRights(rights: Rights | undefined): Rights | undefined { - if (!rights) return rights; - const r = rights as Rights & { mayWrite?: boolean }; - if (r.mayDelete !== undefined || r.mayWrite === undefined) return rights; // already the newer shape - return { ...r, mayAddChildren: r.mayWrite, mayRename: r.mayWrite, mayDelete: r.mayWrite, mayModifyContent: r.mayWrite }; -} - -function isFile(n: Partial): boolean { - return n.blobId != null || n.size != null || n.type != null; + return { parentId, name, blobId, type, nodeType: "file" }; } diff --git a/web/src/lib/settingsSync.ts b/web/src/lib/settingsSync.ts index bfe4b27..62023e4 100644 --- a/web/src/lib/settingsSync.ts +++ b/web/src/lib/settingsSync.ts @@ -15,15 +15,11 @@ * and the file overwrites it once it lands. A browser with no cache (a private * window) therefore shows defaults for one frame before the account's real * settings arrive. - * - * Requires Stalwart 0.16: `FileNode/query` before that cannot see directories - * and the rights model differs. On an older server the settings simply stay - * local, exactly as they were. */ import { CAP, client, setErrorMessage } from "@/jmap/client"; import type { FileNode, Id, SetResponse } from "@/jmap/types"; import { ensureFolder, findInFolder, nodeBlobId } from "@/lib/appFolder"; -import { fileCreate, supportsNodeType } from "@/lib/filenode"; +import { fileCreate } from "@/lib/filenode"; import { useSession } from "@/store/session"; const FILE = "settings.json"; @@ -40,7 +36,7 @@ let armed = false; let listenersBound = false; export function settingsSyncAvailable(): boolean { - return supportsNodeType() && client.hasCapability(CAP.filenode) && Boolean(useSession.getState().accountFor(CAP.filenode)); + return client.hasCapability(CAP.filenode) && Boolean(useSession.getState().accountFor(CAP.filenode)); } /** diff --git a/web/src/store/files.ts b/web/src/store/files.ts index 91d6d18..8a92188 100644 --- a/web/src/store/files.ts +++ b/web/src/store/files.ts @@ -1,6 +1,6 @@ import { create } from "zustand"; -import { CAP, JmapMethodError, client, setErrorMessage } from "@/jmap/client"; -import { directoryCreate, fileCreate, fileNodeProps, normalizeFileNodes, queryOmitsDirectories } from "@/lib/filenode"; +import { CAP, client, setErrorMessage } from "@/jmap/client"; +import { directoryCreate, fileCreate, fileNodeProps } from "@/lib/filenode"; import { isAppFolder } from "@/lib/appFolder"; import type { FileNode, GetResponse, Id, QueryResponse, SetResponse } from "@/jmap/types"; import { useSession } from "./session"; @@ -26,10 +26,6 @@ interface FilesState { } -/** Whether the server supports parentId/isTopLevel query filters (detected at runtime). */ -let filtersSupported = true; - -const byName = (a: FileNode, b: FileNode) => (a.nodeType === b.nodeType ? a.name.localeCompare(b.name, undefined, { numeric: true, sensitivity: "base" }) : a.nodeType === "directory" ? -1 : 1); /** * Drop the client's own `ihasmail` folder, and everything inside it, from a @@ -56,42 +52,6 @@ export function withoutAppFolder(nodes: FileNode[]): FileNode[] { return nodes.filter((n) => !hidden.has(n.id)); } -/** Fetch all nodes (paged, no filter) and rebuild the full children map. */ -async function loadAllNodes(accountId: Id, set: (fn: (s: FilesState) => Partial) => void): Promise { - const all: FileNode[] = []; - if (queryOmitsDirectories()) { - // Query would hand back files only, so every folder — including one just - // created — would be missing with nothing to say why. Ask get for the lot. - const res = await client.call>("FileNode/get", { accountId, ids: null, properties: fileNodeProps() }); - all.push(...normalizeFileNodes(res.list)); - } else { - let position = 0; - for (let guard = 0; guard < 100; guard++) { - const res = await client.chain([ - ["FileNode/query", { accountId, position, limit: 500, calculateTotal: true }, "q"], - ["FileNode/get", { accountId, "#ids": { resultOf: "q", name: "FileNode/query", path: "/ids" }, properties: fileNodeProps() }, "g"], - ]); - const q = res.get("q")?.[0] as unknown as QueryResponse; - const g = res.get("g")?.[0] as unknown as GetResponse; - all.push(...normalizeFileNodes(g.list)); - position += q.ids.length; - if (!q.ids.length || (q.total != null && position >= q.total)) break; - } - } - // After the whole collection, not per page: the folder and its contents can - // land in different pages, and a half-filtered pass would spill the rest. - const visible = withoutAppFolder(all); - const nodes: Record = {}; - const children: Record = { root: [] }; - for (const n of visible) nodes[n.id] = n; - for (const n of visible.sort(byName)) { - const key = n.parentId && nodes[n.parentId] ? n.parentId : "root"; - (children[key] ??= []).push(n.id); - } - for (const n of visible) children[n.id] ??= []; - set(() => ({ nodes, children, loading: false, error: null })); -} - export const useFiles = create((set, get) => ({ accountId: null, available: false, @@ -113,10 +73,6 @@ export const useFiles = create((set, get) => ({ if (!accountId) return; set({ loading: true }); try { - if (!filtersSupported || queryOmitsDirectories()) { - await loadAllNodes(accountId, set); - return; - } const filter = parentId ? { parentId } : { isTopLevel: true }; const res = await client.chain([ ["FileNode/query", { accountId, filter, sort: [{ property: "nodeType", isAscending: false }, { property: "name", isAscending: true }], limit: 1000 }, "q"], @@ -124,7 +80,7 @@ export const useFiles = create((set, get) => ({ ]); const q = res.get("q")?.[0] as unknown as QueryResponse; const g = res.get("g")?.[0] as unknown as GetResponse; - const listed = withoutAppFolder(normalizeFileNodes(g.list)); + const listed = withoutAppFolder(g.list); const keep = new Set(listed.map((n) => n.id)); set((s) => { const nodes = { ...s.nodes }; @@ -132,18 +88,10 @@ export const useFiles = create((set, get) => ({ return { nodes, children: { ...s.children, [parentId ?? "root"]: q.ids.filter((id) => keep.has(id)) }, loading: false, error: null }; }); } catch (err) { - // Older Stalwart releases don't support parentId / isTopLevel filters: fall back to - // fetching every node and building the tree client-side. - if (err instanceof JmapMethodError && (err.type === "unsupportedFilter" || err.type === "unsupportedSort")) { - filtersSupported = false; - try { - await loadAllNodes(accountId, set); - return; - } catch (err2) { - set({ loading: false, error: (err2 as Error).message }); - return; - } - } + // There used to be a fallback here that abandoned filters and fetched + // every node in the account, because 0.15 refused parentId/isTopLevel. + // 0.16 supports them, and quietly loading the whole tree instead would + // hide a real fault behind a performance cliff nobody would notice. set({ loading: false, error: (err as Error).message }); } }, diff --git a/web/src/views/settings/AboutSettings.tsx b/web/src/views/settings/AboutSettings.tsx index 210c345..09ee7a3 100644 --- a/web/src/views/settings/AboutSettings.tsx +++ b/web/src/views/settings/AboutSettings.tsx @@ -28,7 +28,7 @@ export function AboutSettings() { Image privacy proxy{session?.ihasmail?.imageProxy ? "enabled" : "disabled"} -

Stalwart does not publish its version number to mail clients, so ihasmail reports the API generation it detected instead.

+

Stalwart does not publish its version number to mail clients, so ihasmail reports the edition where the server gives one. ihasmail requires 0.16 or newer, and sign-in refuses anything older.

Server capabilities

{caps.map((c) => {c.replace("urn:ietf:params:jmap:", "")})} @@ -39,12 +39,11 @@ export function AboutSettings() { /** * Stalwart deliberately withholds its version from clients (it reports a fixed - * "1.0.0" wherever it publishes one at all), so the most honest thing we can - * show is which generation of its API answered us, plus the edition where the - * server reports it. + * "1.0.0" wherever it publishes one at all), so the edition is all there is to + * show. The generation used to be reported here too, back when ihasmail spoke + * to both 0.15 and 0.16; it requires 0.16 now, so signing in at all is the + * answer to that question. */ -function describeServer(server: { generation?: "0.16+" | "pre-0.16" | null; edition?: string | null } | undefined): string { - if (!server?.generation) return "not detected"; - const generation = server.generation === "0.16+" ? "0.16 or newer" : "older than 0.16"; - return server.edition ? `${generation} (${server.edition})` : generation; +function describeServer(server: { edition?: string | null } | undefined): string { + return server?.edition ? `0.16 or newer (${server.edition})` : "0.16 or newer"; } diff --git a/web/src/views/settings/SecuritySettings.tsx b/web/src/views/settings/SecuritySettings.tsx index ffa6fc7..cf64f38 100644 --- a/web/src/views/settings/SecuritySettings.tsx +++ b/web/src/views/settings/SecuritySettings.tsx @@ -26,17 +26,15 @@ interface AppPasswordRow { } interface SecurityState { - backend: "registry" | "legacy"; otpEnabled: boolean; appPasswords: AppPasswordRow[]; - appPasswordsKeyedByName: boolean; } export function SecuritySettings() { const [rows, setRows] = useState(null); const [current, setCurrent] = useState(""); const [state, setState] = useState(null); - /** Set when the server has no self-service API at all (pre-0.15 or a proxy). */ + /** Set when the server has no self-service API at all (a proxy, say). */ const [unsupported, setUnsupported] = useState(null); const session = useSession((s) => s.session); const logout = useSession((s) => s.logout); @@ -342,12 +340,12 @@ function AppPasswords({ state, reload }: { state: SecurityState | null; reload:

{state.appPasswords.length > 0 && ( - {!state.appPasswordsKeyedByName && } + {state.appPasswords.map((row) => ( - {!state.appPasswordsKeyedByName && } + ))} @@ -361,7 +359,6 @@ function AppPasswords({ state, reload }: { state: SecurityState | null; reload: - {state.appPasswordsKeyedByName &&

This mail server identifies app passwords by name, so give each one a different name.

} setIssued(null)} title="Your new app password" size="sm" footer={}>
NameCreated
NameCreated
{row.description}{row.createdAt ? formatFullDate(row.createdAt) : "—"}{row.createdAt ? formatFullDate(row.createdAt) : "—"}