Enabling background notifications failed with "Invalid filter". The
subscription asked to be notified about mail matching:
filter: { inMailbox: null, notKeyword: "$seen" }
`inMailbox: null` meant "the inbox" in my head and nothing at all to
Stalwart, which needs a mailbox id there. It refused the whole
subscription, so the feature did not work at all for anyone who tried
it.
The Inbox's id is now passed in and used. Where it is not known the
condition is left out rather than sent empty: notifying more widely is a
worse default than filtering to the Inbox, but it is a working one, and
sending a malformed filter is not a fallback.
Two reasons this got out, both worth fixing rather than just the bug:
- The tests checked the properties list and its ordering, and never
looked at the filter. There is now one that walks every condition
and fails on a null or undefined value, for both the known-inbox and
unknown-inbox cases.
- The mock accepted it happily, so nothing local disagreed with the
code. It now refuses a filter condition with a null value and
answers "Invalid filter.", which is what the live server said.
Reproduced: the old payload is rejected, the new one accepted.
ihasmail's notifications came from EventSource, which lives exactly as
long as a tab does -- so "desktop notifications" has always quietly
meant "while you are looking". That switch is now labelled as much, and
a second one does the thing people assumed the first one did.
Stalwart 0.16 signs Web Push with VAPID (RFC 9749) and can put the
message itself in the payload (draft-ietf-jmap-emailpush). The server
pushes straight to the browser's own push service: ihasmail's server is
not in the delivery path, there is no relay to run, and nothing beyond
the browser vendor's endpoint that Web Push requires of everyone.
Checked against the live 0.16.19 before any of this was written, because
an advertised capability is not a configured one:
- the session publishes a real applicationServerKey, so no key
generation or server configuration is needed
- PushSubscription/get answers an ordinary user rather than refusing
- emailpush is advertised, and its draft defines a filter, an ordered
properties list and an urgency -- so the payload can carry sender and
subject, and the server drops properties from the end when it will
not fit rather than failing the notification
Three things this gets right that are easy to get wrong:
- The verification handshake. A JMAP subscription delivers nothing
until the client echoes back a code the server pushed, and the
service worker cannot answer it -- no credentials in that context.
It forwards the code to a tab, or leaves it in the cache when no tab
was open to forward it to.
- Key encoding. The W3C Push API produces unpadded base64url and
Stalwart 0.16 was fixed to accept exactly that, so nothing here pads
on the way out. The VAPID key needs padding on the way *in* for
atob; getting that backwards fails at subscribe() with an opaque
error, so it lives in one named function with tests.
- Sign-out. A subscription belongs to the account, not the session.
Without tearing it down, a shared machine keeps notifying for a
mailbox nobody is signed into -- which is somebody else's mail.
The mock models the JMAP half, including refusing padded keys and
non-https endpoints, and creating subscriptions *unverified*. Delivery
cannot be mocked -- it runs through the browser vendor's real push
service -- but a mock that marked a subscription verified on creation
would let a client ship without the handshake, and the symptom in
production is "registered, and silent".
Not verified end to end: an actual notification arriving. That needs a
real browser, a real push service and real delivery, so it is live
testing or nothing.
Junk Mail can now be emptied in one action, the way every other mail
client offers it: a banner across the top of the folder, and an item in
both the folder's right-click menu and the list's own menu.
The messages are destroyed rather than moved to Deleted Items. Routing
spam through the bin on its way out leaves you with the same problem in
a different folder, and "delete all spam" means gone everywhere else. So
the dialog says it before you commit, and there is no undo.
emptyMailbox already did the hard part -- walking a folder a page at a
time so it survives maxObjectsInSet, which a Deleted Items of 5192 once
did not. All that changed is which folders it will accept. The guard
stays in the store rather than living only in the menus, so a fourth
caller cannot empty the Inbox by asking nicely.
The three entry points share one helper, because three dialogs warning
about a permanent deletion in three slightly different ways is how one
of them ends up not warning at all. A folder with nothing in it offers
the item greyed out rather than hiding it, so it is where you expect it
to be next time.
Folder naming is fixed in the same commit because it changed the same
file, and because testing this is what surfaced it. Two problems, one
cause:
- The mock called its folders "Trash" and "Sent". Stalwart's defaults
follow the Exchange convention -- "Deleted Items", "Sent Items" --
so anything built from a folder's name read differently against the
mock than against a real server, and every screenshot in the README
showed a folder list no user has.
- Worse, and in shipping code: the undo toast took a hardcoded label
in preference to the folder's actual name, so deleting a message
announced "moved to Trash" on a server whose folder is called
"Deleted Items", and reporting spam said "moved to Spam" where it is
"Junk Mail". The one message whose job is saying where mail went was
naming somewhere that does not exist. It now prefers the mailbox's
own name and keeps the hardcoded word only as a fallback.
Verified against the mock: the banner appears only in Junk and only with
something to delete, the dialog counts and pluralises, the messages are
destroyed and Deleted Items stays empty afterwards, the banner
disappears once the folder is, the item greys out when empty, Archive is
offered neither, and Trash still says "Empty Deleted Items".
ihasmail called itself "2.0" on the About page and "2.0.0" from
/api/health, both hardcoded, in four places that had drifted from each
other and from anything meaningful. A build now says what it is:
ihasmail v2.16.57
| | |
| | the pull request the commit came from
| the Stalwart generation this build targets -- 0.16
ihasmail's own major
The first two are the version in the root package.json, so there is a
single place to bump them, and 16 becomes 17 when ihasmail moves to
Stalwart 0.17. Dropping 0.15 is what makes that middle number honest:
while two generations were supported it could not have been either.
The pull request number comes from git at build time and is never
written back into the tree. It cannot be: it does not exist until the
pull request has merged, so a committed version would always describe a
merge that had not happened yet, and every open branch would collide on
the same line. A commit that did not come through a pull request carries
the last number plus its own short SHA -- 2.16.57+g1fa6578 -- which says
it is past that pull request rather than quietly claiming to be it.
.dockerignore excludes .git on purpose, so an image build cannot work
any of this out. It takes --build-arg IHASMAIL_VERSION instead, which
the build stage bakes into the bundle and the runtime stage keeps as an
environment variable for the server. Left out, it falls back to the base
version from package.json rather than failing -- so a version with no PR
number on it means whoever built the image did not pass one.
scripts/ is copied into the runtime image because the server resolves
its version through it. There is no git in there to ask, which is the
fallback's whole purpose.
Verified: 2.16.57 in the bundle and from /api/health on a dev checkout;
the same after a real docker build --build-arg, from inside the
container; and 2.16.0 rather than a crash when the arg is left off.
Note for deploying: ihasmail-deploy.sh on the host builds without the
argument and will produce 2.16.0 until it passes
--build-arg IHASMAIL_VERSION="$(node scripts/version.mjs)".
ihasmail spoke to two generations of Stalwart that are less alike than
their version numbers suggest: 0.16 replaced the REST management API with
JMAP registry objects, changed the shape of FileNode, split its rights
up, and moved configuration into the store. Carrying both meant 34 branch
points across nine files, a 92-line compatibility shim whose only job was
telling them apart, a parallel REST implementation of every credential
operation, and a mock that had to model both.
The branches were not the real cost. The cost was that a wrong answer
about which generation had answered always had somewhere to fall back to,
so it failed quietly rather than loudly: one capability looked for in the
wrong place downgraded every real 0.16 server onto the 0.15 path, which
posted the current password to an endpoint 0.16 had removed, reported the
wrong generation on About, and ran Files on the older code. It reached
production and was recorded as verified when it was not. The mock
mirrored the same wrong placement, which is why the tests agreed.
Removed: the filenode compatibility shim, the dual "registry" | "legacy"
backend in account.ts, the pre-0.16 generation in AccountInfo and
everything that read it, the mock's LEGACY mode and dev:mock:legacy, and
the three test files that existed only to pin 0.15 behaviour.
Sign-in now refuses an older server by name, once, rather than letting
Files, the account locale and credentials each fail in their own way with
nothing connecting them. It says the credentials were fine -- someone
hitting this has typed a correct password, and telling them otherwise
sends them round in circles -- and names the tag to build from. Four
tests cover it, including that no session cookie is minted and that bad
credentials on such a server are still a plain 401.
Two fallbacks went that were not strictly about 0.15, and both for the
same reason the removal is happening. Files no longer answers a refused
filter or sort by fetching every node in the account, which would hide a
real fault behind a performance cliff nobody would notice. And the app
folder lookups now filter on parentId/isTopLevel alone and match names
client-side, since `name` is not a filter Stalwart is known to implement
and one it does not know fails the whole query rather than being ignored.
The last release that runs on 0.15 is tagged stalwart-0.15-support.
Verified against the mock end to end: sign-in, the Files tree on the 0.16
path with the app folder hidden, and self-service credentials over the
registry. 226 web + 75 server tests pass; typecheck and build clean.
Three things a licence audit turned up. None of them is a conflict --
every one of the 182 installed packages is permissive, and the relicence
was within the copyright holder's gift -- but all three are ways the
AGPL fails to stick.
The offer was hard-coded to this repository. Section 13 asks whoever
runs a modified version to offer *that* version's source, so every
deployment with a patch in it was pointing at the wrong tree, and would
have gone on doing so unless its operator noticed and edited the About
page. SOURCE_URL now sets it, alongside APP_NAME, and both the sign-in
page and About read it.
The offer was also only visible after signing in. Whoever is looking at
the sign-in form is interacting with the program over a network too, so
the footer carries it now.
And the two workspace packages declared no licence at all. Private, so
npm never minded, but anything reading the tree saw a blank where the
rest of the project says AGPL-3.0-or-later.
Checked both ways round: with SOURCE_URL set to a fork, the sign-in page
and About both point at the fork; with it unset, both fall back to this
repository.
Guests added to an event vanished on save and no invitation was ever
sent. Not a guard in the editor, and nothing the server complained
about: ihasmail addresses a participant the way RFC 8984 does, with
sendTo and email, and Stalwart 0.16 keeps that address under
calendarAddress. Handed the RFC's spelling it stores the event, drops
the entire participant map, and reports success. Six shapes were tried
against a live 0.16.19, down to sendTo and roles alone; all six were
dropped, and patching a participant onto an existing event fails
outright with "Patch operation failed".
The same disagreement runs through two more properties. The organizer is
organizerCalendarAddress, not replyTo. A recurrence is a single
recurrenceRule, not a recurrenceRules array — and that one Stalwart
refuses honestly, with invalidProperties, so no recurring event could be
created at all and existing ones showed no repeat.
So writes now use Stalwart's names and reads accept either, since a
mailbox may hold events written by other clients. The mock now refuses
what the real server refuses and drops what it drops: advertising the
RFC spelling is exactly how this reached a live server unnoticed, the
same way the capability-placement bug did.
Verified against 0.16.19: participants, organizer and rule all survive a
create, an update and a re-read, with the roles kept as sent.
Fixes#26Fixes#30
Both branches turn on where Stalwart advertises a capability, so they meet
in the same two files. The mock keeps `urn:stalwart:jmap` out of the
session-level capabilities and hands it out per-account, as a real server
does, while the submission capability it grew for scheduled send lives
per-account beside it; the client keeps both accessors, one asking whether a
capability is advertised anywhere and one reading the object itself.
Self-service credentials, the About page and Files all keyed off
`urn:stalwart:jmap`, and all three looked for it in the session-level
`capabilities`. Stalwart has never put it there. `Session::new` builds that
list from a fixed set the capability is not part of, in any 0.16.x from
0.16.0 to 0.16.19; it is handed out per-account instead, so it arrives in
`primaryAccounts` and in each account's `accountCapabilities`.
So every real 0.16 server read as pre-0.16. Password changes, 2FA and app
passwords fell back to `POST /api/account/auth`, which 0.16 removed, and
reported that the server offers no self-service credential management. About
named the wrong generation. Files ran the pre-0.16 path, omitting `nodeType`
and listing the tree through get.
Look in all three places, on both sides. Two nearby soft spots go with it: a
transport error while probing the registry no longer downgrades a server to
the legacy path -- which would have posted the current password to an
endpoint that is not there -- and a locale request that is merely refused no
longer discards a generation the capability had already settled.
The mock advertised the capability in the session, which is why no test ever
caught this; it now advertises it where the real server does, and validates
`using` by the urn rather than by the session, as Stalwart does. Put the old
lookup back and nine tests fail.
Stalwart still publishes no version number to clients -- VERSION_PUBLIC is a
fixed "1.0.0" -- so About continues to report the generation and edition,
which are now the right ones.
Scheduled send, which the README listed as needing server support that
Stalwart has had all along. The delay cannot be asked for directly --
RFC 8621 makes `sendAt` read-only and server-derived -- so it goes on the
envelope as an RFC 4865 `HOLDUNTIL` parameter, and the server reports back
the time it settled on.
Stalwart advertises this in the *account* capability, not the session-level
one (which is empty): `maxDelayedSend` of thirty days and `FUTURERELEASE`
among its `submissionExtensions`. The composer offers scheduling only when
both are there, and never offers a time the server would refuse.
A held message goes to a Scheduled folder rather than Sent, because
`onSuccessUpdateEmail` would otherwise file it as sent the moment the
submission is created, and it has not been sent. Nothing moves it out when
the hold expires, so the folder is reconciled on the way in: released
messages to Sent, cancelled ones back to Drafts. Cancelling uses a separate
`Email/set` rather than `onSuccessUpdateEmail`, whose key Stalwart reads as
an Email id and not, as the RFC says, a submission id.
The mock grows the whole lifecycle, and learns to resolve creation
references while it is there -- it had been quietly declining to create any
submission at all, since sending names its message as `#m`. Because
Stalwart's own `futureRelease` setting defaults to off and then drops the
hold in silence, `npm run dev:mock:no-future-release` reproduces that.
Verified end to end against the mock; not yet against the live server.
Emptying Deleted Items back-referenced one Email/query straight into one
Email/set, so every id in the folder arrived in a single call. Stalwart
refuses the whole call over maxObjectsInSet with requestTooLarge, which
left a folder of 5192 messages impossible to empty at all.
Walk the folder a page at a time instead: the filter re-runs each pass,
so the next page is whatever is still there. A pass that destroys nothing
stops the loop and reports the server's error rather than spinning.
The same defect sat in two neighbours. Delete-forever on a large
selection sent every id in one Email/set, and mark-all-read fed up to
5000 ids into an Email/get that only echoed them back - past
maxObjectsInGet, and for no gain, since the query already returned them.
Both now page through the same ceiling, read from the session rather than
hardcoded.
The mock advertised maxObjectsInSet but never enforced it, so none of
this could fail in a test. It now rejects oversized get and set calls the
way Stalwart does.
While here, restrict emptying to Deleted Items. It was offered on Junk
too, where a permanent one-shot clear is harder to justify; Junk is now
select-all plus Delete, which takes the batched path.
The proxy resolved a hostname, refused it if any answer pointed somewhere
private, and then handed the *hostname* to fetch — which resolved it again when
the socket opened. An attacker who controls the zone answers with a public
address the first time and 127.0.0.1 the second, and the check has been walked
straight past. It is the standard way an SSRF guard gets bypassed.
Resolve once and connect to that address: a `lookup` that returns what we
already approved, on `node:http`/`node:https` rather than fetch, since fetch
gives no say in how the socket is opened. Every redirect hop is re-checked and
re-pinned. TLS is unaffected — the certificate is still validated against the
hostname, which `servername` and the Host header carry.
Pooling had to go with it: sockets are keyed by host and port, not by the
address we pinned, so a connection opened earlier would be reused and the pin
never consulted. Found by a test, not by reading it back.
Also refuse two ranges the old check let through: IPv6 multicast, and the NAT64
prefix, which is a route into IPv4 space.
**The login rate limiter could be sidestepped.** X-Forwarded-For is a list each
hop appends to, and nginx's $proxy_add_x_forwarded_for appends ours — so a
client sending "X-Forwarded-For: 1.2.3.4" arrives as "1.2.3.4, <their real
address>". Reading the leftmost entry, as we did, handed the caller a
rate-limit key they could change per request: unlimited password guessing
against a deployment that looks correctly configured. Read from the right
instead, skip hops that are themselves trusted proxies, and believe the header
only when the peer is one (loopback and the private ranges by default,
TRUSTED_PROXIES to be explicit).
**The upload cap was a suggestion.** It read content-length, which a chunked
request simply omits. Count the bytes through a stream, as the image proxy
already does.
**App password secrets were drawn with a modulo.** 256 is not a multiple of 33,
so the first 25 characters of the alphabet came up on 8 byte values and the
last 8 on only 7. Rejection sampling instead. The test weighs the whole tail of
the alphabet rather than single characters, because a 7/8 skew is invisible
per character against the noise — and it does fail when the bias is put back.
**Upstream headers were relayed wholesale.** Anything the mail server set —
cookies, auth challenges, CORS grants — landed on our origin, where it means
something else. Allowlist what is actually wanted.
MOCK_STALWART=0.15 (or npm run mock:legacy) switches the mock to the
generation before the registry. It is not a cut-down mock: it reproduces the
specific ways that generation differs, and every one of them is a thing the
server does not report as an error.
- urn:stalwart:jmap is not a capability it knows, and naming one it cannot
parse fails the whole request rather than the one call
- x: methods do not exist; credentials live at POST /api/account/auth
- FileNode/query masks out containers, so it returns files and never folders
- FileNode has no nodeType, and rights are only mayRead/mayWrite/mayShare
Both modes now also enforce the 2047-byte signature cap, and `using` is
validated in both — the gap that let the Identity capability bug in #12 ship.
This gives the legacy credential adapter its first automated coverage: it was
the least-tested code here, checked only by hand against the live server. The
new tests also pin the mock's own fidelity, so it cannot quietly drift back to
being 0.16-shaped in the places that matter.
**About said "not detected".** Generation was only worked out from the reply to
a registry method, which we never send to a server that does not advertise
urn:stalwart:jmap — every 0.16 build does, and nothing older knows the
capability at all, so its absence is already the answer. Say so, instead of
shrugging. A session with no capabilities at all stays unknown, which is a
different thing from old.
**The caret jumped out of the OTP field after one digit.** Dialog's autofocus
effect listed onClose in its dependencies, and every caller passes an inline
arrow, so each keystroke in a dialog holding state tore the effect down, set it
up again, and refocused the first field — which in the disable-2FA dialog is
the password. Keep the handler in a ref so the effect depends only on `open`.
This was a bug in the shared dialog rather than in one screen; every dialog
with more than one field had it.
The test for it fails against the old dependency array, not just passes
against the new one.
Stalwart before 0.16 does not know urn:stalwart:jmap, and rejects the whole
request rather than the one call when `using` names a capability it cannot
parse. The probe is only sent when the session advertises that capability, so
this should not arise — but if it ever does, throwing turns a server we can
still manage credentials on into a Security page that only shows an error.
Fall through to the endpoint those servers do have.
The account locale came from `x:Account/get`, which needs `sysAccountGet` — a
permission the built-in `user` role is not given, so the setting silently fell
back to the browser locale for exactly the people most likely to have set it.
Stalwart 0.16 carries the same field on `x:AccountSettings`, whose
`sysAccountSettingsGet` *is* part of that role. Both are now asked for in one
request and whichever answers wins, so admins and older servers keep working.
That pair of replies also says which generation we are talking to: only 0.16+
can parse the method name at all. About now reports that, plus the edition
from /api/account where the server offers it. It does not report a version
number because Stalwart does not publish one to clients — it hardcodes a
public "1.0.0" and keeps the real version to its SMTP internals — so the
screen says what was actually detected rather than inventing precision.
Also adds a light/dark toggle to the top bar, left of the settings button. The
stored setting is three-way, so the button acts on the theme actually on
screen: whichever one you see, a click gives you the other. Choosing "match
system" again stays in Settings › Appearance, where a three-way choice belongs.
Settings › Security grows three working sections instead of a note telling
people to use Stalwart's own portal.
Stalwart moved this API between releases, so ihasmail speaks both: 0.16+ has
the x:AccountPassword singleton and x:AppPassword registry objects over JMAP,
while 0.15.x has the /api/account/auth REST endpoint. Which one answers the
probe is the only reliable way to tell them apart, and the result is cached
per session. The built-in `user` role already grants sysAccountPassword* and
sysAppPassword*, so no administrator setup is needed.
Two problems are worth calling out, because both would bite a user hard:
Stalwart validates the credentials already on the account when 2FA is turned
on and never checks the new secret, so an authenticator that was mistyped or
out of step would lock someone out of their mailbox at the next sign-in. We
verify a code against the new secret ourselves first (RFC 6238, tested against
the spec's vectors) and only then ask the server to store anything.
Every proxied call re-authenticates with the credential sealed into the
session, and from the moment 2FA is on Stalwart wants a fresh TOTP code with
it — which we cannot produce between requests. Turning 2FA on would therefore
sign the user out of the browser they just turned it on in. App passwords
authenticate without a second factor, so the session is moved onto one minted
for this browser, and the session cookie is re-sealed with it. The order
matters: it is minted while the old credential still works, and revoked again
if enabling then fails.
Password changes re-seal this session too and drop the others, whose sealed
copies of the old password would fail on their next call.
The mock now enforces what a real server does — current password, password
policy, a TOTP code on every request once 2FA is on, app passwords exempt —
so the whole flow is exercised in tests rather than only by hand.
Every user-visible date now goes through web/src/lib/datetime.ts, driven by
three settings (Settings > General > Locale):
- Language & region: automatic, or any of the 618 locales CLDR has data for,
each named in its own language and script (web/src/lib/locales.ts, generated
by probing Intl over the subtag space).
- Date format: automatic (locale order), 22.11.2025, 22/11/2025, 11/22/2025,
or ISO 8601 2025-11-22.
- Time format: automatic (locale), 24-hour, or 12-hour.
Automatic takes the locale Stalwart has for the account, read best-effort at
login via x:Account/get (urn:stalwart:jmap) and passed to the client in the
session; servers without the capability, or that deny sysAccountGet to a
regular user, fall back to the browser locale. POSIX forms are normalised
(de_DE.UTF-8 -> de-DE) and script modifiers kept (sr_RS@latin -> sr-Latn-RS,
uz_UZ@cyrillic -> uz-Cyrl-UZ), while dialect/variant/currency modifiers are
dropped and a script the locale already implies is not appended.
Numerals follow the locale (22.11.2025 renders as Arabic-Indic digits under
ar-EG); ISO 8601 is the exception and pins date and clock to Latin digits so
one line never mixes digit systems.
Rewired: message list and headers, quoted reply headers, calendar (titles,
weekday and hour gutters, mini calendar, agenda, popovers, invite cards,
free/busy), contacts, files, sessions. No raw toLocale*String date calls are
left in web/src.
Native <input type="datetime-local"> pickers always follow the browser locale
and cannot be restyled by a page, so the out-of-office fields echo the entered
instant in the chosen format underneath.
Also: month-grid day labels no longer wrap when they hold a date, and the mock
server serves x:Account/get (MOCK_LOCALE, default en_US).
Closes#1
- Move the expand chevron into a gutter left of the folder icon so folders
with and without subfolders line up on their icon; labels share the column.
- Add "Mark all as read, incl. subfolders" to the folder context menu, with
the affected count and a per-folder fallback for servers without filter
operators.
- Re-measure the virtualised message list when row height changes.
- Mock: seed unread mail in a subfolder.
jcoffey-dev