Out of office: jcoffey-dev is traveling from November 1 to November 6. Responses to issues may be slow or delayed during this time, but I will be checking in.
A calendar object whose recurrence rules use up the expansion work budget
was logged with the whole object as the rule expansion error's details.
Anyone who can get an object stored, a sharee or a sender whose invitation
is processed, can cause that on purpose and repeatedly, copying calendar
content into the server log each time.
The error is still logged, with its reason and limit, but without the
object. Other rule expansion errors keep logging it as before.
Test: on the previous commit the budget test's object (40 events) came back
as the log details; now it is left out, and an event with no start date
still logs its object.
The discovery document, the JWKS, user info answers and their error text
were read whole, so a provider that misbehaved, or anyone able to answer
in its place, could make the server hold any amount of memory per sign-in.
Each is now read up to 1 MiB (introspection answers too), checking the
announced length first and counting as it reads; anything larger is a
provider error.
A JWT without a kid was tried against the cached keys only, so once the
provider rotated its key every such token failed until a restart or
reload. When no cached key verifies its signature, the keys are now
fetched again once, at most once every 300 seconds as for an unknown
kid; a failed fetch waits out the interval too, so tokens can't make the
server hammer a provider that is down.
Test: on main a user info answer over 1 MiB signed in (unit test: "user
info"), and a kid-less token signed with a rotated-in key was refused
(unit test: left None, right Some("[email protected]")). Now user info,
introspection and JWKS answers over 1 MiB are refused, the kid-less
token signs in after one fetch, and a forged one right after doesn't
fetch again (unit tests with a local provider).
A bearer token that isn't a JWT was accepted whenever the provider's
user info endpoint answered 200 for it, which it does for any token it
issued to any application. So, with no audience or scope check possible
on that path, an access token another application at the same provider
received for a user signed that user in here.
Each OIDC directory now has opaqueTokens: reject, userinfo or
introspection. introspection posts the token to the provider's RFC 7662
endpoint (introspectionUrl, or the discovery document's) with HTTP Basic
client authentication (introspectionClientId and
introspectionClientSecret), and signs in only an active token whose iss
is the directory's issuer, whose aud includes requireAudience when set,
that hasn't expired, isn't marked as an ID or refresh token and carries
the required scopes. A refusal of the server's own credentials is a
provider error, not a failed sign-in. Left unset, a directory uses
introspection when it has an introspection endpoint and rejects opaque
tokens otherwise; saving one that would introspect without a client ID
and secret is refused.
Directories saved before this read as userinfo, so an upgrade changes
nothing until an administrator chooses. The new fields are stored at
version 2 of x:Directory and of x:Bootstrap (the two objects an OIDC
directory is stored in); older records read without them. The admin
console gets the fields, their descriptions and an Opaque Tokens form
section from the served schema, and the privacy catalog lists the
client secret as a credential. The per-domain directories spec now
describes the audience rule and the policy.
Test: on main a directory signed in an opaque token that the user info
endpoint accepted (unit test: "an opaque token signed in through the
user info endpoint"). Now a new directory refuses it, one saved before
the policy still takes it, and introspection signs in only an active,
same-issuer, right-audience access token (unit tests with a local
provider; directory suite against Keycloak, including a refresh token
and a token Keycloak never issued; records stored at the old versions
read back as userinfo).
An external OpenID Connect directory with no required audience accepted
any token its provider issued, to any application: an access token
minted for another app at the same provider, or an unmarked ID token
(Google, Entra), signed in as that user. Anyone holding such a token,
which includes every other application the user signs in to there,
could use it as a bearer token here.
Creating an OIDC directory without requireAudience is now refused with
a validation error that says what to set, whether it is created through
x:Directory/set, by changing another directory into an OIDC one, or at
first-run setup. An audience, once set, can't be removed. A directory
saved before this keeps working without one and stays editable, and
each time it loads (startup, reload, every node) the server logs an
auth warning naming its issuer and description. x:Directory/get shows
requireAudience as null for such a directory, which is what the admin
console reads to warn about it. A later release will require the
audience on every directory.
Test: on main x:Directory/set created an OIDC directory without an
audience (directory suite: "Missing not created item 0"). Now it is
refused, a directory stored without one still signs John in and can be
edited, and removing the audience from one that has it is refused
(directory suite and a unit test of the rules).
A calendar's recurrence rules were each allowed 100,000 passes that find
nothing before calcard gave up on them, counted afresh for every rule and
after every instance found, and nothing bounded the total. Anyone who can
put an object on a calendar (a CalDAV or JMAP upload, or an iMIP invitation
the server processes) could send a few thousand components whose rules never
match, such as FREQ=MONTHLY;BYMONTH=2;BYMONTHDAY=31, or rules that match
rarely, and storing or processing that object cost tens of seconds
of CPU. The per-period candidate check doesn't catch these: each period
is small, there are just very many of them.
Expansion now has one budget of 1,000,000 work units for all the rules of
the object together (about 50 ms in a release build). Every pass over a
period costs a unit, a pass that finds nothing a unit per 16 days it
scanned, candidates looked at but not produced a unit each, and so does
each time skipped while looking for the next BYHOUR/BYMINUTE/BYSECOND
match. When it runs out, expansion stops the way it does at the instance
limit: the object is kept and the instances found so far are indexed. The
expansion also reports it, so the server logs a calendar rule expansion
error naming it. Rules are no longer started once the instance limit is
reached. Ordinary rules use a small part of the budget: 3,000 daily,
weekly, monthly, yearly or every-leap-day instances expand as before.
Test: in calcard's new budget tests, 40 rules that never match took
3,999,960 iterations on the vendored copy and a sparse daily rule
2,922,306 (both over the 1,000,000 asserted); both now stop within the
budget and report it. 4,000 never-matching components (412 KB) took
13.7 s (daily), 31.7 s (yearly) and 41.8 s (monthly) in a release build,
now 36 ms, 45 ms and 57 ms.
The server now builds calcard, its iCalendar and vCard library, from
vendor/calcard instead of crates.io, patched in with [patch.crates-io]
like sieve-rs, so it can carry fixes to it. The copy is the published
0.3.14 with the same checksum Cargo.lock had, under Apache-2.0 OR MIT;
its 26 MB of test samples are left out. Nothing changes in how the
server behaves: Cargo.lock only drops calcard's registry source.
Test: the workspace builds, calcard's own tests pass from vendor/calcard
(52; the six that read the left-out samples are skipped), and the webdav
suite passes.
A Sieve script's CPU limit counted instructions only, and one header,
address or body test can read a whole message once for every key it
lists. A user's script with a few thousand keys in one body test, tested
against a large message anyone can send, spent seconds of CPU on each
delivery while staying far below the 5000-instruction limit; more such
tests multiplied it.
Tests now charge what they scan: each value compared times the number
of keys, and the header block each time a header is looked up. Every
64 KiB scanned counts as one instruction. A test stops as soon as the
run is over the limit, and the script fails with the same CPU limit
error as for too many instructions (a deleteheader cut short deletes
nothing). Under 64 KiB scanned costs nothing extra, so a usual script on
a usual message runs as before; a user script's default of 5000 allows
about 300 MiB scanned, a system script's 1,048,576 about 64 GiB.
Test: on main one body test with 100 keys over a 4 MiB message ran in 3
instructions and a 100-pass loop over it in 503 (8,000 keys over 20 MiB:
14.2 s); both now stop with CPULimitReached (the 20 MiB case in 10 ms),
and a script on a 16 KiB message fits the same limit as before.
Every IMAP command after sign-in started its own task with no limit, so a
signed-in client could pipeline a burst of commands and have the server
start as many tasks at once as the burst held, bounded only by the
per-account request rate. The tasks also ran side by side whatever they
did: an APPEND into a mailbox created by the CREATE just before it failed
with TRYCREATE, a STATUS or FETCH sent after an APPEND or STORE could miss
its result, and an EXPUNGE could run while an earlier COPY was still
reading the messages.
Each session now has a gate of four permits. A command that only reads
(FETCH without marking messages seen, SEARCH, SORT, THREAD, STATUS, LIST,
GETACL, MYRIGHTS, GETQUOTA) takes one; a command that changes the account
or the session (APPEND, COPY, MOVE, STORE, CREATE, DELETE, RENAME,
SUBSCRIBE, SETACL, a FETCH that sets \Seen, and every command run without
a task of its own, such as SELECT, EXPUNGE and CLOSE) takes all four. So
at most four reads run together, a write starts once everything before it
has finished, and later commands wait for it (RFC 9051, section 5.5). The
session waits for the permit before it starts the command and reads
nothing more from the client meanwhile.
Test: a new imap suite check pipelines CREATE, eight APPENDs and STATUS in
one write; on main the APPENDs failed with TRYCREATE before the CREATE
completed and STATUS saw no mailbox, now all complete in order and STATUS
reports 8; it also pipelines STORE then FETCH. Unit tests of the gate
check the bound and that a write never overlaps another operation.
With plain-text sign-in allowed, IMAP, POP3 and ManageSieve offered
OAUTHBEARER and XOAUTH2 on unencrypted connections and accepted them.
SMTP submission did the same wherever its mechanism setting listed them,
and the default setting listed them on submission ports before
STARTTLS. ManageSieve also offered them on clear text with plain-text
sign-in off, though it then refused them. A mail app would send its
bearer token in the clear, where anyone on the network path could read
it and use it as the account until it expired, in any protocol and over
HTTP.
Bearer token mechanisms now need TLS (implicit, STARTTLS, or a proxy
that reports TLS) in all four protocols, whatever the plain-text or
mechanism settings say. They are left out of the greeting, CAPABILITY,
CAPA and EHLO lists on clear text, and a request for one is refused
before the token is asked for: IMAP with NO [PRIVACYREQUIRED],
ManageSieve with NO (ENCRYPT-NEEDED), POP3 with -ERR and SMTP with 538
5.7.11. Password sign-in keeps following the plain-text setting.
ManageSieve offers an empty SASL list on clear text when plain-text
sign-in is off, next to STARTTLS.
Test: on clear-text IMAP, POP3 and ManageSieve with plain-text sign-in
allowed, the greetings and capability lists offered OAUTHBEARER and
XOAUTH2 and AUTHENTICATE answered "+" or tried the token on main; SMTP
EHLO without TLS offered OAUTHBEARER where the setting allowed every
mechanism. Now none is offered, each request is refused, PLAIN is still
offered, and bearer sign-in over TLS still works.
jcoffey-dev