Sign-in finishes on any replica #113

Merged
jcoffey-dev merged 1 commits from fix/signin-across-replicas into main 2026-11-18 21:59:21 +00:00
Owner

Found while building the chart's webmail Deployment (K-C3, tracker inbuxa/inbuxa-charts#14). It finishes what #103 (K-W1) set out to do.

The bug: a sign-in through the server's page kept its state and PKCE verifier in a Map in the process that started it. With two replicas behind a round-robin balancer, the callback reaches the other replica about half the time, and the person sees "This sign-in didn't start in this browser". Sealed cookie sessions already move between replicas once someone is signed in; the sign-in itself couldn't.

The fix: the pending sign-in travels in the state cookie the browser already carried, as <state>.<sealed record>.

  • Sealed with AES-256-GCM, under a key derived from APP_SECRET with its own HKDF label, so it can never open a session cookie or the reverse. APP_SECRET_PREVIOUS keys open it too.
  • The callback compares the state inside the sealed record with the URL's.
  • Still one attempt per sign-in: the callback deletes the cookie, and the server takes each code once and only with that sign-in's verifier.
  • The 10,000-pending cap goes with the Map, since nothing is held in the process any more.
  • The cookie is about 420 bytes, lives 10 minutes, and is sent only on /api/auth.

This applies to the memory session backend too: one code path, and a single-node install behaves the same.

Tests (308 server, 1,602 web, typecheck, build):

  • New: the callback succeeds after the process forgets everything (as on another replica); an altered cookie, a stray character, and a state with nothing sealed are all refused; a genuine cookie for a different state is refused.
  • Changed: "a state is good for one attempt" now replays a copy of the cookie and finds the code spent (exchange_failed, no session). Before, it got state_mismatch from the Map.
  • The start test checks the cookie holds nothing readable and is under 1 KB.

I'll verify it in the kind lab with two webmail replicas as part of K-C3.

Found while building the chart's webmail Deployment (K-C3, tracker inbuxa/inbuxa-charts#14). It finishes what #103 (K-W1) set out to do. **The bug:** a sign-in through the server's page kept its `state` and PKCE verifier in a `Map` in the process that started it. With two replicas behind a round-robin balancer, the callback reaches the other replica about half the time, and the person sees "This sign-in didn't start in this browser". Sealed cookie sessions already move between replicas once someone is signed in; the sign-in itself couldn't. **The fix:** the pending sign-in travels in the state cookie the browser already carried, as `<state>.<sealed record>`. - Sealed with AES-256-GCM, under a key derived from `APP_SECRET` with its own HKDF label, so it can never open a session cookie or the reverse. `APP_SECRET_PREVIOUS` keys open it too. - The callback compares the state inside the sealed record with the URL's. - Still one attempt per sign-in: the callback deletes the cookie, and the server takes each code once and only with that sign-in's verifier. - The 10,000-pending cap goes with the `Map`, since nothing is held in the process any more. - The cookie is about 420 bytes, lives 10 minutes, and is sent only on `/api/auth`. This applies to the memory session backend too: one code path, and a single-node install behaves the same. **Tests** (308 server, 1,602 web, typecheck, build): - New: the callback succeeds after the process forgets everything (as on another replica); an altered cookie, a stray character, and a state with nothing sealed are all refused; a genuine cookie for a different state is refused. - Changed: "a state is good for one attempt" now replays a copy of the cookie and finds the code spent (`exchange_failed`, no session). Before, it got `state_mismatch` from the `Map`. - The start test checks the cookie holds nothing readable and is under 1 KB. I'll verify it in the kind lab with two webmail replicas as part of K-C3.
jcoffey-dev added this to the 27.1 milestone 2026-11-18 21:43:22 +00:00
jcoffey-dev added 1 commit 2026-11-18 21:43:22 +00:00
Sign-in finishes on any replica
ci / node (pull_request) Skipped
ci / version (pull_request) Skipped
ci / docker-build (pull_request) Skipped
ci / publish (pull_request) Skipped
github/ci (branch) GitHub Actions
ci / github (pull_request) Successful in 2m5s
ci / announce (pull_request) Skipped
a39bafccaf
A sign-in through the server's page kept its state and PKCE verifier in a
Map in the process that started it. With two replicas behind a balancer,
the callback reached the other one about half the time, and the person
saw "This sign-in didn't start in this browser". Sealed cookie sessions
(K-W1) let a signed-in session move between replicas; the sign-in itself
still couldn't.

The pending sign-in now travels in the state cookie the browser already
carried: <state>.<sealed record>, AES-256-GCM under a key derived from
APP_SECRET with its own label, opened under APP_SECRET_PREVIOUS too. The
callback checks the state inside the sealed record against the URL's.

Each sign-in is still good for one attempt: the callback deletes the
cookie, and the server takes each code once and only with that sign-in's
verifier. The 10,000-sign-in cap goes with the Map: nothing is held here
any more. The cookie is about 420 bytes and lives 10 minutes on
/api/auth only.
Author
Owner

Verified in the kind lab with two replicas: the sign-in starts on one, finishes on the other, and the session works on both (inbuxa/inbuxa-charts K-C3, lab lab/chart/signin.py).

Control: the same test against main without this change fails at the callback on the second replica with signin_error=state_mismatch, and its state cookie is the bare 32-byte state. With this change the cookie is 407 bytes.

Verified in the kind lab with two replicas: the sign-in starts on one, finishes on the other, and the session works on both (inbuxa/inbuxa-charts K-C3, lab `lab/chart/signin.py`). Control: the same test against `main` without this change fails at the callback on the second replica with `signin_error=state_mismatch`, and its state cookie is the bare 32-byte state. With this change the cookie is 407 bytes.
jcoffey-dev merged commit 4fe234c2ea into main 2026-11-18 21:59:21 +00:00
jcoffey-dev deleted branch fix/signin-across-replicas 2026-11-18 21:59:21 +00:00
Sign in to join this conversation.