Specs: how to run the eight compat tests on a copy of INBUXA's data, and the statuses of tests 10, 11, 17 to 19

docs/spec/compat-tests.md lists each compat test, what it needs, what it
checks and what a failure means, and says plainly that they run against a
copy only, since the monitoring one purges the history it reads. The
scale-out and per-domain statuses record the acceptance tests that now
run.
This commit is contained in:
2026-09-19 15:11:04 -07:00
parent c0377df942
commit cee4fd6bc6
3 changed files with 96 additions and 18 deletions
+14 -8
View File
@@ -551,18 +551,18 @@ marked `inbuxa:`.
- **Tests.** `directory::per_domain::per_domain_directory_tests` covers
tests 1, 3, 4, 6, 7, 9, 11, 15 and 19, and DIR-20 and DIR-21, over SQL
directories on SQLite files, with no container. `directory_tests` runs a
new `oidc` module in place of the removed one, against Keycloak: tests 5,
8, 12, 13, 14, 16 and 17 in part (below). SCIM's acceptance test 5
new `oidc` module in place of the removed one, against Keycloak, whose
container now imports a second realm: tests 5, 8, 10, 12, 13, 14, 16, 17
and 18, the last two in part (below). SCIM's acceptance test 5
(`scim_oidc_tests`) now runs and passes.
- **Test 20 (compat)** is `per_domain_directory_compat`, ignored, and unrun
until a copy of INBUXA's data is provided. It checks observed 1.
- **Not exercised, or only in part:**
- Test 2 and test 9 use an SQL directory that can't open instead of a
stopped LDAP server, and test 18 (a stopped provider, and the sign-in
ban) isn't run: the Keycloak container is shared.
- Test 10 needs a second provider, and test 12's later sign-ins (an empty
groups claim clearing groups, a missing one keeping them) need changes
to Keycloak users; neither is run.
stopped LDAP server.
- Test 12's later sign-ins (an empty groups claim clearing groups, a
missing one keeping them) need changes to Keycloak users, and aren't
run.
- Test 13 is checked through synchronization itself, since the realm's
users aren't on the tenant's domain. Test 14 reuses an account an
administrator made, not one from an earlier LDAP directory.
@@ -570,7 +570,10 @@ marked `inbuxa:`.
token. Keycloak grants every required scope whatever is asked, so the
missing-scope refusal isn't reached; audience and key rotation aren't
run. Test 17 checks password sign-in and a malformed token; an opaque
token and `usernameDomain` aren't run.
token that the provider accepts, and `usernameDomain`, aren't run.
Test 18 stops the provider and checks an outage doesn't ban the
client, and that bad tokens do; it doesn't measure the failure's
latency.
- DIR-22's rule that `Authentication.directoryId` names a server-level
directory, and DIR-24 (a tenant administrator setting its own domains'
directory), aren't tested.
@@ -583,6 +586,9 @@ marked `inbuxa:`.
reload from applying.
- A server default naming no directory is unavailable, like a domain's
(DIR-5); it used to mean the internal directory.
- A token the OIDC directory refuses is an authentication failure, so it
counts toward the sign-in ban (DIR-30); it used to be an error, which
counts toward nothing.
- **Known limits, not requirements of this spec:**
- An SQL directory on a SQLite path that can't be opened holds the reload,
and the request that caused it, for the pool's 30-second connection
+12 -10
View File
@@ -553,18 +553,19 @@ Decision).
treats as two members. `store::replica::replica_tests`, built with
`postgres` and run with `STORE=PostgreSqlReplicated`, runs a primary and a
streaming hot standby in containers and covers tests 9, 10, 12, 13, 14
and 15. Test 1 is the existing store, blob and protocol suites passing
unchanged.
and 15. `replica_cluster_tests` (with `redis`) covers test 11, and
`replica_mysql` covers tests 17 to 19 against two MySQL pairs, one
replicating with GTIDs and one by binary log position. Test 1 is the
existing store, blob and protocol suites passing unchanged.
- **Not exercised:** test 3's downloads over JMAP and IMAP after a restart
(the same blob reads are checked at the store), test 5's queued delivery
(the failing write is checked), test 21 (one of two Redis servers stopped),
the `resetRateLimiters` and `removeLock*` maintenance types (the store
operations they use are checked), test 11 (two nodes), test 16 (the
primary stopped), and tests 17 to 19: the MySQL code (GTID and
`Seconds_Behind_Source` lag, the read-only and commit-order checks) is
built but hasn't run against a MySQL replica. Test 9 checks that the
replica served the reads, not the replica's statement log, and test 15
checks full-text search, not a SQL directory.
operations they use are checked), and test 16 (the primary stopped).
Test 9 checks that the replica served the reads, not the replica's
statement log, and test 15 checks full-text search, not a SQL directory.
Test 11 uses one server and a second replicated store with its own
marks, rather than two whole nodes.
- **Settled from the code, not a change of intent:**
- The FileSystem backend reports any unreadable file as missing, so a
FileSystem member that can't be read looks like a miss (ST-17's search
@@ -590,8 +591,9 @@ Decision).
- Placement is the fork's own (ST-16), so an install coming from a
sharded upstream deployment reads through ST-17's search (open
question).
- ST-7's step 4 raises the mark from a JMAP `sinceState` only; IMAP
`CONDSTORE` and `QRESYNC` values and push resumption don't yet.
- ST-7's step 4 raises the mark from a JMAP `sinceState` and an IMAP
`FETCH ... CHANGEDSINCE`; a JMAP `queryChanges` state, `QRESYNC` on
SELECT and push resumption don't yet.
- The store's `enterprise` Cargo feature stays: other crates' feature
lists name it.