Specs: how to run the eight compat tests on a copy of INBUXA's data, and the statuses of tests 10, 11, 17 to 19
docs/spec/compat-tests.md lists each compat test, what it needs, what it checks and what a failure means, and says plainly that they run against a copy only, since the monitoring one purges the history it reads. The scale-out and per-domain statuses record the acceptance tests that now run.
This commit is contained in:
@@ -551,18 +551,18 @@ marked `inbuxa:`.
|
||||
- **Tests.** `directory::per_domain::per_domain_directory_tests` covers
|
||||
tests 1, 3, 4, 6, 7, 9, 11, 15 and 19, and DIR-20 and DIR-21, over SQL
|
||||
directories on SQLite files, with no container. `directory_tests` runs a
|
||||
new `oidc` module in place of the removed one, against Keycloak: tests 5,
|
||||
8, 12, 13, 14, 16 and 17 in part (below). SCIM's acceptance test 5
|
||||
new `oidc` module in place of the removed one, against Keycloak, whose
|
||||
container now imports a second realm: tests 5, 8, 10, 12, 13, 14, 16, 17
|
||||
and 18, the last two in part (below). SCIM's acceptance test 5
|
||||
(`scim_oidc_tests`) now runs and passes.
|
||||
- **Test 20 (compat)** is `per_domain_directory_compat`, ignored, and unrun
|
||||
until a copy of INBUXA's data is provided. It checks observed 1.
|
||||
- **Not exercised, or only in part:**
|
||||
- Test 2 and test 9 use an SQL directory that can't open instead of a
|
||||
stopped LDAP server, and test 18 (a stopped provider, and the sign-in
|
||||
ban) isn't run: the Keycloak container is shared.
|
||||
- Test 10 needs a second provider, and test 12's later sign-ins (an empty
|
||||
groups claim clearing groups, a missing one keeping them) need changes
|
||||
to Keycloak users; neither is run.
|
||||
stopped LDAP server.
|
||||
- Test 12's later sign-ins (an empty groups claim clearing groups, a
|
||||
missing one keeping them) need changes to Keycloak users, and aren't
|
||||
run.
|
||||
- Test 13 is checked through synchronization itself, since the realm's
|
||||
users aren't on the tenant's domain. Test 14 reuses an account an
|
||||
administrator made, not one from an earlier LDAP directory.
|
||||
@@ -570,7 +570,10 @@ marked `inbuxa:`.
|
||||
token. Keycloak grants every required scope whatever is asked, so the
|
||||
missing-scope refusal isn't reached; audience and key rotation aren't
|
||||
run. Test 17 checks password sign-in and a malformed token; an opaque
|
||||
token and `usernameDomain` aren't run.
|
||||
token that the provider accepts, and `usernameDomain`, aren't run.
|
||||
Test 18 stops the provider and checks an outage doesn't ban the
|
||||
client, and that bad tokens do; it doesn't measure the failure's
|
||||
latency.
|
||||
- DIR-22's rule that `Authentication.directoryId` names a server-level
|
||||
directory, and DIR-24 (a tenant administrator setting its own domains'
|
||||
directory), aren't tested.
|
||||
@@ -583,6 +586,9 @@ marked `inbuxa:`.
|
||||
reload from applying.
|
||||
- A server default naming no directory is unavailable, like a domain's
|
||||
(DIR-5); it used to mean the internal directory.
|
||||
- A token the OIDC directory refuses is an authentication failure, so it
|
||||
counts toward the sign-in ban (DIR-30); it used to be an error, which
|
||||
counts toward nothing.
|
||||
- **Known limits, not requirements of this spec:**
|
||||
- An SQL directory on a SQLite path that can't be opened holds the reload,
|
||||
and the request that caused it, for the pool's 30-second connection
|
||||
|
||||
@@ -553,18 +553,19 @@ Decision).
|
||||
treats as two members. `store::replica::replica_tests`, built with
|
||||
`postgres` and run with `STORE=PostgreSqlReplicated`, runs a primary and a
|
||||
streaming hot standby in containers and covers tests 9, 10, 12, 13, 14
|
||||
and 15. Test 1 is the existing store, blob and protocol suites passing
|
||||
unchanged.
|
||||
and 15. `replica_cluster_tests` (with `redis`) covers test 11, and
|
||||
`replica_mysql` covers tests 17 to 19 against two MySQL pairs, one
|
||||
replicating with GTIDs and one by binary log position. Test 1 is the
|
||||
existing store, blob and protocol suites passing unchanged.
|
||||
- **Not exercised:** test 3's downloads over JMAP and IMAP after a restart
|
||||
(the same blob reads are checked at the store), test 5's queued delivery
|
||||
(the failing write is checked), test 21 (one of two Redis servers stopped),
|
||||
the `resetRateLimiters` and `removeLock*` maintenance types (the store
|
||||
operations they use are checked), test 11 (two nodes), test 16 (the
|
||||
primary stopped), and tests 17 to 19: the MySQL code (GTID and
|
||||
`Seconds_Behind_Source` lag, the read-only and commit-order checks) is
|
||||
built but hasn't run against a MySQL replica. Test 9 checks that the
|
||||
replica served the reads, not the replica's statement log, and test 15
|
||||
checks full-text search, not a SQL directory.
|
||||
operations they use are checked), and test 16 (the primary stopped).
|
||||
Test 9 checks that the replica served the reads, not the replica's
|
||||
statement log, and test 15 checks full-text search, not a SQL directory.
|
||||
Test 11 uses one server and a second replicated store with its own
|
||||
marks, rather than two whole nodes.
|
||||
- **Settled from the code, not a change of intent:**
|
||||
- The FileSystem backend reports any unreadable file as missing, so a
|
||||
FileSystem member that can't be read looks like a miss (ST-17's search
|
||||
@@ -590,8 +591,9 @@ Decision).
|
||||
- Placement is the fork's own (ST-16), so an install coming from a
|
||||
sharded upstream deployment reads through ST-17's search (open
|
||||
question).
|
||||
- ST-7's step 4 raises the mark from a JMAP `sinceState` only; IMAP
|
||||
`CONDSTORE` and `QRESYNC` values and push resumption don't yet.
|
||||
- ST-7's step 4 raises the mark from a JMAP `sinceState` and an IMAP
|
||||
`FETCH ... CHANGEDSINCE`; a JMAP `queryChanges` state, `QRESYNC` on
|
||||
SELECT and push resumption don't yet.
|
||||
- The store's `enterprise` Cargo feature stays: other crates' feature
|
||||
lists name it.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user