Close the last tenant-isolation adversarial probe (mid-provisioning tenants)

Phase 4 task 8's verification plan named four adversarial probes;
three were closed earlier this phase, the fourth (an evaluator tick,
or any other caller, hitting a tenant that exists but hasn't reached
the active+credentialed gate yet -- must be refused, not served) was
still an explicitly-skipped stub in
api/queryapi/tenant_isolation_gap_test.go.

Investigating it found the two storage engines needed genuinely
different treatment:

- ClickHouse (enterprise/internal/chrunner) already had this property
  structurally, for free: Registry is built once at startup from
  rbacstore.ListProvisionedDataSources, which already filters to
  active+credentialed tenants only, so a mid-provisioning tenant is
  simply absent from the connection map. New test
  TestRegistryRefusesMidProvisioningTenant proves this without Docker
  -- an empty DataSource list never dials ClickHouse, so this genuinely
  runs in this environment, unlike every other test in that file.

- Tantivy (search/src/registry.rs's IndexRegistry) was a real, different
  gap, not just an unverified assumption: it opens-or-creates an index
  for any syntactically-valid tenant_id on first request, because it's
  a separate process with no Postgres access and structurally can't
  know which tenants are actually provisioned. A query against a
  mid-provisioning tenant would have silently succeeded with zero
  results from a freshly-created empty index -- "ambient success"
  indistinguishable from "no matching logs," exactly the failure mode
  this item was worried about.

Fixed the Tantivy gap with a new enterprise/internal/searchclient.
TenantChecker interface (backed by a new rbacstore.TenantIsActive,
implemented structurally, no new import edge needed), consulted before
every gRPC call: Client.Search now refuses a non-active tenant before
it ever reaches `search`. Dial's signature gained a required
TenantChecker parameter; enterprise-api's main.go passes its existing
rbacstore.Store (already satisfies the interface). Verified Docker-free
via searchclient's existing real-in-process-gRPC-server test harness
(TestSearchRefusesMidProvisioningTenant, plus
TestSearchPropagatesTenantCheckerError for the fail-closed-on-error
case) -- both genuinely run in this environment, same bar as the rest
of the Tantivy isolation work.

rbacstore.TenantIsActive itself has two new skip-gated live-Postgres
tests (TestTenantIsActive, TestTenantIsActiveNonexistentTenant) --
disclosed as not run against a live database here, same gap as the
rest of this phase's Postgres-backed pieces.

api/queryapi/tenant_isolation_gap_test.go rewritten from a checklist
with one skipped stub to a full accounting of all four now-closed
probes. Docs updated in lockstep: CLAUDE.md, threat-model.md,
phase-4-isolation-design.md (implementation note added after its
original sign-off), phase-4-runbook.md (§9), enterprise/README.md.
This commit is contained in:
2026-08-14 08:07:52 -07:00
parent b8b6a8fd7b
commit 2e698f5623
12 changed files with 345 additions and 51 deletions
+22
View File
@@ -315,3 +315,25 @@ suite must include, against the live stack:
- A simulated evaluator tick firing mid-provisioning (tenant row exists,
grants not yet confirmed) to confirm it's refused, not silently served
against a partially-provisioned or default-profile connection.
**Implementation note (Phase 4 task 8, added after this design was
signed off):** closed for both storage engines, via
`api/queryapi/tenant_isolation_gap_test.go`'s pointers. ClickHouse's
refusal turned out to be structural, not something that needed new
code: `chrunner.Registry` is built once at startup from
`rbacstore.ListProvisionedDataSources`, which already excludes
anything short of active+credentialed, so a mid-provisioning tenant is
simply absent from the connection map — proven Docker-free
(`chrunner_test.go`'s `TestRegistryRefusesMidProvisioningTenant`,
since an empty `DataSource` list never dials ClickHouse). Tantivy was
a real, different gap: `search/src/registry.rs`'s `IndexRegistry`
opens-or-creates an index for *any* syntactically-valid `tenant_id` on
first request, because it's a separate process with no Postgres
access and structurally cannot know which tenants are provisioned --
without a check upstream, a mid-provisioning tenant's query would have
silently succeeded with zero results from a freshly-created empty
index. Fixed by adding `enterprise/internal/searchclient.
TenantChecker` (backed by a new `rbacstore.TenantIsActive`): `Client.
Search` now refuses before the gRPC call goes out if the tenant isn't
active, verified Docker-free via a real in-process gRPC server
(`searchclient_test.go`'s `TestSearchRefusesMidProvisioningTenant`).
+33 -9
View File
@@ -386,14 +386,27 @@ cargo test
cd ../enterprise
go test ./internal/searchclient/... -v
# real in-process gRPC server, confirms SearchRequest.tenant_id is set
# correctly and that a request with no/invalid tenant identity is refused.
# correctly, that a request with no/invalid tenant identity is refused,
# and (since TenantChecker was added to close verification-plan item 4)
# that a tenant which exists but isn't active yet -- e.g. right after
# enterprise-auth -create-tenant but before enterprise-api
# -provision-tenant -- is refused too, not silently searched against a
# freshly-created empty index.
go test ./internal/chrunner/... -run MidProvisioning -v
# the ClickHouse half of the same item-4 probe -- also Docker-free,
# since an empty DataSource list never dials out.
```
This is the one piece of Phase 4's tenant isolation work that has
**actually been run and confirmed passing** in an environment without
Docker access, alongside `enterprise/internal/loginhandler`'s OIDC
tests (§3a) — both are unusually strong evidence precisely because they
needed no infrastructure this environment lacked.
Docker access, alongside `enterprise/internal/loginhandler`'s OIDC/SAML
tests (§3a/§3b) — both are unusually strong evidence precisely because
they needed no infrastructure this environment lacked. Writing the
`TenantChecker` test here is also what found the Tantivy mid-
provisioning gap in the first place, not just what closed it after the
fact -- see `api/queryapi/tenant_isolation_gap_test.go`'s item 4 for the
full story.
## 10. Confirm the Helm chart actually enforces the binary swap
@@ -466,11 +479,22 @@ Full accounting: `/docs/security/threat-model.md`. Headline items:
(`enterprise/internal/rbacstore/rbacstore_test.go`) haven't run
against a live database in this environment, same gap as the rest of
this phase's Postgres-backed pieces.
- Three of the four adversarial ClickHouse/Tantivy probes named in
`/docs/phase-4-isolation-design.md`'s verification plan are closed
(§8, §9); the last (mid-provisioning-race handling) is still stubbed
as an explicitly-skipped test in
`api/queryapi/tenant_isolation_gap_test.go`.
- All four adversarial ClickHouse/Tantivy probes named in
`/docs/phase-4-isolation-design.md`'s verification plan are now closed
-- see `api/queryapi/tenant_isolation_gap_test.go` for the full
accounting. The fourth (mid-provisioning-race handling) turned out to
need a real code fix on the Tantivy side, not just a test: `search/
src/registry.rs`'s `IndexRegistry` opened-or-created an index for any
syntactically-valid `tenant_id`, so a mid-provisioning tenant's query
would have silently succeeded with zero results instead of being
refused. Fixed via `enterprise/internal/searchclient.TenantChecker`
(backed by `rbacstore.TenantIsActive`). Both the ClickHouse and
Tantivy halves of this probe run genuinely, without Docker, in this
environment (`chrunner_test.go`'s
`TestRegistryRefusesMidProvisioningTenant`, `searchclient_test.go`'s
`TestSearchRefusesMidProvisioningTenant`) -- `rbacstore.TenantIsActive`
itself has skip-gated live-Postgres tests that haven't run here, same
disclosed gap as the rest of this phase's Postgres-backed pieces.
## Tearing down
+1 -1
View File
@@ -412,5 +412,5 @@ terms:
| Query audit logging (routine queries) | **Enforced**, fail-open, and now wired to a real writer via `enterprise-api` (`audit.QueryAPILogger`) |
| Audit log tamper detection (hash chain) | **Enforced**, verified live |
| Audit log tamper prevention (external anchoring) | **Design only**`FileSink` is a dev stand-in |
| Mid-provisioning-race handling (evaluator ticks against a not-yet-active tenant) | **Unverified** — see `api/queryapi/tenant_isolation_gap_test.go` |
| Mid-provisioning-race handling (evaluator ticks against a not-yet-active tenant) | **Closed on both storage engines** — see `api/queryapi/tenant_isolation_gap_test.go`; ClickHouse verified Docker-free (structural, not just tested), Tantivy fixed and verified Docker-free after finding it was a real gap, not just an unverified assumption |
| Protection against a privileged DB administrator | **Explicit non-goal** |