Phase 4 task 8's verification plan named four adversarial probes; three were closed earlier this phase, the fourth (an evaluator tick, or any other caller, hitting a tenant that exists but hasn't reached the active+credentialed gate yet -- must be refused, not served) was still an explicitly-skipped stub in api/queryapi/tenant_isolation_gap_test.go. Investigating it found the two storage engines needed genuinely different treatment: - ClickHouse (enterprise/internal/chrunner) already had this property structurally, for free: Registry is built once at startup from rbacstore.ListProvisionedDataSources, which already filters to active+credentialed tenants only, so a mid-provisioning tenant is simply absent from the connection map. New test TestRegistryRefusesMidProvisioningTenant proves this without Docker -- an empty DataSource list never dials ClickHouse, so this genuinely runs in this environment, unlike every other test in that file. - Tantivy (search/src/registry.rs's IndexRegistry) was a real, different gap, not just an unverified assumption: it opens-or-creates an index for any syntactically-valid tenant_id on first request, because it's a separate process with no Postgres access and structurally can't know which tenants are actually provisioned. A query against a mid-provisioning tenant would have silently succeeded with zero results from a freshly-created empty index -- "ambient success" indistinguishable from "no matching logs," exactly the failure mode this item was worried about. Fixed the Tantivy gap with a new enterprise/internal/searchclient. TenantChecker interface (backed by a new rbacstore.TenantIsActive, implemented structurally, no new import edge needed), consulted before every gRPC call: Client.Search now refuses a non-active tenant before it ever reaches `search`. Dial's signature gained a required TenantChecker parameter; enterprise-api's main.go passes its existing rbacstore.Store (already satisfies the interface). Verified Docker-free via searchclient's existing real-in-process-gRPC-server test harness (TestSearchRefusesMidProvisioningTenant, plus TestSearchPropagatesTenantCheckerError for the fail-closed-on-error case) -- both genuinely run in this environment, same bar as the rest of the Tantivy isolation work. rbacstore.TenantIsActive itself has two new skip-gated live-Postgres tests (TestTenantIsActive, TestTenantIsActiveNonexistentTenant) -- disclosed as not run against a live database here, same gap as the rest of this phase's Postgres-backed pieces. api/queryapi/tenant_isolation_gap_test.go rewritten from a checklist with one skipped stub to a full accounting of all four now-closed probes. Docs updated in lockstep: CLAUDE.md, threat-model.md, phase-4-isolation-design.md (implementation note added after its original sign-off), phase-4-runbook.md (§9), enterprise/README.md.
enterprise
Commercial license, not AGPLv3 — see /CLAUDE.md's licensing
boundary. SSO (OIDC/SAML), tenant provisioning, and RBAC. Nothing in
/agent, /ingest, /storage, /api, /web core, or /cli imports
from this module — confirmed by hack/check-tenant-boundary.sh, run in
CI. enterprise/ supplies tenant-scoped implementations of core's
already-shipped api/querylang/executor.SQLRunner/
SearchClient interfaces rather than core growing tenant awareness —
see /docs/phase-4-isolation-design.md for why.
Status
What's built and wired end-to-end. Verification status varies by
piece -- internal/audit was confirmed against a real Postgres earlier
in this phase's work; everything else below has real integration tests
written the same way (skipped unless a live database's connection
details are supplied via env var, same pattern throughout this package)
but they have not actually been run against a live database in this
environment -- see /docs/phase-4-runbook.md's verification-status
section for exactly what "not yet run" means here and why. Don't read
"has a test for this" as "this was confirmed to work."
internal/sessionissues/validates signed (HS256/JWT) tokens for both human sessions and/alerting'sRoleServicecredential.internal/authhandlerservesPOST /internal/authorize(the endpointapi/authz.HTTPAuthorizercalls) andGET /auth/features(the runtime-capability check/web's settings page reads).api's/queryand/dashboardsendpoints enforce RBAC viaauthz.RequireRole/RequireRoleOrService, nil-safe (no-op) whenENTERPRISE_AUTH_URLisn't configured -- matches Phase 0-3 behavior./alerting'squeryclientpresents aRoleServiceBearer token (API_SERVICE_TOKEN) when configured -- see/docs/phase-4-isolation-design.md'salerting↔apigap.sentryctlpresents$SENTRYCTL_TOKENas a Bearer credential on every request when set.internal/rbacstore: full CRUD overusers/tenants/tenant_memberships/data_sources(metadata/migrations/0017-0032).internal/tenantprovision: realCREATE DATABASE/CREATE USER/GRANTagainst ClickHouse. Its tests assert a tenant A user cannot read tenant B's database by fully-qualified name, and thatsystem.query_log/system.tables/SHOW DATABASESdon't leak across tenants either (task 2's finding was that the latter is version-dependent) -- not yet run against a live ClickHouse in this environment, see the note above.internal/chrunner: the tenant-scopedSQLRunner-- a per-tenant connection registry that resolves which tenant's ClickHouse connection to use from the authenticated identity in request context, never a parameter. Same adversarial probe, now through the actual production code path (chrunner.Registry.RunSQL, not just tenantprovision's raw grants).internal/audit.QueryAPILogger: the realapi/queryapi.AuditLoggerimplementation -- wired intoenterprise-api, no longernil.internal/loginhandler:GET /auth/oidc/login+GET /auth/oidc/callback-- the actual human login flow, previously entirely missing. Redirects to the configured IdP with CSRF-protection state in a short-lived cookie, exchanges the code, verifies the ID token viainternal/oidc, upserts ausersrow, resolves tenant/role from exactly onetenant_membershipsrow (refuses with a clear error on zero or multiple -- no tenant-picker UI yet), and issues a session cookie. This one genuinely is verified, unlike the ClickHouse pieces above:loginhandler_test.goruns the full flow against a real fake IdP (coreos/go-oidc's ownoidctestpackage, real RS256 signing and verification, no live database or Docker needed) and every test passes. Not yet tried against a real external IdP or a runningenterprise-authcontainer.internal/searchclient: the Tantivy-side sibling ofchrunner-- implementsapi/querylang/executor.SearchClient, resolvingSearchRequest.tenant_id(new field,proto/sentry/search/v1/ search.proto) from the authenticated request identity, same fail-closed shape aschrunner.Registry.RunSQL. Paired withsearch/src/registry.rs'sIndexRegistry(Rust, opens a per-tenant Tantivy index on demand). Both genuinely verified -- unlike the ClickHouse pieces, Tantivy is an embedded library, so the isolation probe (three tenants, shared search term, scoped search returns only that tenant's document) actually ran:search'scargo test/cargo clippy --all-targets -- -D warningsand this package'sgo testboth pass clean, no Docker or live database needed for either.Clientalso carries aTenantChecker(backed byrbacstore.TenantIsActive) sincesearch/src/registry.rs'sIndexRegistryopens-or-creates an index for any syntactically-validtenant_id-- a real gap found while closing/docs/phase-4-isolation-design.md's verification-plan item 4: a mid-provisioning tenant would otherwise get a silently-empty search result instead of a refusal. Verified the same Docker-free way.cmd/enterprise-api: a second binary (alongsideapi/cmd/api, unchanged) importing bothapi's handler packages and the tenant-aware implementations above -- see its own doc comment for why this shape exists (enterprise → apiis the allowed import direction;apican never importenterprise/).-provision-tenant=<id>is the operator action that provisions ClickHouse and marks a tenant active, same "offline action, not a network endpoint" shape asenterprise-auth -mint-service-token.
OIDC and SAML login are both now fully wired: internal/loginhandler
serves GET /auth/oidc/login+GET /auth/oidc/callback and
GET /auth/saml/login+POST /auth/saml/acs, converging on the same
upsert-user/resolve-tenant/issue-session path. Both are verified the
same way -- a real fake IdP with genuine cryptographic signing and
verification (coreos/go-oidc's oidctest for OIDC,
crewjam/saml/samlidp for SAML), no Docker needed, every test in
loginhandler_test.go/saml_test.go passing including the full login
round trip and negative paths (bad state/InResponseTo, expired/missing
credential, no/multiple tenant memberships). Writing the SAML test
caught two real bugs in internal/saml.ParseResponse, both fixed:
missing r.ParseForm() before reading the POSTed SAMLResponse field,
and email-attribute matching that missed the standard LDAP "mail" OID
(urn:oid:0.9.2342.19200300.100.1.3) that IdPs send by default absent
an explicit AttributeConsumingService request for "email" -- exactly
what samlidp's own default assertion builder does. Neither protocol
has been tried against a real external IdP or a running
enterprise-auth container -- see /docs/phase-4-runbook.md §3a/§3b.
dashboard_permissions is now wired end to end: rbacstore/ dashboard_permissions.go is the raw CRUD, rbacstore/ dashboards_adapter.go's DashboardPermissions implements
api/dashboards.PermissionStore (the interface core defines and
carries as a nil-by-default field, same shape as
queryapi.AuditLogger), and enterprise-api's main.go wires it in.
api/dashboards' handler now enforces the matrix's "(own/granted)"
qualifier: an Editor may edit/delete a dashboard (or its panels) they
created, or one where a grant raises their effective role to Editor;
managing grants themselves is stricter still -- creator or Admin/Owner
only, closing a self-escalation path where a granted-but-not-creator
Editor could otherwise extend their own access. metadata/migrations/ 0033_restrict_dashboard_permissions_role.sql fixes a divergence
between 0024's actual CHECK constraint (allowed role='admin', nullable
granted_by) and the design doc's schema (viewer/editor only,
granted_by required) found while wiring this up. Verified against a
fake PermissionStore in api/dashboards/handler_test.go (the full
own/granted/admin/creator matrix, including the granted-editor-cannot-
manage-grants regression); real integration tests exist in
rbacstore_test.go but haven't run against a live Postgres in this
environment, same disclosed gap as the rest of this package's
Postgres-backed pieces.
Deliberately deferred, not half-built -- named explicitly rather than silently left out:
- A tenant-picker UI/flow for an identity with more than one
tenant_membershipsrow --loginhandlerrefuses these logins outright rather than guessing (ErrMultipleMemberships). - Ingest tenant-awareness, for either storage engine --
chrunner/searchclientprove read isolation given tenant-scoped data exists, but nothing writes it: every recordingestproduces still lands in the single shared ClickHouse database and the single shared Tantivy index. A newly-provisioned tenant's storage is real and isolated, and permanently empty. Undesigned, not just unbuilt -- see/docs/security/threat-model.md. - Any deployment-topology mechanism that actually routes traffic to
enterprise-apiinstead ofapi-- both binaries exist,docker-compose.ymlincludesenterprise-apiavailable but not wired intoweb's default base URL, and the Helm chart has no service for it at all yet. This is now the single largest gap -- both storage engines' isolation mechanisms themselves are built.
Package layout
cmd/enterprise-auth/ config loading, OIDC discovery at startup, health/authorize/features endpoints, -mint-service-token, -create-tenant, -grant-membership-*
cmd/enterprise-api/ multi-tenant-aware alternative to api/cmd/api -- see its own doc comment
internal/tenant/ the ID type -- see its package doc comment before touching it
internal/oidc/ coreos/go-oidc wiring: discovery, login redirect, code exchange + ID token verification
internal/saml/ crewjam/saml wiring: SP setup, login redirect, response parsing/validation
internal/session/ issues/validates signed session + RoleService tokens
internal/authhandler/ POST /internal/authorize, GET /auth/features
internal/loginhandler/ GET /auth/oidc/{login,callback} + GET /auth/saml/login + POST /auth/saml/acs -- the human login flow
internal/rbacstore/ users/tenants/tenant_memberships/data_sources/dashboard_permissions CRUD (pgx against sentry_metadata)
internal/tenantprovision/ real ClickHouse CREATE DATABASE/USER/GRANT
internal/chrunner/ tenant-scoped api/querylang/executor.SQLRunner
internal/searchclient/ tenant-scoped api/querylang/executor.SearchClient
internal/audit/ append-only, hash-chained query audit log, plus the
api/queryapi.AuditLogger adapter (queryapi_adapter.go)
internal/apiconfig/ enterprise-api's own env-var config
internal/config/ enterprise-auth's env-var config
Future additions: ingest tenant-awareness (undesigned), and real
deployment-topology wiring for enterprise-api -- see "Status" above.
Why OIDC and SAML aren't hand-rolled
coreos/go-oidc (built on golang.org/x/oauth2) and crewjam/saml
handle token/assertion signature verification, XML signing, and the
protocol-level trust establishment — exactly the parts of an SSO
integration where a from-scratch implementation is the highest-risk
code in the whole feature. Both are well-established libraries, matching
this project's existing "boring, well-understood dependency" pattern
(clickhouse-go/v2, jackc/pgx/v5).
Building & testing
go build ./...
go vet ./...
go test ./...
internal/audit's real guarantees (the audit_writer grant
restriction, the immutability trigger, hash-chain correctness under
concurrency) can only be proven against a real Postgres — those
integration tests are skipped by default and only run with
AUDIT_TEST_POSTGRES_ADDR set:
docker run --rm --network sentry_default -v $(pwd)/..:/src -w /src/enterprise \
-e AUDIT_TEST_POSTGRES_ADDR=metadata-postgres:5432 \
-e AUDIT_TEST_POSTGRES_PASSWORD=audit-writer-dev-only \
-e AUDIT_TEST_ADMIN_PASSWORD=sentry-dev-only \
golang:1.25-alpine go test ./internal/audit/... -v
internal/rbacstore's tests are the same shape (real SQL, real
constraints), skipped unless RBACSTORE_TEST_POSTGRES_ADDR is set:
docker run --rm --network sentry_default -v $(pwd)/..:/src -w /src/enterprise \
-e RBACSTORE_TEST_POSTGRES_ADDR=metadata-postgres:5432 \
-e RBACSTORE_TEST_POSTGRES_PASSWORD=sentry-dev-only \
golang:1.25-alpine go test ./internal/rbacstore/... -v
internal/tenantprovision and internal/chrunner need a real
ClickHouse instead (they mount the repo root, not just enterprise/,
since internal/chrunner imports api/authz/api/querylang/executor
via go.mod's replace directives to ../api):
docker run --rm --network sentry_default -v $(pwd)/..:/src -w /src/enterprise \
-e TENANTPROVISION_TEST_CLICKHOUSE_ADDR=clickhouse:9000 \
-e TENANTPROVISION_TEST_CLICKHOUSE_PASSWORD=sentry-dev-only \
golang:1.25-alpine go test ./internal/tenantprovision/... -v
docker run --rm --network sentry_default -v $(pwd)/..:/src -w /src/enterprise \
-e CHRUNNER_TEST_CLICKHOUSE_ADDR=clickhouse:9000 \
-e CHRUNNER_TEST_CLICKHOUSE_PASSWORD=sentry-dev-only \
golang:1.25-alpine go test ./internal/chrunner/... -v
Turning on auth enforcement for manual testing
Off by default (see "Status" above). To exercise the RoleService path
end to end:
docker compose up -d enterprise-auth
TOKEN=$(docker compose run --rm enterprise-auth -mint-service-token=alerting)
# api: set ENTERPRISE_AUTH_URL=http://enterprise-auth:8082 and restart
# alerting: set API_SERVICE_TOKEN=$TOKEN and restart
docker build -f Dockerfile -t sentry-enterprise-auth . # context is enterprise/, not the repo root
Bootstrapping a tenant and its first human user
-create-tenant/-grant-membership-* are enterprise-auth operator
flags, same "offline action gated by access to enterprise-auth's own
environment, not a network-reachable endpoint" shape as
-mint-service-token -- deliberately not an authenticated HTTP admin
API, which would have to solve "who's allowed to create the very first
tenant/membership" itself. Replaces what used to be a manual psql
dance (see /docs/phase-4-runbook.md §3a's history if you're wondering
why old references to it still show up in git blame):
docker compose run --rm enterprise-auth -create-tenant=acme -display-name="Acme Corp"
# Log in once via /auth/oidc/login or /auth/saml/login -- it fails with
# "no tenant membership" (403), but UpsertUserBySSO already created the
# users row by then, which -grant-membership-user-email needs.
docker compose run --rm enterprise-auth \
-grant-membership-tenant=acme -grant-membership-user-email=[email protected] -grant-membership-role=owner
-create-tenant only touches rbacstore -- pair with enterprise-api -provision-tenant (below) for a tenant to actually be able to run
queries, not just log in. role=owner also calls SetOwner, since a
tenant's Owner is a dedicated tenants.owner_user_id column, not just
the highest tenant_memberships role. Not yet built: revoking a
membership, listing a tenant's members, or a flag for
dashboard_permissions grants (those go through the HTTP endpoints
api/dashboards' handler now exposes -- PUT/DELETE /dashboards/{id}/permissions/{userId}, GET .../permissions).
Provisioning a tenant and running enterprise-api
docker compose build enterprise-api # context is the repo root, not enterprise/ -- see cmd/enterprise-api/Dockerfile
docker compose run --rm enterprise-api -provision-tenant=acme -display-name="Acme Corp"
docker compose up -d enterprise-api
curl -s http://localhost:8083/healthz
-provision-tenant creates the tenant/data_source rows in rbacstore if
they don't exist, provisions ClickHouse, persists the credentials, and
marks the tenant active -- refuses to run twice for the same tenant
(re-provisioning would either rotate a live credential or silently fail
to, see tenantprovision.ProvisionClickHouse's doc comment). web
still points at plain api by default (VITE_API_BASE_URL) --
pointing it at enterprise-api instead is a manual docker-compose.yml
edit today, not a supported flag.
Environment variables (enterprise-auth)
| Var | Default |
|---|---|
HTTP_LISTEN_ADDR |
:8082 |
POSTGRES_ADDR |
localhost:5432 |
POSTGRES_DATABASE |
sentry_metadata |
POSTGRES_USERNAME |
sentry |
POSTGRES_PASSWORD |
(empty) |
OIDC_ISSUER_URL |
(empty — OIDC discovery skipped if unset) |
OIDC_CLIENT_ID |
(empty) |
OIDC_CLIENT_SECRET |
(empty) |
OIDC_REDIRECT_URL |
(empty — must be <enterprise-auth base URL>/auth/oidc/callback, registered with the IdP) |
SAML_ENTITY_ID |
(empty) |
SAML_ACS_URL |
(empty) |
SAML_IDP_METADATA_URL |
(empty — SAML disabled if unset; if set, fetched and parsed at startup via samlsp.FetchMetadata, same trust level as OIDC_ISSUER_URL's discovery fetch) |
ENTERPRISE_SESSION_SIGNING_KEY |
required, min 32 bytes |
POST_LOGIN_REDIRECT_URL |
http://localhost:3000 — where the browser lands after internal/loginhandler sets a session cookie |
Environment variables (enterprise-api)
| Var | Default |
|---|---|
HTTP_LISTEN_ADDR |
:8083 |
CLICKHOUSE_ADDR |
localhost:9000 |
CLICKHOUSE_ADMIN_USERNAME |
default |
CLICKHOUSE_ADMIN_PASSWORD |
(empty) |
SEARCH_GRPC_ADDR |
localhost:50052 |
POSTGRES_ADDR |
localhost:5432 |
POSTGRES_DATABASE |
sentry_metadata |
POSTGRES_USERNAME |
sentry |
POSTGRES_PASSWORD |
(empty) |
AUDIT_WRITER_USERNAME |
audit_writer |
AUDIT_WRITER_PASSWORD |
(empty) |
ENTERPRISE_AUTH_URL |
(empty — RBAC becomes a no-op, but chrunner.Registry.RunSQL still refuses every query with no resolved tenant identity, so leaving this unset does not mean "open access," it means "every query fails") |
CORS_ALLOWED_ORIGIN |
* |
QUERY_TIMEOUT_SECONDS |
30 |