Close search's active-tenant write-routing gap with a polled allowlist

search/src/consumer.rs's write-routing (built last pass) had no active-
tenant check at all: IndexRegistry.resolve() would open-or-create an
index directory for any syntactically-valid tenant_id, active or not --
unlike ClickHouse's chwriter.Registry (an active-tenants-only snapshot
built at enterprise-ingest startup) or the read side (gated by
searchclient.TenantChecker, a direct rbacstore query). search is AGPL
core with no Postgres access and no enterprise/ import allowed, so it
needed a network boundary instead -- the same shape ingest's
TenantResolver already uses against enterprise-auth, just Rust calling
Go instead of Go calling Go.

New GET /internal/active-tenants endpoint on enterprise-auth
(rbacstore.ListActiveTenantIDs + authhandler.handleActiveTenants),
gated on a RoleService Bearer credential -- server-to-server auth, the
same shape alerting presents to api, minted via the already-generic
enterprise-auth -mint-service-token search. search/src/tenants.rs's
ActiveTenantTracker polls it every 60s, blocking startup on the first
fetch succeeding (fail-closed cold start -- a control-plane outage at
boot must not silently accept every tenant_id) and keeping the last-
known-good set on any later refresh failure (a transient blip shouldn't
stop every tenant's indexing, only prevent the allowlist from growing/
shrinking until connectivity resumes). consumer.rs refuses any tagged
record whose tenant isn't in the polled set, before ever calling
resolve() -- IndexRegistry itself stays policy-free, matching the same
mechanism/policy split clickhousewriter.Writer vs. chwriter.Registry
already draws on the ClickHouse side.

Off unless ENTERPRISE_AUTH_URL/ENTERPRISE_AUTH_SERVICE_TOKEN are both
set (search/src/config.rs rejects exactly one being set) -- every
existing deployment is unaffected.

Verified with real HTTP round trips in this environment: tenants.rs's
tests exercise real reqwest requests (actual Authorization: Bearer
header, actual JSON parsing) against a hand-rolled dependency-free TCP
test server, including both fail-closed paths (rejected first fetch,
unreachable server). authhandler's new tests cover the credential-kind
distinction this endpoint exists to enforce -- a real human session,
even for a genuine Owner, must not satisfy a check meant for a service
identity.

One asymmetry remains, disclosed rather than fixed: chwriter.Registry's
snapshot still never refreshes (stale until enterprise-ingest restarts),
while ActiveTenantTracker's 60s poll gives Tantivy a materially tighter
staleness window. Neither is a live per-write check -- that would mean
a database/HTTP round trip per record, a throughput cost neither
implementation accepts -- so both have some staleness window by design;
the gap between the two windows is what's disclosed, not a claim either
is fully live.

search/README.md

@@ -62,19 +62,28 @@ binary" needed. The periodic Tantivy commit (`COMMIT_INTERVAL_MS`) now
 commits every tenant index that's actually seen a write, plus the
 default index, via `IndexRegistry::commit_all`, not just one index.
 
-**One residual gap, disclosed rather than fixed here**: unlike the read
-side (gated by `enterprise/internal/searchclient`'s `TenantChecker`,
-which refuses to search a tenant that isn't `active` in `rbacstore`) and
-unlike ClickHouse's write side (`chwriter.Registry`, built from an
-active-tenants-only snapshot at startup, so an unrecognized tenant has
-no writer at all), this consumer's `registry.resolve()` call has no
-active-tenant gate — this process has no Postgres access to check
-against, the same reason `IndexRegistry` couldn't do the mid-provisioning
-check itself before `TenantChecker` was added for the read side. A
-still-valid (not yet revoked) ingest credential for a tenant that's no
-longer active can cause an index directory to be created for it here.
-See `src/registry.rs`'s doc comment on `resolve` for the full writeup,
-including why closing it fully isn't scoped yet.
+**The active-tenant gap is closed too, the same way the read side closes
+it**: `src/tenants.rs`'s `ActiveTenantTracker` polls a new
+`GET /internal/active-tenants` endpoint on `enterprise-auth` (this
+process has no Postgres access, so unlike `enterprise/internal/
+searchclient`'s `TenantChecker` — a direct `rbacstore.TenantIsActive`
+call, since that code runs in `enterprise/` — this needed a network
+call instead), and `consumer.rs` refuses (logs and skips, never falls
+back to the default or another tenant's index) any tagged record whose
+`tenant_id` isn't in the polled allowlist. Off unless
+`ENTERPRISE_AUTH_URL`/`ENTERPRISE_AUTH_SERVICE_TOKEN` are both set (see
+Configuration below) — when they aren't, write-routing behaves exactly
+as it did before this tracker existed, trusting any syntactically-valid
+`tenant_id`. When they are, startup blocks on the first fetch succeeding
+(fail-closed cold start — see `tenants.rs`'s doc comment for why a
+partial/degraded startup isn't the safer choice, and for the
+last-known-good behavior periodic refresh failures fall back to).
+Verified with real HTTP round trips against a hand-rolled TCP test
+server (no mocking crate needed for one endpoint) — the Bearer token
+actually sent, the initial-fetch-fails-closed path, and an unreachable
+server also failing closed. See `src/registry.rs`'s doc comment on
+`resolve` for how the mechanism (index lifecycle) and policy (who gets
+gated) responsibilities are split.
 
 ## Offset tracking: why this isn't a Kafka consumer group
 
@@ -118,6 +127,8 @@ Environment variables (see `src/config.rs`):
 | `TENANTS_INDEX_PATH` | `/var/lib/sentry-search/tenants` | Per-tenant index directories live under here, one subdirectory per tenant_id (Phase 4) |
 | `OFFSETS_PATH` | `/var/lib/sentry-search/offsets.json` | Offset tracking file |
 | `COMMIT_INTERVAL_MS` | `2000` | How often buffered writes become searchable |
+| `ENTERPRISE_AUTH_URL` | (empty) | Enables `tenants::ActiveTenantTracker` -- empty means write-routing has no active-tenant gate, same as every deployment before Phase 4. Must be set together with `ENTERPRISE_AUTH_SERVICE_TOKEN` below, or `Config::load` fails |
+| `ENTERPRISE_AUTH_SERVICE_TOKEN` | (empty) | RoleService Bearer credential for `GET /internal/active-tenants`, minted via `enterprise-auth -mint-service-token search` |
 
 ## Building & testing
 
@@ -147,7 +158,11 @@ pure `tenant_id_from_headers` header-extraction helper it uses is
 factored out and unit-tested the same way `ingest/consumer`'s Go
 equivalent is, including a guard test
 (`test_tenant_id_header_key_matches_go`) against the header-key literal
-drifting from the Go side's.
+drifting from the Go side's. `tenants.rs`'s tests genuinely exercise
+`reqwest` against a real (if hand-rolled, dependency-free) TCP server —
+the actual `Authorization: Bearer` header construction, JSON response
+parsing, and both fail-closed paths (a rejected first fetch, an
+unreachable server), not a fake HTTP client substituted in.
 
 ```sh
 # from the repo root, not search/