Give chwriter.Registry periodic refresh, matching Tantivy's tracker
Closing search's active-tenant gap last commit surfaced a real asymmetry by comparison: chwriter.Registry's per-tenant writer map was still a snapshot built once at enterprise-ingest startup with no refresh at all, while search's new ActiveTenantTracker refreshes every minute. A tenant deprovisioned after enterprise-ingest started would keep writing successfully to ClickHouse until the next restart -- a real, disclosed staleness gap, not matched by anything on the Tantivy side anymore. Registry.StartRefreshing spawns a goroutine that re-lists active tenants every minute (dataSourceRefreshInterval, same interval as search's tracker) via a new SourceLister callback and reconciles the writer map: opens a connection for a newly-active tenant, closes and removes one no longer active. New connections are dialed before taking the write lock, so a slow/unreachable ClickHouse for one newly-active tenant never blocks WriteBatch's read lock. A refresh failure (lister error, or one tenant's connection failing to open) logs and leaves the existing map untouched for that tick -- the same last-known-good posture ActiveTenantTracker already uses, so a transient rbacstore/Postgres blip doesn't evict every other tenant's already-working writer. WriteBatch now takes a read lock and Close takes a write lock -- the writer map was safe unsynchronized before only because it was immutable after New() returned; StartRefreshing makes it mutable at runtime. enterprise-ingest/main.go extracts the existing rbacstore-row-to- DataSource adaptation into tenantDataSourceLister, reused for both the initial synchronous load and StartRefreshing's periodic calls, so the two can't drift into checking different things. Verified: the lister-error-keeps-last-known-good path is Docker-free (same "construct a Registry directly, bypass New" trick the existing fail-closed tests use). The actual add/remove reconciliation against real ClickHouse connections (TestRefreshAddsNewlyActiveTenant, TestRefreshRemovesNoLongerActiveTenant) are skip-gated live-ClickHouse tests, same CHWRITER_TEST_CLICKHOUSE_ADDR convention as this package's existing integration tests -- not run against a live database in this environment. This closes the last disclosed gap from Phase 4's write-routing work: both storage engines now share the same one-minute active-tenant staleness bound instead of one being materially staler than the other.
This commit is contained in:
@@ -283,8 +283,16 @@ default as everything else optional in this codebase); when they are,
|
||||
startup blocks on the first fetch succeeding and later refresh failures
|
||||
keep serving the last-known-good set rather than clearing it. Verified
|
||||
with real HTTP round trips against a hand-rolled TCP test server in this
|
||||
environment, no live enterprise-auth needed. **The tenant-picker page is
|
||||
now built too**:
|
||||
environment, no live enterprise-auth needed. **This closing move exposed
|
||||
the ClickHouse side's own gap by comparison** — `chwriter.Registry`'s
|
||||
writer map was still a startup-only snapshot with no refresh at all, a
|
||||
real asymmetry once Tantivy's tracker refreshed every minute and
|
||||
ClickHouse's didn't — so `Registry.StartRefreshing` (new) closes that
|
||||
too: same one-minute interval, same last-known-good posture on a failed
|
||||
refresh, opening connections for newly-active tenants and closing ones
|
||||
no longer active. Both engines now share the same active-tenant
|
||||
staleness bound instead of one being materially staler than the other.
|
||||
**The tenant-picker page is now built too**:
|
||||
`web/src/routes/select-tenant` calls `GET /auth/memberships`/
|
||||
`POST /auth/select-tenant` via `fetch(..., {credentials: 'include'})`
|
||||
(new `$lib/api.ts` functions), which needed a second CORS posture
|
||||
|
||||
Reference in New Issue
Block a user