Build per-tenant Tantivy write-routing, closing the last ingest write gap

search/src/consumer.rs now resolves each record's tenant_id Kafka header
through the same IndexRegistry the read side (search/src/registry.rs +
enterprise/internal/searchclient) already used, and writes into that
tenant's own index instead of always the default one. The periodic
Tantivy commit now commits every tenant index that's actually seen a
write (IndexRegistry::commit_all), not just the default index.

Unlike ClickHouse, this needed no "second binary": Tantivy has no grant
system to gate a commercially-licensed credential behind, so
IndexRegistry already lived directly in this AGPL-core search binary --
there was never an import-boundary reason to split the write side into
an enterprise/ binary the way chwriter/enterprise-ingest was for
ClickHouse. Read and write simply share one registry.

Because Tantivy is an embedded library, this is genuinely verified in
this environment, not just written: registry.rs's
commit_all_commits_default_and_every_opened_tenant_index writes into the
default index plus two tenant indices, confirms nothing is searchable
pre-commit, then confirms all three are post-commit. consumer.rs's
tenant_id_from_headers is factored out as a small pure helper (mirroring
ingest/consumer.tenantIDFromHeaders) with its own unit tests, plus a
guard test against the "tenant_id" header-key literal drifting from the
Go side's -- the same guard-test pattern ingest/cmd/ingest already used
for its own two Go copies of the constant, now mirrored a third time
across the language boundary.

One gap is disclosed, not fixed, by this change: unlike chwriter.Registry
(an active-tenants-only snapshot built at enterprise-ingest startup, so
an unrecognized tenant_id is refused outright) and unlike the read side
(gated by searchclient.TenantChecker), this consumer's registry.resolve()
call has no active-tenant check at all -- search has no Postgres access
to check tenant status against. A still-valid-but-should-be-revoked
ingest credential can cause an index directory to be created for a
tenant that's no longer active. Narrow blast radius (an orphan, isolated,
empty index, not cross-tenant leakage, and only reachable with a real
signed credential), but real -- see registry.rs's doc comment on
resolve(). Closing it fully would mean giving search some way to learn
which tenants are active without an enterprise/ import, which isn't
designed yet.

This closes the last of Phase 4's ingest write-routing gaps (ClickHouse
was closed last commit). The one remaining gap in the whole phase is now
the tenant-picker frontend page, deliberately deferred earlier in this
phase as out of scope for this environment.

search/src/main.rs

@@ -35,11 +35,10 @@ async fn main() -> Result<()> {
     let index = Arc::new(
         SearchIndex::open_or_create(&cfg.index_path).context("opening tantivy index")?,
     );
-    // Per-tenant indices (Phase 4) are resolved on demand by
-    // IndexRegistry, opened under cfg.tenants_index_path -- see
-    // registry.rs's doc comment for what this does and doesn't isolate
-    // yet (read-side only; the consumer below still only ever writes
-    // into the single default `index` above).
+    // Per-tenant indices are resolved on demand by IndexRegistry, opened
+    // under cfg.tenants_index_path -- see registry.rs's doc comment for
+    // what this isolates (both read and write now) and the one residual
+    // gap it doesn't close.
     let registry = Arc::new(IndexRegistry::new(Arc::clone(&index), cfg.tenants_index_path.clone()));
 
     let partition_count: i32 = std::env::var("REDPANDA_TOPIC_PARTITIONS")
@@ -48,9 +47,9 @@ async fn main() -> Result<()> {
         .unwrap_or(DEFAULT_PARTITION_COUNT);
 
     let consumer_cfg = Arc::clone(&cfg);
-    let consumer_index = Arc::clone(&index);
+    let consumer_registry = Arc::clone(&registry);
     let consumer_handle = tokio::spawn(async move {
-        if let Err(e) = consumer::run(consumer_cfg, consumer_index, partition_count).await {
+        if let Err(e) = consumer::run(consumer_cfg, consumer_registry, partition_count).await {
             tracing::error!(error = %e, "redpanda consumer exited with error");
         }
     });