deploy/helm/sentry/templates/api.yaml and the new enterprise-api.yaml
are mutually exclusive, gated on opposite sides of the same
enterprise.enabled flag -- exactly one renders, both as a Deployment+
Service named {{ .Release.Name }}-api on port 8080, so every consumer
(alerting's API_QUERY_URL, web's build args) needs zero conditional
logic of its own. This is the concrete fix for what the threat model
named as the single largest remaining gap once both storage engines'
isolation mechanisms were built: previously nothing forced or flagged
whether a deployment ran the tenant-isolated binary. Now the same flag
that turns on RBAC/audit/SSO also chooses the query binary.
Verified by parsing (not eyeballing) helm template's rendered output
under both value sets: exactly one sentry-api Deployment/Service either
way, with the right image, and kubeconform -strict clean against the
real Kubernetes 1.31 schema. Not applied to a live cluster (still no
cluster in this environment) -- docker-compose.yml also still runs
plain api unconditionally, so this enforcement is Helm-only for now.
Updated the threat model, architecture doc, CLAUDE.md, and deploy/
READMEs to reflect this and to name what's left: ingest has no tenant
concept for either storage engine (undesigned), and the Tenant CRD
(deploy/operator) and enterprise-api -provision-tenant are still two
separate, unreconciled provisioning mechanisms.
Closes the last named "isolation mechanism" gap: search.proto gains a
tenant_id field on SearchRequest; search/src/registry.rs's IndexRegistry
resolves it to an on-demand-opened, per-tenant Tantivy index (empty
tenant_id keeps today's single default index, so this is purely
additive); enterprise/internal/searchclient sets that field from the
authenticated request identity in ctx, mirroring chrunner's exact
fail-closed "never a parameter" shape. Wired into enterprise-api in
place of the shared api/searchclient.
Unlike the ClickHouse pieces from the previous two commits, this one is
genuinely verified end to end in this environment: Tantivy is an
embedded library, not a networked service, so both the Rust index
registry (cargo test, cargo clippy --all-targets -- -D warnings, both
clean) and the Go client (a real in-process gRPC server) could actually
run. registry.rs's tenant_index_is_isolated_from_default_and_other_tenants
seeds three real indices with the same term and confirms a tenant-scoped
search returns only that tenant's document -- item 3 of the isolation
design doc's verification plan, closed for real, not just written.
With both ClickHouse and Tantivy isolation now built, the single largest
remaining gap is no longer a missing mechanism: it's that nothing forces
or flags whether a deployment actually runs enterprise-api instead of
plain api, and that ingest itself has no tenant concept for either
storage engine (every record still lands in the one shared database/
index no matter what -- undesigned, not just unbuilt). Updated the
threat model, architecture doc, CLAUDE.md, and both READMEs accordingly.
Closes the other major named gap from this phase: until now, there was
no way for a human to actually log in -- only /alerting's RoleService
credential could be minted. GET /auth/oidc/login and GET
/auth/oidc/callback drive the real coreos/go-oidc flow already wired in
enterprise/internal/oidc: CSRF state in a short-lived cookie, code
exchange, ID token verification, upserting a users row, resolving
tenant/role from exactly one tenant_memberships row (refusing outright
on zero or more than one, rather than guessing), and issuing a real
session cookie.
Unlike everything else built this phase, this one is genuinely verified
end to end: the tests spin up coreos/go-oidc's own oidctest fake IdP,
which signs real RS256 ID tokens, and drive the full login->callback->
session-cookie round trip through actual signature verification -- no
live database or Docker needed, so nothing here is asserted without
having actually been run in this session. Also fixes a real bug caught
while wiring this into enterprise-auth's main.go: assigning a nil
*oidc.Provider to the handler's interface field would have produced a
non-nil interface wrapping a nil pointer (Go's classic typed-nil trap),
silently breaking the "OIDC not configured" no-op path -- New() now
takes the concrete pointer type and checks it before ever converting to
the interface, with a regression test pinning the fix down.
Still missing: SAML's equivalent (ACS endpoint), a tenant-picker UI for
multi-membership identities, and any admin UI to actually create a
tenant_memberships row (today that's manual SQL, documented in the
runbook's new bootstrap walkthrough).
Closes the threat model's headline finding for the SQL query path:
enterprise/internal/tenantprovision does real CREATE DATABASE/USER/GRANT
against ClickHouse, and enterprise/internal/chrunner is a per-tenant
connection registry implementing api's SQLRunner interface, resolving
the tenant from the authenticated request identity -- never a
caller-suppliable parameter. Both are wired into a new binary,
enterprise/cmd/enterprise-api, alongside the unchanged single-tenant
api/cmd/api, since AGPL core can never import enterprise/ and Go's own
internal/ package visibility rules meant enterprise/ couldn't implement
core's SQLRunner interface without importing the package that defines
it. That required moving api/internal/{authz,queryapi,dashboards,
querylang/executor,searchclient,httpserver} out of internal/ -- the
minimal set enterprise-api needs to import; querylang's compiler
internals (planner/lexer/parser/ast/ir) and api's own config stay
internal, since nothing outside api needs them directly.
Also finally wires enterprise/internal/audit into queryapi.AuditLogger
(nil since Phase 4 task 4) via a new adapter, and adds live-ClickHouse
integration tests for two of the four adversarial probes named in
docs/phase-4-isolation-design.md's verification plan.
Corrected several overclaims in the docs while writing this up: an
earlier claim that rbacstore's CRUD was "verified against a live
Postgres" was never actually true in this environment (only
internal/audit was, earlier in this phase, before Docker access was
lost) -- threat-model.md, phase-4-runbook.md, CLAUDE.md, and
enterprise/README.md all now distinguish "a real integration test
exists" from "this was confirmed against a live database."
Still not built: Tantivy/free-text tenant isolation
(enterprise/internal/searchclient), and any deployment-topology
mechanism that actually routes traffic to enterprise-api instead of
plain api -- both binaries exist side by side today with nothing
enforcing or flagging which one a deployment runs.
RBAC (api/internal/authz) is live on /query and /dashboards, backed by a
new enterprise/ module (session issuance, audit logging, RBAC storage,
OIDC/SAML protocol wiring) that core never imports -- only calls over
HTTP. Found and fixed a real cross-tenant vulnerability in dashboards
(no tenant_id filtering at all) while writing the threat model doc.
Two things are explicitly NOT done, documented rather than hidden:
tenant isolation for log data itself (/query still shares one ClickHouse
connection and Tantivy index across every tenant -- RBAC controls who
can query, not what a query can see), and human SSO login (protocol
wiring exists, no HTTP handler calls it yet). See
docs/security/threat-model.md and docs/phase-4-runbook.md.
Also adds deploy/ (Go Operator + Helm chart, validated offline only --
no cluster was reachable in this environment).
Saved, shareable multi-panel dashboards (table/line/bar/single-stat
panels via gridstack + uPlot, global + per-panel time range, JSON
export/import) and threshold/absence alert rules with an
ok/pending/firing evaluator and webhook/Slack/PagerDuty delivery.
- New /metadata component: Postgres control-plane store for dashboards,
panels, notification targets, alert rules/state, and delivery log --
see docs/phase-3-dashboard-design.md for why ClickHouse's MergeTree
family isn't a fit for this access pattern (needs real row-level
locking and read-your-writes consistency).
- api/internal/dashboards: dashboard/panel CRUD, pure -- panel query
execution stays client-side, reusing the existing /query endpoint.
- New /alerting service: rule/target CRUD, a ticker-driven evaluator
(claim-then-evaluate concurrency control, transactional-outbox
delivery, query errors and threshold zero-rows never coerced into a
false transition) and webhook/Slack/PagerDuty delivery with
retry/backoff. See docs/phase-3-alerting-design.md for the full
state-machine design and the four correctness properties it
implements.
- web: /dashboards and /alerts UIs; cli: sentryctl dashboards/alerts
list/get/apply, seeding a future Terraform provider's JSON contract.
- hack/alert-load-test: 500 rules against real ClickHouse data, real
measured results in docs/phase-3-runbook.md.
Five real bugs found by actually running this against a live stack
(documented in the runbook, not just fixed silently): a latent Phase 2
bug where ClickHouse rejected the timestamp format used for
earliest=/latest= queries; a "now" literal token injected into query
text; a GridStack/uPlot layout-timing race; JS's Date.parse being too
lenient to use as a timestamp-detection heuristic; a rule's "enabled"
field silently defaulting to false when omitted; and the evaluator's
claim-batch-size and worker-pool-concurrency defaulting to the same
value, causing 500 concurrently-due rules to take 125s to cycle through
instead of the configured 60s.
Replaces the separate SQL-only /query and text-only /search endpoints
with one pipe-syntax query language (plus raw SQL escape hatch) that
compiles to a single IR and execution plan across both backends, so a
query like `message:"connection refused" | stats count by host` runs
as one request instead of two disjoint tools.
- api/internal/querylang: lexer -> ast -> parser -> ir -> planner ->
executor, each layer independently tested.
- Execution generalizes Phase 1's proven Tantivy-prefilter pattern
into a 4-way routing table (pure ClickHouse / text-only / text +
aggregation / raw SQL passthrough).
- Unified web query page and `sentryctl query`, both hitting the same
POST /query endpoint.
- Benchmarked against a real 1,022,000-row dataset
(hack/benchmark-fixture); caught and fixed a real bug where the
Tantivy prefilter cap (10,000) produced an IN-clause exceeding
ClickHouse's default max_query_size -- lowered to 5,000, documented
in docs/query-language-design.md and docs/phase-2-runbook.md.
- docs/query-language-reference.md: customer-facing syntax reference.
Extends the agent, ingest, storage, api, and web with Windows Event
Log/ETW sourcing and Tantivy-backed free-text search, per the approved
Phase 1 plan.
- CLAUDE.md: materialized on disk (never existed as a file before) with
a new Phase 1 "done looks like" section.
- agent: Windows Event Log (EvtSubscribe) and ETW sources, Windows
service wrapper (install/uninstall/run-service), both feature- and
target_os-gated so Linux builds/tests/clippy stay unaffected. Also
fixed two pre-existing Phase 0 clippy gaps (dead-code on
default-features-only builds, a type-inference edge case) found while
testing every feature combination properly for the first time.
UNVERIFIED on real Windows -- no Windows toolchain existed anywhere in
the build environment; flagged prominently in three places.
- proto/ingest: new record_id field, assigned once server-side in
ingest's gRPC front end so ClickHouse and Tantivy agree on the same ID
for the same record.
- storage: record_id column + bloom filter index, verified against a
live ClickHouse.
- search: new service, Tantivy index, rskafka consumer as an independent
second consumer group on the same Redpanda topic ingest already reads.
- api/web: new /search endpoint and page, sharing the query page's
result-table shape and component.
- hack/windows-fixture: sends realistic Windows-shaped data straight to
ingest, so the pipeline's handling of it is verifiable without a
Windows host.
Verified end-to-end on the live docker-compose stack: the same record_id
comes back from both /query and /search for the same log line, including
for windows-fixture's synthetic Windows Event Log data. Real bugs found
and fixed along the way: api/Dockerfile missing proto/ in its build
context, search's logs being completely silent (RUST_LOG gap), and
search/target/ missing from .gitignore/.dockerignore.
End-to-end log pipeline for Linux hosts, per /docs/architecture.md:
- proto: shared gRPC contract (agent <-> ingest), Go bindings checked in
- agent: Rust, musl-targeted, journald/file sourcing, RFC5424 parser,
mTLS gRPC client, no required config for the common case
- ingest: Go, single binary with --mode server|consumer|all; gRPC front
end forwards to Redpanda unchanged, consumer normalizes and
batch-writes to ClickHouse with at-least-once delivery
- storage: ClickHouse schema + a plain SQL-file migration runner
- api: minimal SELECT-only query endpoint, plain REST (not gRPC+gateway
yet -- see api/README.md)
- web: SvelteKit static SPA, one query page
- transport: Redpanda compose + topic provisioning
- cli: sentryctl ping stub
- hack/dev-certs: throwaway CA + cert generation for local mTLS
- root docker-compose.yml + docs/phase-0-runbook.md tie it together
Not yet run end-to-end against real Docker/ClickHouse/Redpanda -- see the
runbook's caveats section before relying on this working as-is.
jcoffey-dev