main
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 jcoffey-dev
|
13cf9a30cb |
Rebrand: Sentry -> Cairn OBS
Full rebrand across cosmetic branding, code identifiers, and infrastructure/data-plane naming, using the supplied Cairn OBS logo package. Cosmetic: favicon/logo swap (also closes a stale license-audit finding -- the old favicon was SvelteKit's unreplaced scaffold logo), new centered welcome landing page, larger/legible sidebar logo, page titles, CLAUDE.md/README/docs prose. Code identifiers: Go module path github.com/sentry/sentry -> github.com/cairnobs/cairnobs across all 13 modules and ~91 files (protoc regenerated); Rust crates sentry-agent/sentry-parser/sentry-search -> cairnobs-*; CLI sentryctl -> cairnobsctl; Terraform provider fully renamed (sentry_dashboard etc. -> cairnobs_dashboard, provider type, env vars); every session/auth cookie name; agent config paths and Windows service identity. Deliberately preserved: the gRPC wire protocol's protobuf packages (sentry.logs.v1, sentry.agent.v1) and their Go import directory (proto/sentry/...) -- renaming the wire-level package would break every currently-deployed agent binary (confirmed two real hosts, including mail.inbuxa.com, are actively streaming through this exact contract) until rebuilt and redeployed in lockstep with an ingest cutover. Only the Go module path wrapping the generated code changes. Infrastructure: every docker-compose container name (root and three component-level compose files); the Helm chart (directory, Chart.yaml, named-template helpers, all templates, values.yaml image repos); Kubernetes Operator (CRD group sentry.io -> cairnobs.io, both CRD YAML files, Go identifiers, RBAC markers); the coupled enterprise/tenantcrd package. Caught and fixed real path-coupling bugs along the way: the Helm chart's search/ingest volume mounts and the dev-only-credential detection constant vs. docker-compose.yml's literal values had to move together or a security warning would have silently stopped firing. Data plane: Postgres database sentry_metadata -> cairnobs_metadata and role sentry -> cairnobs; ClickHouse database sentry -> cairnobs; Kafka topic sentry.logs.raw -> cairnobs.logs.raw and its consumer groups. Source-level defaults, docker-compose.yml, and every migrate.sh/ provision script default updated together; already-applied migration files left untouched per this repo's immutable-migration convention. Verified at every layer: all 13 Go modules build/vet/test clean, both Rust workspaces (agent, search) build/clippy/test clean, npm run check/ build clean, docker compose config validates on all four compose files. Live-verified against a real docker stack multiple times through this work, including a final fresh-volume run confirming the actual renamed Postgres database/role, ClickHouse database, and Kafka topic all work end to end with a real login and query, zero console errors. |
||
|
jcoffey-dev
|
088677643f |
Close search's active-tenant write-routing gap with a polled allowlist
search/src/consumer.rs's write-routing (built last pass) had no active- tenant check at all: IndexRegistry.resolve() would open-or-create an index directory for any syntactically-valid tenant_id, active or not -- unlike ClickHouse's chwriter.Registry (an active-tenants-only snapshot built at enterprise-ingest startup) or the read side (gated by searchclient.TenantChecker, a direct rbacstore query). search is AGPL core with no Postgres access and no enterprise/ import allowed, so it needed a network boundary instead -- the same shape ingest's TenantResolver already uses against enterprise-auth, just Rust calling Go instead of Go calling Go. New GET /internal/active-tenants endpoint on enterprise-auth (rbacstore.ListActiveTenantIDs + authhandler.handleActiveTenants), gated on a RoleService Bearer credential -- server-to-server auth, the same shape alerting presents to api, minted via the already-generic enterprise-auth -mint-service-token search. search/src/tenants.rs's ActiveTenantTracker polls it every 60s, blocking startup on the first fetch succeeding (fail-closed cold start -- a control-plane outage at boot must not silently accept every tenant_id) and keeping the last- known-good set on any later refresh failure (a transient blip shouldn't stop every tenant's indexing, only prevent the allowlist from growing/ shrinking until connectivity resumes). consumer.rs refuses any tagged record whose tenant isn't in the polled set, before ever calling resolve() -- IndexRegistry itself stays policy-free, matching the same mechanism/policy split clickhousewriter.Writer vs. chwriter.Registry already draws on the ClickHouse side. Off unless ENTERPRISE_AUTH_URL/ENTERPRISE_AUTH_SERVICE_TOKEN are both set (search/src/config.rs rejects exactly one being set) -- every existing deployment is unaffected. Verified with real HTTP round trips in this environment: tenants.rs's tests exercise real reqwest requests (actual Authorization: Bearer header, actual JSON parsing) against a hand-rolled dependency-free TCP test server, including both fail-closed paths (rejected first fetch, unreachable server). authhandler's new tests cover the credential-kind distinction this endpoint exists to enforce -- a real human session, even for a genuine Owner, must not satisfy a check meant for a service identity. One asymmetry remains, disclosed rather than fixed: chwriter.Registry's snapshot still never refreshes (stale until enterprise-ingest restarts), while ActiveTenantTracker's 60s poll gives Tantivy a materially tighter staleness window. Neither is a live per-write check -- that would mean a database/HTTP round trip per record, a throughput cost neither implementation accepts -- so both have some staleness window by design; the gap between the two windows is what's disclosed, not a claim either is fully live. |
||
|
jcoffey-dev
|
17fdc212c2 |
Give ingest a real tenant identity (write-routing deferred, disclosed)
Ingest tenant-awareness was named "undesigned, not just unbuilt" across CLAUDE.md/threat-model.md/the runbook since early Phase 4 -- the last major standing gap. Scoping was agreed via AskUserQuestion: a config-supplied tenant_id + shared-secret token ingest validates (smaller real implementation, no new PKI), over per-tenant mTLS certs. This change builds that identity mechanism end to end and attaches it to every record at the point it enters the system; it deliberately does NOT build per-tenant write-routing for ClickHouse or Tantivy -- that's real, separately-scoped follow-up work, disclosed explicitly everywhere this was previously called undesigned, not silently left half-done. New pieces: - metadata/migrations/0034 + enterprise/internal/rbacstore/ ingest_credentials.go: a per-tenant bearer credential, only its SHA-256 hash ever persisted (same reasoning a password gets hashed, not stored raw) -- CreateIngestCredential returns the plaintext exactly once, ValidateIngestCredential/RevokeIngestCredential/ ListIngestCredentialsForTenant round it out. - enterprise-auth gains -create-ingest-credential-tenant/ -list-ingest-credentials-tenant/-revoke-ingest-credential (same offline-operator-flag shape as every other credential-minting flag in this binary) and a new POST /internal/authorize-ingest endpoint (internal/authhandler) validating a presented token and resolving its tenant -- a genuinely different credential type from session-backed /internal/authorize, so it doesn't touch session.Manager at all. - ingest (AGPL core) gains an optional TenantResolver (internal/grpcserver, nil by default) and its HTTP client implementation (internal/tenantresolver.HTTPResolver) -- a plain HTTP call to enterprise-auth's new endpoint, never an enterprise/ import, same "network boundary, not import boundary" shape api/authz.HTTPAuthorizer already uses for the query path. PushBatch now requires an `authorization: Bearer <token>` gRPC metadata entry once a resolver is configured, fails the whole batch closed on a missing/invalid credential (never falls back to "no tenant"), and attaches the resolved tenant ID to every record as a `tenant_id` Kafka message header before producing it. Verified with real round trips at every layer, no Docker needed: rbacstore's credential CRUD (skip-gated on live Postgres, same as every other rbacstore integration test this phase), authhandler's new endpoint (real HTTP via httptest, including the regression test that a session token must not validate as an ingest credential), tenantresolver (real HTTP client against httptest, same pattern as authz.HTTPAuthorizer's own tests), and grpcserver's PushBatch (fake resolver/producer -- no resolver leaves messages unchanged, a configured resolver attaches the right header or fails closed on a bad/missing token). Helm: ingest.requireTenantCredential (default false) is a deliberate, separate opt-in from enterprise.enabled -- turning ENTERPRISE_AUTH_URL on for ingest requires every agent to already hold a credential or be refused outright, so it must not default on just because enterprise.enabled does (same reasoning api.yaml's ENTERPRISE_AUTH_URL isn't tied to enterprise.enabled directly either). docker-compose.yml leaves it unset, same as ever. Docs updated everywhere this was called "undesigned": CLAUDE.md, docs/architecture.md, docs/security/threat-model.md (including its summary table, now split into "identity: built" vs "write-routing: not yet"), docs/phase-4-runbook.md (new §13), enterprise/README.md. |
||
|
jcoffey-dev
|
1d57e697b1 |
Phase 4: real per-tenant ClickHouse isolation via a new enterprise-api binary
Closes the threat model's headline finding for the SQL query path:
enterprise/internal/tenantprovision does real CREATE DATABASE/USER/GRANT
against ClickHouse, and enterprise/internal/chrunner is a per-tenant
connection registry implementing api's SQLRunner interface, resolving
the tenant from the authenticated request identity -- never a
caller-suppliable parameter. Both are wired into a new binary,
enterprise/cmd/enterprise-api, alongside the unchanged single-tenant
api/cmd/api, since AGPL core can never import enterprise/ and Go's own
internal/ package visibility rules meant enterprise/ couldn't implement
core's SQLRunner interface without importing the package that defines
it. That required moving api/internal/{authz,queryapi,dashboards,
querylang/executor,searchclient,httpserver} out of internal/ -- the
minimal set enterprise-api needs to import; querylang's compiler
internals (planner/lexer/parser/ast/ir) and api's own config stay
internal, since nothing outside api needs them directly.
Also finally wires enterprise/internal/audit into queryapi.AuditLogger
(nil since Phase 4 task 4) via a new adapter, and adds live-ClickHouse
integration tests for two of the four adversarial probes named in
docs/phase-4-isolation-design.md's verification plan.
Corrected several overclaims in the docs while writing this up: an
earlier claim that rbacstore's CRUD was "verified against a live
Postgres" was never actually true in this environment (only
internal/audit was, earlier in this phase, before Docker access was
lost) -- threat-model.md, phase-4-runbook.md, CLAUDE.md, and
enterprise/README.md all now distinguish "a real integration test
exists" from "this was confirmed against a live database."
Still not built: Tantivy/free-text tenant isolation
(enterprise/internal/searchclient), and any deployment-topology
mechanism that actually routes traffic to enterprise-api instead of
plain api -- both binaries exist side by side today with nothing
enforcing or flagging which one a deployment runs.
|
||
|
jcoffey-dev
|
3eb0f4c589 |
Phase 4: SSO scaffolding, RBAC enforcement, tenant-scoped dashboards, audit logging, K8s deployment
RBAC (api/internal/authz) is live on /query and /dashboards, backed by a new enterprise/ module (session issuance, audit logging, RBAC storage, OIDC/SAML protocol wiring) that core never imports -- only calls over HTTP. Found and fixed a real cross-tenant vulnerability in dashboards (no tenant_id filtering at all) while writing the threat model doc. Two things are explicitly NOT done, documented rather than hidden: tenant isolation for log data itself (/query still shares one ClickHouse connection and Tantivy index across every tenant -- RBAC controls who can query, not what a query can see), and human SSO login (protocol wiring exists, no HTTP handler calls it yet). See docs/security/threat-model.md and docs/phase-4-runbook.md. Also adds deploy/ (Go Operator + Helm chart, validated offline only -- no cluster was reachable in this environment). |