0ee2e9183baabcd895889332d98bf071a5e0878f
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 jcoffey-dev
|
200f801e2c |
Clear the Dependabot findings
Dependabot alerts were switched on for this repo today and reported 12 open findings. Ten are fixed here; the other two are addressed below. gRPC 1.83.0 -> 1.83.1, in all nine modules that require it. This is GHSA-vp52-pcj8-j9qc / CVE-2026-84304, heap memory exhaustion via HTTP/2 DATA frame fragmentation, affecting <= 1.83.0. It matters more than the version delta suggests: ingest/ is a gRPC listener deliberately exposed to the internet on :4317, so a remote OOM is reachable. mTLS narrows that to holders of a client certificate, which is why this was not an emergency, but the fix is one patch release away and there is no reason to carry it. golang.org/x/oauth2 0.21.0 -> 0.27.0 in deploy/operator, an indirect dependency (GHSA-6v2p-p543-phr9). enterprise/ was already past it at 0.36.0. npm cookie 0.6.0 -> 0.7.2, via an overrides entry rather than a dependency bump. @sveltejs/kit requires ^0.6.0 and still does at 2.70.3, the latest release, so there is no version of kit that resolves this on its own -- an override is the only route that does not involve waiting on upstream. Three incidental changes came out of `go mod tidy` and are not mine: genproto/googleapis/rpc moved forward as a transitive of the new grpc; pgx/v5 was reclassified from indirect to direct in enterprise/, which is simply correct, since audit.go and cmd/enterprise-auth import it; and the proto replace directive shuffled between require blocks at the same version. The twelfth finding, lru (GHSA-rhfx-m35p-ff5j), is not fixed and is not fixable here -- see the note in the pull request. It is CVSS 0, a Stacked Borrows soundness issue in IterMut, and reaching a patched version means tantivy 0.22 -> 0.26, which is a search engine migration rather than a dependency bump. Verified: all ten Go modules build, 40 test packages pass, the web app builds and svelte-check reports 0 errors across 288 files. |
||
|
jcoffey-dev
|
13cf9a30cb |
Rebrand: Sentry -> Cairn OBS
Full rebrand across cosmetic branding, code identifiers, and infrastructure/data-plane naming, using the supplied Cairn OBS logo package. Cosmetic: favicon/logo swap (also closes a stale license-audit finding -- the old favicon was SvelteKit's unreplaced scaffold logo), new centered welcome landing page, larger/legible sidebar logo, page titles, CLAUDE.md/README/docs prose. Code identifiers: Go module path github.com/sentry/sentry -> github.com/cairnobs/cairnobs across all 13 modules and ~91 files (protoc regenerated); Rust crates sentry-agent/sentry-parser/sentry-search -> cairnobs-*; CLI sentryctl -> cairnobsctl; Terraform provider fully renamed (sentry_dashboard etc. -> cairnobs_dashboard, provider type, env vars); every session/auth cookie name; agent config paths and Windows service identity. Deliberately preserved: the gRPC wire protocol's protobuf packages (sentry.logs.v1, sentry.agent.v1) and their Go import directory (proto/sentry/...) -- renaming the wire-level package would break every currently-deployed agent binary (confirmed two real hosts, including mail.inbuxa.com, are actively streaming through this exact contract) until rebuilt and redeployed in lockstep with an ingest cutover. Only the Go module path wrapping the generated code changes. Infrastructure: every docker-compose container name (root and three component-level compose files); the Helm chart (directory, Chart.yaml, named-template helpers, all templates, values.yaml image repos); Kubernetes Operator (CRD group sentry.io -> cairnobs.io, both CRD YAML files, Go identifiers, RBAC markers); the coupled enterprise/tenantcrd package. Caught and fixed real path-coupling bugs along the way: the Helm chart's search/ingest volume mounts and the dev-only-credential detection constant vs. docker-compose.yml's literal values had to move together or a security warning would have silently stopped firing. Data plane: Postgres database sentry_metadata -> cairnobs_metadata and role sentry -> cairnobs; ClickHouse database sentry -> cairnobs; Kafka topic sentry.logs.raw -> cairnobs.logs.raw and its consumer groups. Source-level defaults, docker-compose.yml, and every migrate.sh/ provision script default updated together; already-applied migration files left untouched per this repo's immutable-migration convention. Verified at every layer: all 13 Go modules build/vet/test clean, both Rust workspaces (agent, search) build/clippy/test clean, npm run check/ build clean, docker compose config validates on all four compose files. Live-verified against a real docker stack multiple times through this work, including a final fresh-volume run confirming the actual renamed Postgres database/role, ClickHouse database, and Kafka topic all work end to end with a real login and query, zero console errors. |
||
|
jcoffey-dev
|
4b5dae5879 |
Add local login, agent extra log paths, IPv4/IPv6 metrics; remediate security audit findings
This is a large squashed commit covering two batches of prior uncommitted work plus a full security-audit remediation pass, kept together because go.mod/go.sum and several shared files (main.go, handler.go) were touched by both and splitting risked non-building intermediate commits. Features (built earlier, previously uncommitted): - Local username/password login for single-tenant deployments with no SSO configured (api/localauth, alerting/internal/sessioncheck, sentryctl users, web/src/routes/login, metadata migrations 0040/0041). - Remotely-editable additional log file paths for agents, on top of their existing primary source (api/agents, agent/sentry-agent extra-file-path diffing, web agent config UI). - IPv4/IPv6 addresses reported alongside other host system metrics. Security audit remediation (this pass, all live-verified in production): - Critical: block ClickHouse SSRF table functions (url/remote/file/s3/...) in the raw-SQL query escape hatch. - High: deny sensitive paths and require Admin to add agent extra_file_paths (Editor could previously point an agent at /etc/shadow or an SSH key); alerting webhook targets now validate against internal/metadata/loopback addresses, both at creation and send time; alerting's session middleware now enforces an Editor+ floor on mutating requests instead of "any authenticated session"; bumped goxmldsig to close a SAML signature-verification bypass (GO-2026-4753). - Medium: per-IP login rate limiting; security response headers (HSTS/CSP/nosniff/X-Frame-Options/Referrer-Policy/Permissions-Policy) on web/nginx.conf; a DevCredentialWarnings check in every Go service's config loader, logging loudly at startup if a deployment is still on docker-compose.yml's literal dev-only credentials; dependency bumps (golang.org/x/text, grpc, x/net, quick-xml, h2) across every affected Go module and both Rust crates, including a previously-uncovered x/net vulnerability in deploy/operator; a new security-scan.yml CI workflow running cargo-deny/govulncheck/npm-audit, mirroring the existing license-compliance.yml matrix shape. - Low: removed sentryctl's plaintext --password flag (shell history/`ps` exposure) in favor of stdin and a --password-stdin flag for reset-password's optional specific-password path; a dummy bcrypt comparison closes a login response-time username-enumeration side-channel. |
||
|
jcoffey-dev
|
1de77b969f |
Build per-tenant ClickHouse write-routing for ingest (Tantivy still deferred)
ingest tags every record with a tenant_id Kafka header (built previously), but nothing consumed it to actually route the write. This closes that for ClickHouse: enterprise/cmd/enterprise-ingest (a second binary, mirroring enterprise-api) reuses ingest/consumer's own flush loop unchanged, with enterprise/internal/chwriter.Registry -- a per-tenant clickhousewriter.Writer registry -- swapped in as the writer. A batch pulled from the single shared Redpanda topic can mix records from many tenants, so WriteBatch groups by TenantID and dispatches each group to its own tenant's connection, fail- closed on an empty or unrecognized tenant_id. ingest/consumer and ingest/clickhousewriter move out of internal/ (same reason api/internal/* moved earlier this phase: enterprise/ can't import anything under another module's internal/). Their New() constructors now take small local Config structs instead of ingest/internal/config types, so enterprise/ doesn't need that import either. Building this surfaced a real bug: tenantprovision.ProvisionClickHouse only granted SELECT on a tenant's ClickHouse user, correct for chrunner's read-only use but not enough for chwriter reusing the same credential to write -- every real per-tenant write would have failed closed with a permission error. Fixed by widening the grant to SELECT, INSERT; no cross-tenant boundary is crossed by also allowing INSERT within a tenant's own database. Helm gates enterprise-ingest's Deployment on the same ingest.requireTenantCredential flag that already gates tag validation -- write-routing is meaningless without tagging already being required, so they're one decision, not two. docker-compose.yml's version is a disclosed, weaker approximation: it can't achieve Helm's genuine -mode=server/-mode=consumer split, so with the enterprise profile active both ingest and enterprise-ingest independently consume every message via different consumer groups -- harmless duplication for local verification only. Not built: Tantivy's independent Redpanda consumer (search/src/consumer.rs) still doesn't read the tenant_id header at all -- every record still lands in the one shared index regardless of tenant. Not run: the live-ClickHouse- gated tests (chwriter's cross-tenant routing test, tenantprovision's INSERT regression test) -- no Docker/database access in this environment; they're correct Go that has never executed, disclosed as such in docs/security/ threat-model.md and docs/phase-4-runbook.md §14. |
||
|
jcoffey-dev
|
823f5d48d1 |
Unify the Tenant CRD with enterprise-api -provision-tenant (lightweight)
Closes a gap named across CLAUDE.md/docs/architecture.md/deploy/README.md
since early Phase 4: the operator's Tenant CRD and -provision-tenant
were two disconnected mechanisms. The operator's reconciler generated a
K8s Secret with a locally-generated random password that authenticated
against nothing (nothing ever called ClickHouse to create a matching
user), and unconditionally claimed status.phase=Active the moment a
Tenant object existed -- actively misleading, not just incomplete.
Two unification shapes were considered (surfaced to the user via
AskUserQuestion, given the real difference in blast radius): the
operator's reconcile loop becoming a second real actor (new Postgres +
ClickHouse admin credentials flowing into the K8s controller, plus real
reconcile-loop idempotency/retry design for an inherently one-shot
external side effect), or keeping -provision-tenant as the sole real
actor and having it also sync its result into the CRD. Went with the
lighter option.
enterprise/internal/tenantcrd (new): a Syncer using the K8s dynamic
client (unstructured.Unstructured + a GroupVersionResource, not
deploy/operator's typed Tenant struct -- avoids a cross-module Go
dependency between two independently-versioned modules for one type).
Upserts the Tenant object, creates/updates a Secret with the *real*
ClickHouse credentials owned by that Tenant via an OwnerReference, then
patches status.{clickHouseDatabaseName,clickHouseSecretRef,
tantivyIndexPath}. Idempotent and safe to retry: never rotates a
credential across a re-sync, never overwrites a pre-existing
spec.displayName a human/GitOps process set.
cmd/enterprise-api/main.go's runProvisionTenant calls Sync when
TENANT_CRD_NAMESPACE is set (empty = no-op, same shape as every other
optional dependency in this codebase). Its "already active" refusal is
now split: ClickHouse re-provisioning is still refused (rotating a live
credential would break every open connection for no benefit), but CR
sync alone is now retryable using the credentials already on file in
rbacstore -- needed for retrying a previously-failed sync, or
backfilling CR sync for a tenant provisioned before this existed.
deploy/operator's reconciler rewritten to match: it never claims
PhaseActive on its own initiative anymore, only once
status.ClickHouseDatabaseName is non-empty (the field -provision-tenant,
and only -provision-tenant, sets). Phase is now a pure function of
{spec.suspended, status.ClickHouseDatabaseName != ""} recomputed every
reconcile, not toggled in place -- fixes a related bug the old code
would have hit once suspension was involved: un-suspending an
already-provisioned tenant needs to return straight to Active, which
isn't derivable from "last observed phase was Suspended" alone. The
reconciler no longer creates or manages any Secret, dropped its
`secrets` RBAC grant entirely, and gained zero new dependencies.
Helm chart: enterprise-api gets its own ServiceAccount/Role/RoleBinding
(get/list/create tenants, get/update/patch tenants/status, get/create/
update secrets -- least-privilege, scoped to the release namespace, not
a ClusterRole) and a TENANT_CRD_NAMESPACE env var, both gated on
tenantOperator.enabled. tenant-operator's ClusterRole loses the
secrets grant it no longer needs.
Verified in this environment: enterprise/internal/tenantcrd's tests run
against k8s.io/client-go's fake dynamic + typed clientsets (real client
library, fake transport, no cluster needed); deploy/operator's rewritten
tenant_controller_test.go runs against controller-runtime's fake
client, including new regression tests for the "must not claim Active
without confirmation" and "un-suspending returns to Active, not
Provisioning" properties; helm template + parsing the rendered YAML
confirms the RBAC split renders exactly as designed under both
tenantOperator.enabled=true/false. Not verified: an actual
-provision-tenant run against a real cluster with the operator watching
(no live cluster in this environment, same disclosed limitation as the
rest of /deploy). Docs updated in lockstep: CLAUDE.md, docs/architecture.md,
deploy/README.md, deploy/helm/sentry/README.md (including a corrected
"Trying the two-tenant example" walkthrough), phase-4-runbook.md (new
§11), enterprise/README.md. Also fixed two unrelated stale claims found
along the way: docs/architecture.md still said docker-compose.yml ran
plain api unconditionally (fixed in an earlier commit, doc not updated
then), and enterprise-api's own main.go doc comment still said Helm/
docker-compose wiring wasn't built yet.
|
||
|
jcoffey-dev
|
08a90a27aa |
Build SAML login (enterprise/internal/loginhandler), mirroring OIDC
Adds GET /auth/saml/login + POST /auth/saml/acs alongside the existing OIDC pair, both converging on the same upsert-user/resolve-tenant/ issue-session path. loginhandler.New now takes an optional *saml.ServiceProvider, RegisterRoutes registers each protocol's routes independently so either, both, or neither can be configured. SAML's replay/unsolicited-response defense (InResponseTo, standing in for OIDC's state) is carried via a SameSite=None sentry_saml_request cookie -- None because the ACS endpoint receives a cross-site POST from the IdP's origin, which SameSite=Lax cookies are never sent on. enterprise-auth's main.go now fetches+parses SAML_IDP_METADATA_URL at startup (samlsp.FetchMetadata) and wires the result through. Verified to the same bar as OIDC: a real fake IdP (crewjam/saml/samlidp, genuine XML signing/verification) drives the full login->ACS->session-cookie round trip and negative paths (bad InResponseTo, missing request cookie, missing email/NameID, no/multiple tenant memberships), all in loginhandler/saml_test.go, no Docker needed. The login-form HTML is bypassed by pre-seeding a saml.Session directly into samlidp's session store and presenting the matching `session` cookie -- an IdP-supported shortcut (confirmed by reading GetSession), the same "skip the UI, keep the crypto real" approach oidctest gave the OIDC tests. Writing that test caught two real bugs in internal/saml.ParseResponse, both fixed here: it never called r.ParseForm() before reading the POSTed SAMLResponse field, so every real ACS POST would have silently decoded an empty response; and its email-attribute matching missed urn:oid:0.9.2342.19200300.100.1.3 (the standard LDAP "mail" OID), which is what an IdP sends by default absent an explicit AttributeConsumingService request for "email" -- exactly what samlidp's own DefaultAssertionMaker does, and plausibly what real IdPs' default SAML app templates do too. Docs (CLAUDE.md, threat-model.md, architecture.md, enterprise/README.md, phase-4-runbook.md, docker-compose.yml's enterprise-auth comment) updated in lockstep: SAML login moves from "protocol mechanics only" to "built, verified with a real fake IdP, not yet tried against a real external IdP or a running enterprise-auth container" -- the same disclosed gap OIDC already carried. |
||
|
jcoffey-dev
|
ba2276aa1a |
Phase 4: real Tantivy per-tenant isolation (search/src/registry.rs, enterprise/internal/searchclient)
Closes the last named "isolation mechanism" gap: search.proto gains a tenant_id field on SearchRequest; search/src/registry.rs's IndexRegistry resolves it to an on-demand-opened, per-tenant Tantivy index (empty tenant_id keeps today's single default index, so this is purely additive); enterprise/internal/searchclient sets that field from the authenticated request identity in ctx, mirroring chrunner's exact fail-closed "never a parameter" shape. Wired into enterprise-api in place of the shared api/searchclient. Unlike the ClickHouse pieces from the previous two commits, this one is genuinely verified end to end in this environment: Tantivy is an embedded library, not a networked service, so both the Rust index registry (cargo test, cargo clippy --all-targets -- -D warnings, both clean) and the Go client (a real in-process gRPC server) could actually run. registry.rs's tenant_index_is_isolated_from_default_and_other_tenants seeds three real indices with the same term and confirms a tenant-scoped search returns only that tenant's document -- item 3 of the isolation design doc's verification plan, closed for real, not just written. With both ClickHouse and Tantivy isolation now built, the single largest remaining gap is no longer a missing mechanism: it's that nothing forces or flags whether a deployment actually runs enterprise-api instead of plain api, and that ingest itself has no tenant concept for either storage engine (every record still lands in the one shared database/ index no matter what -- undesigned, not just unbuilt). Updated the threat model, architecture doc, CLAUDE.md, and both READMEs accordingly. |
||
|
jcoffey-dev
|
1d57e697b1 |
Phase 4: real per-tenant ClickHouse isolation via a new enterprise-api binary
Closes the threat model's headline finding for the SQL query path:
enterprise/internal/tenantprovision does real CREATE DATABASE/USER/GRANT
against ClickHouse, and enterprise/internal/chrunner is a per-tenant
connection registry implementing api's SQLRunner interface, resolving
the tenant from the authenticated request identity -- never a
caller-suppliable parameter. Both are wired into a new binary,
enterprise/cmd/enterprise-api, alongside the unchanged single-tenant
api/cmd/api, since AGPL core can never import enterprise/ and Go's own
internal/ package visibility rules meant enterprise/ couldn't implement
core's SQLRunner interface without importing the package that defines
it. That required moving api/internal/{authz,queryapi,dashboards,
querylang/executor,searchclient,httpserver} out of internal/ -- the
minimal set enterprise-api needs to import; querylang's compiler
internals (planner/lexer/parser/ast/ir) and api's own config stay
internal, since nothing outside api needs them directly.
Also finally wires enterprise/internal/audit into queryapi.AuditLogger
(nil since Phase 4 task 4) via a new adapter, and adds live-ClickHouse
integration tests for two of the four adversarial probes named in
docs/phase-4-isolation-design.md's verification plan.
Corrected several overclaims in the docs while writing this up: an
earlier claim that rbacstore's CRUD was "verified against a live
Postgres" was never actually true in this environment (only
internal/audit was, earlier in this phase, before Docker access was
lost) -- threat-model.md, phase-4-runbook.md, CLAUDE.md, and
enterprise/README.md all now distinguish "a real integration test
exists" from "this was confirmed against a live database."
Still not built: Tantivy/free-text tenant isolation
(enterprise/internal/searchclient), and any deployment-topology
mechanism that actually routes traffic to enterprise-api instead of
plain api -- both binaries exist side by side today with nothing
enforcing or flagging which one a deployment runs.
|
||
|
jcoffey-dev
|
3eb0f4c589 |
Phase 4: SSO scaffolding, RBAC enforcement, tenant-scoped dashboards, audit logging, K8s deployment
RBAC (api/internal/authz) is live on /query and /dashboards, backed by a new enterprise/ module (session issuance, audit logging, RBAC storage, OIDC/SAML protocol wiring) that core never imports -- only calls over HTTP. Found and fixed a real cross-tenant vulnerability in dashboards (no tenant_id filtering at all) while writing the threat model doc. Two things are explicitly NOT done, documented rather than hidden: tenant isolation for log data itself (/query still shares one ClickHouse connection and Tantivy index across every tenant -- RBAC controls who can query, not what a query can see), and human SSO login (protocol wiring exists, no HTTP handler calls it yet). See docs/security/threat-model.md and docs/phase-4-runbook.md. Also adds deploy/ (Go Operator + Helm chart, validated offline only -- no cluster was reachable in this environment). |