jcoffey-dev
4b5dae5879
This is a large squashed commit covering two batches of prior uncommitted work plus a full security-audit remediation pass, kept together because go.mod/go.sum and several shared files (main.go, handler.go) were touched by both and splitting risked non-building intermediate commits. Features (built earlier, previously uncommitted): - Local username/password login for single-tenant deployments with no SSO configured (api/localauth, alerting/internal/sessioncheck, sentryctl users, web/src/routes/login, metadata migrations 0040/0041). - Remotely-editable additional log file paths for agents, on top of their existing primary source (api/agents, agent/sentry-agent extra-file-path diffing, web agent config UI). - IPv4/IPv6 addresses reported alongside other host system metrics. Security audit remediation (this pass, all live-verified in production): - Critical: block ClickHouse SSRF table functions (url/remote/file/s3/...) in the raw-SQL query escape hatch. - High: deny sensitive paths and require Admin to add agent extra_file_paths (Editor could previously point an agent at /etc/shadow or an SSH key); alerting webhook targets now validate against internal/metadata/loopback addresses, both at creation and send time; alerting's session middleware now enforces an Editor+ floor on mutating requests instead of "any authenticated session"; bumped goxmldsig to close a SAML signature-verification bypass (GO-2026-4753). - Medium: per-IP login rate limiting; security response headers (HSTS/CSP/nosniff/X-Frame-Options/Referrer-Policy/Permissions-Policy) on web/nginx.conf; a DevCredentialWarnings check in every Go service's config loader, logging loudly at startup if a deployment is still on docker-compose.yml's literal dev-only credentials; dependency bumps (golang.org/x/text, grpc, x/net, quick-xml, h2) across every affected Go module and both Rust crates, including a previously-uncovered x/net vulnerability in deploy/operator; a new security-scan.yml CI workflow running cargo-deny/govulncheck/npm-audit, mirroring the existing license-compliance.yml matrix shape. - Low: removed sentryctl's plaintext --password flag (shell history/`ps` exposure) in favor of stdin and a --password-stdin flag for reset-password's optional specific-password path; a dummy bcrypt comparison closes a login response-time username-enumeration side-channel.
api
Sentry's query API: a single POST /query endpoint accepting either the
pipe syntax or raw SQL, compiled and routed across ClickHouse and Tantivy
by internal/querylang. Replaces Phase 0/1's two separate placeholder
endpoints (raw-SQL-only /query, free-text-only /search) — see
/docs/query-language-design.md for the grammar, IR, and routing design,
and /docs/query-language-reference.md for the user-facing syntax.
Why plain REST, not gRPC + REST gateway
CLAUDE.md pins the control plane to "Go, gRPC + REST gateway." This
service is plain net/http instead — a deliberate simplification, not a
change to the pinned stack. Wiring up a .proto service,
google.api.http annotations, and protoc-gen-grpc-gateway codegen for
one endpoint doesn't buy much at this size. api does speak gRPC
internally — to /search — this simplification is about the
public-facing surface only.
Endpoint
POST /query — body {"query": "...", "language": ""}, response
{"columns": [...], "rows": [[...], ...]} or {"error": "..."}.
queryis either pipe syntax (service=api | where status>=500 | stats count by host) or raw SQL (SELECT ...). Auto-detected by whether the query starts withSELECT(case-insensitive).languageoptionally overrides detection:"sql"or"spl". Exists for the rare case a pipe query legitimately starts with the literal word "select" as a bare search term.- Both syntaxes compile to the same
querylang/ir.Planand execute through the same code path — seeinternal/querylang/executorfor the four routing cases (pure ClickHouse; Tantivy prefilter + ClickHouse rows; Tantivy prefilter + ClickHouse aggregation; raw SQL passthrough).
GET /healthz — for docker-compose/k8s liveness checks.
No auth. Not scoped yet — don't expose this beyond a trusted dev/homelab network.
Configuration
Environment variables (see internal/config/config.go):
| Var | Default | Purpose |
|---|---|---|
HTTP_LISTEN_ADDR |
:8080 |
|
CLICKHOUSE_ADDR |
localhost:9000 |
Native protocol port |
CLICKHOUSE_DATABASE / _USERNAME / _PASSWORD |
sentry / default / `` |
|
SEARCH_GRPC_ADDR |
localhost:50052 |
Must match /search's GRPC_LISTEN_ADDR |
QUERY_TIMEOUT_SECONDS |
30 |
Per-request timeout |
CORS_ALLOWED_ORIGIN |
* |
Wide open by default since there's no auth yet; tighten together |
searchclient.Dial connects to /search over plain TCP, no TLS — same
trust boundary as api's existing plain-TCP connection to ClickHouse.
mTLS in this project is specifically the agent↔ingest edge boundary, not
every internal hop.
Building & testing
go build ./...
go vet ./...
go test ./...
# from the repo root, not api/
docker build -f api/Dockerfile -t sentry-api .
Testing notes
internal/queryapi's HTTP handler depends on ClickHouse and /search
only through the narrow interfaces querylang/executor defines
(SQLRunner, SearchClient), so routing, compilation, JSON encoding,
and error-status mapping are all unit-tested against fakes — no live
ClickHouse or /search instance needed, and the real lexer/parser/
planner run unmocked in these tests, only the backends are faked. See
internal/querylang's own package docs for how compilation and
execution are tested independently of each other. executor.ChRunner
(the reflection-based row scanning against ClickHouse's driver.Rows)
and internal/searchclient's actual gRPC dial are not unit-tested — the
former because faking driver.Rows fully would be significant
test-only scaffolding the driver's own docs say isn't meant to be
implemented by adopters; the latter because it's a thin wrapper with
nothing but wiring to test. Both are exercised end-to-end via the
docker-compose flow in /docs/phase-2-runbook.md.