jcoffey-dev
fe854b1091
Both surfaced only by running docker compose up for real, not from review: - ClickHouse's official image silently disables network access for the default user unless CLICKHOUSE_USER or CLICKHOUSE_PASSWORD is set to a genuinely non-empty value (an explicit empty password still triggers it). Set a dev-only password across clickhouse, clickhouse-migrate, ingest, and api in both docker-compose.yml files. - rpk cluster health and rpk topic ... don't accept --brokers; health checks need -X admin.hosts=... (port 9644), topic commands need -X brokers=... (port 9092). The old script's retry loop silently swallowed the resulting "unknown flag" error and retried forever, which blocked ingest from ever starting. Verified: agent -> ingest -> Redpanda -> ClickHouse -> api round-trip confirmed with a real log line on a real host.
95 lines
3.9 KiB
Markdown
95 lines
3.9 KiB
Markdown
# storage
|
|
|
|
ClickHouse schema and migration tooling for Sentry's analytical store.
|
|
|
|
## Schema
|
|
|
|
One table for Phase 0, `logs`:
|
|
|
|
```sql
|
|
CREATE TABLE logs
|
|
(
|
|
`timestamp` DateTime64(9, 'UTC'),
|
|
`host` String,
|
|
`service` String,
|
|
`severity` LowCardinality(String),
|
|
`message` String,
|
|
`attributes` Map(String, String)
|
|
)
|
|
ENGINE = MergeTree
|
|
PARTITION BY toDate(timestamp)
|
|
ORDER BY (service, timestamp)
|
|
```
|
|
|
|
Notes on choices that weren't fully specified by the task description:
|
|
|
|
- **`DateTime64(9, 'UTC')`** (nanosecond precision) rather than second or
|
|
millisecond precision, to match the agent's `timestamp_unix_nano` field
|
|
end to end without truncation.
|
|
- **`severity` as `LowCardinality(String)`**, not a numeric OTel
|
|
`SeverityNumber`. `/ingest`'s `normalize` package writes short text
|
|
values (`TRACE`/`DEBUG`/`INFO`/`WARN`/`ERROR`/`FATAL`/`UNSPECIFIED`).
|
|
`LowCardinality` gets you most of the storage/query efficiency of an enum
|
|
without committing to one at the schema level. Splitting into a proper
|
|
`SeverityNumber` + `SeverityText` pair (full OTel shape) is one of the
|
|
open questions already flagged in `/docs/architecture.md`.
|
|
- **`PARTITION BY toDate(timestamp)`** (daily partitions) and
|
|
**`ORDER BY (service, timestamp)`** are exactly what the task asked for
|
|
— service-scoped queries over a time range are the dominant access
|
|
pattern this is optimized for.
|
|
- No TTL/retention clause yet — also an open question in architecture.md,
|
|
deferred until storage sizing is a real concern.
|
|
|
|
## Migration tooling: a plain SQL-file runner, not golang-migrate
|
|
|
|
`migrate.sh` applies `migrations/*.sql` in filename order over
|
|
ClickHouse's HTTP interface, tracking what's applied in a
|
|
`schema_migrations` table. Chosen over `golang-migrate` for Phase 0
|
|
because there's exactly one migration to run — pulling in a migration
|
|
framework (another dependency, another thing to configure/vendor) for a
|
|
single `CREATE TABLE` is exactly the kind of premature machinery this
|
|
project's conventions say to avoid. Revisit `golang-migrate` once there's
|
|
real schema churn across environments (rollback support, checksums,
|
|
concurrent-apply safety become worth their cost at that point, not before).
|
|
|
|
**Convention:** one DDL statement per migration file. The ClickHouse HTTP
|
|
interface isn't reliably multi-statement, so `migrate.sh` doesn't try to
|
|
split multi-statement files — keep each migration to a single statement.
|
|
|
|
## Running
|
|
|
|
```sh
|
|
docker compose up -d # starts a standalone ClickHouse for local work
|
|
CLICKHOUSE_PASSWORD=sentry-dev-only ./migrate.sh # applies migrations/*.sql
|
|
```
|
|
|
|
`CLICKHOUSE_PASSWORD` here has to match whatever `docker-compose.yml` set
|
|
on the `clickhouse` service — found this the hard way running the Phase 0
|
|
runbook for real: the official ClickHouse image silently disables *all*
|
|
network access (including the published port, not just container-to-
|
|
container traffic) for the `default` user unless `CLICKHOUSE_USER` or
|
|
`CLICKHOUSE_PASSWORD` is a genuinely non-empty value. An empty
|
|
`CLICKHOUSE_PASSWORD=""` still triggers the lockdown — it has to actually
|
|
have a value. Not a real secret, just what this image demands.
|
|
|
|
Environment variables `migrate.sh` reads (all optional except
|
|
`CLICKHOUSE_PASSWORD` as of the above, matching `/ingest`'s ClickHouse
|
|
defaults so the two stay in sync out of the box):
|
|
|
|
| Var | Default |
|
|
|---|---|
|
|
| `CLICKHOUSE_HTTP` | `http://localhost:8123` |
|
|
| `CLICKHOUSE_USER` | `default` |
|
|
| `CLICKHOUSE_PASSWORD` | (empty — override, see above) |
|
|
| `CLICKHOUSE_DATABASE` | `sentry` |
|
|
|
|
There's also a `Dockerfile` (bash + curl baked in, `migrations/` copied in
|
|
at build time) used by the root-level `docker-compose.yml` as a one-shot
|
|
init service — no runtime package install, no host volume mount needed.
|
|
|
|
## Adding a migration
|
|
|
|
Add `migrations/000N_description.sql` with the next sequential number and
|
|
a single DDL statement. `migrate.sh` picks it up automatically — no
|
|
registration step.
|