jcoffey-dev
fe854b1091
Both surfaced only by running docker compose up for real, not from review: - ClickHouse's official image silently disables network access for the default user unless CLICKHOUSE_USER or CLICKHOUSE_PASSWORD is set to a genuinely non-empty value (an explicit empty password still triggers it). Set a dev-only password across clickhouse, clickhouse-migrate, ingest, and api in both docker-compose.yml files. - rpk cluster health and rpk topic ... don't accept --brokers; health checks need -X admin.hosts=... (port 9644), topic commands need -X brokers=... (port 9092). The old script's retry loop silently swallowed the resulting "unknown flag" error and retried forever, which blocked ingest from ever starting. Verified: agent -> ingest -> Redpanda -> ClickHouse -> api round-trip confirmed with a real log line on a real host.
35 lines
1.3 KiB
Markdown
35 lines
1.3 KiB
Markdown
# transport
|
|
|
|
Redpanda for local development, plus the script that provisions the topic
|
|
`/ingest` depends on.
|
|
|
|
## Topic naming contract
|
|
|
|
`ingest` defaults to `REDPANDA_TOPIC=sentry.logs.raw` (see
|
|
`/ingest/internal/config`). `provision-topics.sh` defaults to the same
|
|
name. These aren't wired together automatically — if you change one,
|
|
change the other, or override `REDPANDA_TOPIC` consistently wherever
|
|
both are invoked.
|
|
|
|
## Running standalone
|
|
|
|
```sh
|
|
docker compose up -d
|
|
./provision-topics.sh # defaults (localhost:9092 / localhost:9644) match this compose file
|
|
```
|
|
|
|
Two separate addresses matter here, confirmed by actually running this
|
|
against a live Redpanda container: `rpk cluster health` talks to the
|
|
**Admin API** (`REDPANDA_ADMIN_HOSTS`, port 9644), while `rpk topic ...`
|
|
talks to the **Kafka API** (`REDPANDA_BROKERS`, port 9092) — and neither
|
|
accepts a `--brokers` flag directly, both need `-X admin.hosts=...` /
|
|
`-X brokers=...`. Get this wrong and it doesn't error loudly: it just
|
|
retries the health check forever without ever reporting why.
|
|
|
|
## In the full stack
|
|
|
|
The root-level `docker-compose.yml` builds this directory's `Dockerfile`
|
|
(FROM the Redpanda image itself, so `rpk` is already present) as a
|
|
one-shot init service that runs after Redpanda reports healthy. See
|
|
`/docs/phase-0-runbook.md`.
|