Both surfaced only by running docker compose up for real, not from review:
- ClickHouse's official image silently disables network access for the
default user unless CLICKHOUSE_USER or CLICKHOUSE_PASSWORD is set to a
genuinely non-empty value (an explicit empty password still triggers
it). Set a dev-only password across clickhouse, clickhouse-migrate,
ingest, and api in both docker-compose.yml files.
- rpk cluster health and rpk topic ... don't accept --brokers; health
checks need -X admin.hosts=... (port 9644), topic commands need
-X brokers=... (port 9092). The old script's retry loop silently
swallowed the resulting "unknown flag" error and retried forever,
which blocked ingest from ever starting.
Verified: agent -> ingest -> Redpanda -> ClickHouse -> api round-trip
confirmed with a real log line on a real host.
End-to-end log pipeline for Linux hosts, per /docs/architecture.md:
- proto: shared gRPC contract (agent <-> ingest), Go bindings checked in
- agent: Rust, musl-targeted, journald/file sourcing, RFC5424 parser,
mTLS gRPC client, no required config for the common case
- ingest: Go, single binary with --mode server|consumer|all; gRPC front
end forwards to Redpanda unchanged, consumer normalizes and
batch-writes to ClickHouse with at-least-once delivery
- storage: ClickHouse schema + a plain SQL-file migration runner
- api: minimal SELECT-only query endpoint, plain REST (not gRPC+gateway
yet -- see api/README.md)
- web: SvelteKit static SPA, one query page
- transport: Redpanda compose + topic provisioning
- cli: sentryctl ping stub
- hack/dev-certs: throwaway CA + cert generation for local mTLS
- root docker-compose.yml + docs/phase-0-runbook.md tie it together
Not yet run end-to-end against real Docker/ClickHouse/Redpanda -- see the
runbook's caveats section before relying on this working as-is.
jcoffey-dev