Scaffold Phase 0: agent -> Redpanda -> ingest -> ClickHouse -> api -> web

End-to-end log pipeline for Linux hosts, per /docs/architecture.md:

- proto: shared gRPC contract (agent <-> ingest), Go bindings checked in
- agent: Rust, musl-targeted, journald/file sourcing, RFC5424 parser,
  mTLS gRPC client, no required config for the common case
- ingest: Go, single binary with --mode server|consumer|all; gRPC front
  end forwards to Redpanda unchanged, consumer normalizes and
  batch-writes to ClickHouse with at-least-once delivery
- storage: ClickHouse schema + a plain SQL-file migration runner
- api: minimal SELECT-only query endpoint, plain REST (not gRPC+gateway
  yet -- see api/README.md)
- web: SvelteKit static SPA, one query page
- transport: Redpanda compose + topic provisioning
- cli: sentryctl ping stub
- hack/dev-certs: throwaway CA + cert generation for local mTLS
- root docker-compose.yml + docs/phase-0-runbook.md tie it together

Not yet run end-to-end against real Docker/ClickHouse/Redpanda -- see the
runbook's caveats section before relying on this working as-is.
This commit is contained in:
2026-08-13 08:25:19 -07:00
commit b6b092c912
92 changed files with 7796 additions and 0 deletions
+34
View File
@@ -0,0 +1,34 @@
# Standalone Redpanda for local development against /transport in
# isolation. The root-level docker-compose.yml runs the full Phase 0 stack
# and defines its own redpanda service separately — this file is not
# included by it.
#
# Note advertise-kafka-addr is "localhost" here (host tools connect via the
# mapped port), vs "redpanda" in the root compose (other containers connect
# via the compose network's service DNS name). Getting this wrong is the
# classic Redpanda/Kafka docker-compose footgun — clients can connect
# initially but then fail on the broker's advertised address once they try
# to actually produce/consume.
services:
redpanda:
image: docker.redpanda.com/redpandadata/redpanda:v24.2.7
container_name: sentry-redpanda
command:
- redpanda
- start
- --smp=1
- --memory=1G
- --reserve-memory=0M
- --overprovisioned
- --node-id=0
- --check=false
- --kafka-addr=PLAINTEXT://0.0.0.0:9092
- --advertise-kafka-addr=PLAINTEXT://localhost:9092
ports:
- "9092:9092"
- "9644:9644" # admin API, used by rpk/healthchecks
volumes:
- redpanda-data:/var/lib/redpanda/data
volumes:
redpanda-data: