Scaffold Phase 0: agent -> Redpanda -> ingest -> ClickHouse -> api -> web
End-to-end log pipeline for Linux hosts, per /docs/architecture.md: - proto: shared gRPC contract (agent <-> ingest), Go bindings checked in - agent: Rust, musl-targeted, journald/file sourcing, RFC5424 parser, mTLS gRPC client, no required config for the common case - ingest: Go, single binary with --mode server|consumer|all; gRPC front end forwards to Redpanda unchanged, consumer normalizes and batch-writes to ClickHouse with at-least-once delivery - storage: ClickHouse schema + a plain SQL-file migration runner - api: minimal SELECT-only query endpoint, plain REST (not gRPC+gateway yet -- see api/README.md) - web: SvelteKit static SPA, one query page - transport: Redpanda compose + topic provisioning - cli: sentryctl ping stub - hack/dev-certs: throwaway CA + cert generation for local mTLS - root docker-compose.yml + docs/phase-0-runbook.md tie it together Not yet run end-to-end against real Docker/ClickHouse/Redpanda -- see the runbook's caveats section before relying on this working as-is.
This commit is contained in:
@@ -0,0 +1,26 @@
|
||||
# transport
|
||||
|
||||
Redpanda for local development, plus the script that provisions the topic
|
||||
`/ingest` depends on.
|
||||
|
||||
## Topic naming contract
|
||||
|
||||
`ingest` defaults to `REDPANDA_TOPIC=sentry.logs.raw` (see
|
||||
`/ingest/internal/config`). `provision-topics.sh` defaults to the same
|
||||
name. These aren't wired together automatically — if you change one,
|
||||
change the other, or override `REDPANDA_TOPIC` consistently wherever
|
||||
both are invoked.
|
||||
|
||||
## Running standalone
|
||||
|
||||
```sh
|
||||
docker compose up -d
|
||||
REDPANDA_BROKERS=localhost:9092 ./provision-topics.sh
|
||||
```
|
||||
|
||||
## In the full stack
|
||||
|
||||
The root-level `docker-compose.yml` builds this directory's `Dockerfile`
|
||||
(FROM the Redpanda image itself, so `rpk` is already present) as a
|
||||
one-shot init service that runs after Redpanda reports healthy. See
|
||||
`/docs/phase-0-runbook.md`.
|
||||
Reference in New Issue
Block a user