Files
cairnobs/agent/README.md
T
jcoffey-dev cd8aa290ca Phase 1: Windows log collection + full-text search
Extends the agent, ingest, storage, api, and web with Windows Event
Log/ETW sourcing and Tantivy-backed free-text search, per the approved
Phase 1 plan.

- CLAUDE.md: materialized on disk (never existed as a file before) with
  a new Phase 1 "done looks like" section.
- agent: Windows Event Log (EvtSubscribe) and ETW sources, Windows
  service wrapper (install/uninstall/run-service), both feature- and
  target_os-gated so Linux builds/tests/clippy stay unaffected. Also
  fixed two pre-existing Phase 0 clippy gaps (dead-code on
  default-features-only builds, a type-inference edge case) found while
  testing every feature combination properly for the first time.
  UNVERIFIED on real Windows -- no Windows toolchain existed anywhere in
  the build environment; flagged prominently in three places.
- proto/ingest: new record_id field, assigned once server-side in
  ingest's gRPC front end so ClickHouse and Tantivy agree on the same ID
  for the same record.
- storage: record_id column + bloom filter index, verified against a
  live ClickHouse.
- search: new service, Tantivy index, rskafka consumer as an independent
  second consumer group on the same Redpanda topic ingest already reads.
- api/web: new /search endpoint and page, sharing the query page's
  result-table shape and component.
- hack/windows-fixture: sends realistic Windows-shaped data straight to
  ingest, so the pipeline's handling of it is verifiable without a
  Windows host.

Verified end-to-end on the live docker-compose stack: the same record_id
comes back from both /query and /search for the same log line, including
for windows-fixture's synthetic Windows Event Log data. Real bugs found
and fixed along the way: api/Dockerfile missing proto/ in its build
context, search's logs being completely silent (RUST_LOG gap), and
search/target/ missing from .gitignore/.dockerignore.
2026-08-13 11:27:35 -07:00

231 lines
9.7 KiB
Markdown

# sentry-agent
Distro-agnostic Linux/Windows log collector. On Linux, statically linked
against musl, no glibc runtime dependency. Tails journald (Linux default),
a file, Windows Event Log, or ETW, batches lines, and ships them over mTLS
gRPC to the ingest service.
**Windows support status:** the Windows-specific code
(`source/windows_eventlog.rs`, `source/etw.rs`, `service.rs`) was written
against documented Win32/ETW API shapes but has **not been compiled or run
on Windows** — no Windows toolchain was available in the environment this
was built in (confirmed: only the Linux target's std library was
installed, no way to even `cargo check --target x86_64-pc-windows-*`).
Linux builds/tests/clippy are verified clean across every feature
combination; Windows code is a first draft to compile-check and test for
real before trusting it. See `/docs/phase-1-runbook.md`.
## Workspace layout
- `sentry-parser` — pure-`std` RFC 5424 syslog parser with raw-passthrough
fallback. No I/O, easy to unit test in isolation.
- `sentry-agent` — the binary: config loading, sourcing (journald/file/
Windows Event Log/ETW), batching, mTLS gRPC client, Windows service
wrapper.
## Why one crate for both platforms, not a platform split
`config.rs`, `batch.rs`, `grpc.rs`, and `main.rs`'s event loop are already
100% cross-platform Rust — nothing in them is Linux- or Windows-specific.
Only the `source/` modules differ per platform, and that boundary already
existed before Windows support was added (it's exactly what made adding
Windows sources a matter of adding two files, not restructuring anything).
Windows-only dependencies (`windows`, `windows-service`, `quick-xml`) live
in a `[target.'cfg(windows)'.dependencies]` section in `Cargo.toml`, so
they're not in the Linux build's dependency graph at all — no crate split
needed to keep the two platforms from stepping on each other.
## Why journalctl, not libsystemd
The journald source shells out to `journalctl -f -o json` rather than
linking `libsystemd` via FFI. Statically linking libsystemd into a musl
binary is fragile — it pulls in dbus/libcap transitively and isn't designed
for static linking — and would undermine the no-glibc-runtime-deps goal
even where technically possible. `journalctl` ships on every systemd distro
this agent targets, so shelling out sidesteps the problem entirely. See
`/docs/architecture.md`.
## Building
Native build (whatever target your machine is):
```sh
cargo build --release
```
musl targets (what actually ships):
```sh
rustup target add x86_64-unknown-linux-musl aarch64-unknown-linux-musl
# x86_64: works with musl-gcc installed locally (musl-tools on Debian,
# musl on Arch, etc.) — the musl target is fully static by default.
cargo build --release --target x86_64-unknown-linux-musl
# aarch64 cross-compilation needs a cross toolchain; the boring, reliable
# option is `cross` (https://github.com/cross-rs/cross), which builds
# inside a Docker container with the right linker preinstalled:
cross build --release --target aarch64-unknown-linux-musl
```
Building requires `protoc` on PATH (used by `tonic-build`/`prost-build` at
compile time to generate the gRPC client from `/proto/sentry/logs/v1/logs.proto`).
Container build (see caveat below):
```sh
# from the repo root, not agent/
docker build -f agent/Dockerfile -t sentry-agent .
```
**Caveat:** the container image is provided for CI/completeness, but
journald sourcing needs `journalctl` and access to the host journal —
neither of which exist in the `scratch` image or are available to a
container without deliberately bind-mounting `/var/log/journal` (or
`/run/log/journal`) and the `journalctl` binary in. The intended Phase 0
deployment for journald sourcing is as a native binary managed by systemd
on the host, not containerized.
### Building for Windows
```sh
# Cross-compiling FROM Linux, for the build step only:
rustup target add x86_64-pc-windows-gnu
cargo build --release --target x86_64-pc-windows-gnu \
--no-default-features --features windows-eventlog,etw
# Natively on Windows (MSVC toolchain):
cargo build --release --target x86_64-pc-windows-msvc \
--no-default-features --features windows-eventlog,etw
```
`--no-default-features` matters: the default feature set is `journald`,
which is Linux-only (the module is `target_os = "linux"`-gated and simply
won't compile in on Windows, but there's no reason to carry the dead
feature flag). Drop `,etw` from `--features` if you only want Event Log —
see the privilege note below for why most environments will want to.
**Cross-compilation only covers the *build* step.** Running/testing the
Windows sources — actually calling `EvtSubscribe`, starting an ETW
session, registering a Windows service — needs a real or virtualized
Windows host. There is no way around that, and nothing in this repo
pretends otherwise; see `/docs/phase-1-runbook.md` for exactly what's
automatable vs. manual-only.
## Running
No CLI flags are required for the common case:
```sh
./sentry-agent
```
This uses the platform's conventional config path if present
(`/etc/sentry-agent/agent.toml` on Linux, `C:\ProgramData\SentryAgent\agent.toml`
on Windows), otherwise built-in defaults: journald source on Linux (whole
journal, no unit filter), service name `default`, and mTLS material
expected under the same conventional directory
(`{ca,client,client-key}.pem`). mTLS is mandatory per the project's
transport requirements, so a from-scratch run with no certs in place will
fail fast with a clear error rather than connecting insecurely.
See `config/agent.example.toml` for all fields.
```sh
./sentry-agent --config /path/to/agent.toml
```
## Running as a Windows service
"A native Windows service, not a WSL wrapper" means implementing the Win32
Service Control Manager protocol, not just running the binary in a
console — that's what `service.rs` (via the `windows-service` crate)
does. From an administrator shell:
```powershell
sentry-agent.exe install # registers the service, Automatic start, LocalSystem account
sc.exe start SentryAgent
sc.exe stop SentryAgent
sentry-agent.exe uninstall
```
`install`/`uninstall`/`run-service` are subcommands only present in
Windows builds (`sentry-agent` with no subcommand is still the normal
foreground/console run, same as on Linux) — `run-service` specifically is
what the SCM itself invokes at service start; don't run it directly.
**Known limitation:** when running as a service, there's no console
attached, so `tracing_subscriber::fmt()`'s stdout writer has nowhere to
go — logs won't be visible anywhere useful until this is redirected to a
file or a proper Windows Event Log tracing sink is written. Not addressed
in Phase 1; flagging it here rather than shipping it silently broken.
## ETW: read this before enabling it
ETW needs elevated privileges to subscribe to most providers — running
the agent under an administrator token or a service account with
`SeSystemProfilePrivilege`/ETW-specific rights. This is a real privilege
escalation, not a footnote: think about whether your environment wants
the log-shipping agent running with that level of access before turning
on the `etw` feature and an `[source] kind = "etw"` config. Event Log
alone (no elevated privileges needed) covers the common case and is what
Phase 1's exit criteria in `/CLAUDE.md` actually requires to be running.
Providers are configured by **GUID**, not friendly name — ETW's own API
requires it. Look one up with `logman query providers "<Friendly Name>"`.
## Windows Event Forwarding (WEF)
Two different things people mean by "WEF support," worth being explicit
about since they're very different amounts of work:
1. **What this repo supports today, with zero extra code:** WEF is a
native Windows-to-Windows mechanism (`wecsvc`, the built-in Windows
Event Collector role) — endpoints forward to a Windows Server acting
as collector using Windows' own mechanism, no Sentry code involved in
the forwarding itself. Run this agent *on the collector box*,
subscribed to the `ForwardedEvents` channel instead of the usual three:
```toml
[source]
kind = "eventlog"
channels = ["ForwardedEvents"]
```
2. **What this repo does *not* implement:** a true agentless receiver —
Sentry itself speaking the WS-Management/WinRM event-subscription
protocol so endpoints can forward directly to `ingest` without any
Windows Event Collector role or Sentry agent anywhere. That's a
standalone protocol implementation (SOAP-ish subscription/heartbeat/
delivery over WinRM), not an agent or ingest-side tweak, and it's out
of scope for Phase 1. If you need this, it's a real project of its
own — say so before assuming it's a small addition.
## Testing
```sh
cargo test --workspace
```
## Feature flags
- `journald` (default) — journalctl-based journald source. `target_os =
"linux"`-gated: enabling this on a Windows build is a no-op, not a
build failure.
- `file-tail` — polling-based file tailer (no inotify dependency; doesn't
follow rename-based log rotation yet). Cross-platform, works on Windows
too.
- `windows-eventlog` — Windows Event Log via `EvtSubscribe`.
`target_os = "windows"`-gated the same way; a no-op on Linux.
- `etw` — ETW real-time session. Same gating. See the privilege section
above before enabling.
Any combination can be enabled together; `[source].kind` in config picks
which one actually runs. Building without a feature and configuring that
source at runtime fails at startup with a clear error rather than
silently doing nothing.
Dependencies added for Windows support, worth knowing about:
`windows` (Microsoft's official Win32/ETW bindings), `windows-service`
(Windows Service Control Manager wrapper), `quick-xml` (parses
EvtSubscribe's rendered event XML). All three are `[target.'cfg(windows)'.dependencies]`
— not in the Linux build's dependency graph at all.