coffey-labs/cairnobs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Phase 1: Windows log collection + full-text search

Browse Source 
Extends the agent, ingest, storage, api, and web with Windows Event
Log/ETW sourcing and Tantivy-backed free-text search, per the approved
Phase 1 plan.

- CLAUDE.md: materialized on disk (never existed as a file before) with
  a new Phase 1 "done looks like" section.
- agent: Windows Event Log (EvtSubscribe) and ETW sources, Windows
  service wrapper (install/uninstall/run-service), both feature- and
  target_os-gated so Linux builds/tests/clippy stay unaffected. Also
  fixed two pre-existing Phase 0 clippy gaps (dead-code on
  default-features-only builds, a type-inference edge case) found while
  testing every feature combination properly for the first time.
  UNVERIFIED on real Windows -- no Windows toolchain existed anywhere in
  the build environment; flagged prominently in three places.
- proto/ingest: new record_id field, assigned once server-side in
  ingest's gRPC front end so ClickHouse and Tantivy agree on the same ID
  for the same record.
- storage: record_id column + bloom filter index, verified against a
  live ClickHouse.
- search: new service, Tantivy index, rskafka consumer as an independent
  second consumer group on the same Redpanda topic ingest already reads.
- api/web: new /search endpoint and page, sharing the query page's
  result-table shape and component.
- hack/windows-fixture: sends realistic Windows-shaped data straight to
  ingest, so the pipeline's handling of it is verifiable without a
  Windows host.

Verified end-to-end on the live docker-compose stack: the same record_id
comes back from both /query and /search for the same log line, including
for windows-fixture's synthetic Windows Event Log data. Real bugs found
and fixed along the way: api/Dockerfile missing proto/ in its build
context, search's logs being completely silent (RUST_LOG gap), and
search/target/ missing from .gitignore/.dockerignore.
This commit is contained in:
jcoffey-dev
2026-08-13 11:27:35 -07:00
parent fe854b1091
commit cd8aa290ca
66 changed files with 6084 additions and 171 deletions
Download Patch File Download Diff File Expand all files Collapse all files
hack/README.md
+5
View File
@@ -14,3 +14,8 @@ monorepos (Kubernetes among them).
- `dev-certs/` — generates a throwaway CA + server/client cert pair for
local mTLS between the agent and ingest. See `/docs/phase-0-runbook.md`
for when to run it.
- `windows-fixture/` — sends synthetic Windows Event Log-shaped records
directly to `ingest`, bypassing the real Windows agent. Tests whether
the pipeline handles Windows-shaped data; doesn't test the real
`EvtSubscribe`/ETW integration, which needs actual Windows. See
`/docs/phase-1-runbook.md`.
hack/windows-fixture/README.md
+52
View File
@@ -0,0 +1,52 @@
# windows-fixture
Sends synthetic Windows Event Log-shaped `PushBatchRequest`s directly to
`ingest`'s gRPC endpoint, bypassing the actual Windows agent entirely.
## What this does and doesn't test
**Tests:** can the pipeline (ingest → ClickHouse → search → api → web)
correctly handle Windows-*shaped* data — the `winevt.*` attributes, the
`record_id` join between SQL and full-text search, Windows severity
levels mapping onto the right column values? This is exactly what's
automatable without a Windows host, and it's genuinely exercised: five
realistic, well-known Windows events (failed/successful logon, a service
state change, an application crash, an unexpected reboot) with real
EventIDs and providers.
**Does not test:** whether the real Windows agent's `EvtSubscribe`/ETW
integration actually works, whether Windows service registration
succeeds, whether ETW session creation/provider enabling works. Those are
fundamentally different questions — they need a real or virtualized
Windows host, and nothing here pretends otherwise. See
`/docs/phase-1-runbook.md` for exactly which is which.
## Running
Requires the docker-compose stack up (`ingest` reachable, dev certs
generated):
```sh
cd hack/windows-fixture
go run . --count 5
```
```
sent 5 synthetic Windows-shaped records, ingest accepted 5
[SEVERITY_WARN] An account failed to log on. (event_id=4625 provider=Microsoft-Windows-Security-Auditing)
...
```
Then confirm both query paths see it:
```sh
curl -s -X POST http://localhost:8080/query -H 'Content-Type: application/json' \
-d '{"sql": "SELECT host, severity, message, attributes['"'"'winevt.event_id'"'"'] AS event_id FROM logs WHERE host = '"'"'WIN-FIXTURE-01'"'"' ORDER BY timestamp DESC"}'
curl -s -X POST http://localhost:8080/search -H 'Content-Type: application/json' \
-d '{"query": "notepad"}'
```
Flags: `--addr` (default `localhost:4317`), `--ca`/`--cert`/`--key`
(default to `../dev-certs/out/{ca,client,client-key}.pem`), `--count`
(default 5, cycles through the fixed event list if higher).
hack/windows-fixture/go.mod
+18
View File
@@ -0,0 +1,18 @@
module github.com/sentry/sentry/hack/windows-fixture
go 1.25.0
replace github.com/sentry/sentry/proto => ../../proto
require (
github.com/sentry/sentry/proto v0.0.0-00010101000000-000000000000
google.golang.org/grpc v1.83.0
)
require (
golang.org/x/net v0.55.0 // indirect
golang.org/x/sys v0.45.0 // indirect
golang.org/x/text v0.37.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa // indirect
google.golang.org/protobuf v1.36.12 // indirect
)
hack/windows-fixture/go.sum
+38
View File
@@ -0,0 +1,38 @@
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/otel v1.44.0 h1:JjwHmHpA4iZ3wBxluu2fbbE7j4kqlE8jXyAyPXH7HqU=
go.opentelemetry.io/otel v1.44.0/go.mod h1:BMgjTHL9WPRlRjL2oZCBTL4whCGtXch2H4BhOPIAyYc=
go.opentelemetry.io/otel/metric v1.44.0 h1:1w0gILTcHdr3YI+ixLyjemwrVnsMURbTZFrSYCdDdmc=
go.opentelemetry.io/otel/metric v1.44.0/go.mod h1:8O7hanEPBNgEMmybD3s2VBKcgWOCsA6tzHBPODAiquo=
go.opentelemetry.io/otel/sdk v1.44.0 h1:nHYwb9lK+fJPU/dnT6s7W7Z8itMWyqrnVfbheVYrZ58=
go.opentelemetry.io/otel/sdk v1.44.0/go.mod h1:Osuydd3Se74nqjAKxid74N5eC+jfEqfTegHRnq58oK0=
go.opentelemetry.io/otel/sdk/metric v1.44.0 h1:3LlKgI+VjbVsjNRFZJZAJ30WjXC5VkNRks6si09iEfI=
go.opentelemetry.io/otel/sdk/metric v1.44.0/go.mod h1:5B5pMARnXxKhltooO4xUuCBorl65a4EpnTalObqOigA=
go.opentelemetry.io/otel/trace v1.44.0 h1:jxF5CsGYCe74MCRx2X4g7WsY/VBKRqqpNvXlX/6gtIk=
go.opentelemetry.io/otel/trace v1.44.0/go.mod h1:oLl1jrMQAVo6v3GAggN+1VH9VIz9iUSvW53sW1Q8PIE=
golang.org/x/net v0.55.0 h1:bcvxaJn3e1U6InsFWt1JUq1aSjnRxLzT2rtD2KfkDF8=
golang.org/x/net v0.55.0/go.mod h1:L5U2KuzuOe1lY7Z+aWVIKK6qEeJXnXV9yzGA+WCHJww=
golang.org/x/sys v0.45.0 h1:dO4czNzziLiiXplLQgBCEpCvXQ3dnkn0SdaZSYdQ+FY=
golang.org/x/sys v0.45.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/text v0.37.0 h1:Cqjiwd9eSg8e0QAkyCaQTNHFIIzWtidPahFWR83rTrc=
golang.org/x/text v0.37.0/go.mod h1:a5sjxXGs9hsn/AJVwuElvCAo9v8QYLzvavO5z2PiM38=
gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa h1:mZHHdPZl0dbGHCflZgAq/Q468DWVFcU2whhB2KAo8fk=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8=
google.golang.org/grpc v1.83.0 h1:JeNZEKJFbQxArAMl+hiytHauacDNqJUllNfmIMmpqnQ=
google.golang.org/grpc v1.83.0/go.mod h1:kDyl6SKsiHKt0uylY5gtn5cEjkrIOhQOGDgIc4JGwzQ=
google.golang.org/protobuf v1.36.12 h1:pJOKDDOyeXErUroCihFAd5LQuwXBSpVnKGrj5o/fwxc=
google.golang.org/protobuf v1.36.12/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
hack/windows-fixture/main.go
+130
View File
@@ -0,0 +1,130 @@
// Command windows-fixture sends synthetic Windows Event Log-shaped
// PushBatchRequests directly to ingest's gRPC endpoint, bypassing the
// actual Windows agent entirely.
//
// This tests one specific thing: can the pipeline (ingest -> ClickHouse
// -> search -> api -> web) correctly handle Windows-*shaped* data (the
// winevt.* attributes, the record_id join, severity mapping)? It does
// NOT test whether the real Windows agent's EvtSubscribe/ETW integration
// actually works -- that's a fundamentally different question that can
// only be answered on a real or virtualized Windows host. See
// /docs/phase-1-runbook.md for exactly which is which.
package main
import (
"context"
"crypto/tls"
"crypto/x509"
"flag"
"fmt"
"os"
"time"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
logsv1 "github.com/sentry/sentry/proto/sentry/logs/v1"
)
func main() {
addr := flag.String("addr", "localhost:4317", "ingest gRPC address")
caFile := flag.String("ca", "../dev-certs/out/ca.pem", "CA cert path")
certFile := flag.String("cert", "../dev-certs/out/client.pem", "client cert path")
keyFile := flag.String("key", "../dev-certs/out/client-key.pem", "client key path")
count := flag.Int("count", 5, "number of synthetic events to send")
flag.Parse()
tlsConf, err := loadTLSConfig(*caFile, *certFile, *keyFile)
if err != nil {
fmt.Fprintln(os.Stderr, "loading TLS config:", err)
os.Exit(1)
}
conn, err := grpc.NewClient(*addr, grpc.WithTransportCredentials(credentials.NewTLS(tlsConf)))
if err != nil {
fmt.Fprintln(os.Stderr, "dialing ingest:", err)
os.Exit(1)
}
defer conn.Close()
client := logsv1.NewLogIngestClient(conn)
records := syntheticWindowsRecords(*count)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
resp, err := client.PushBatch(ctx, &logsv1.PushBatchRequest{
BatchId: "windows-fixture",
Records: records,
})
if err != nil {
fmt.Fprintln(os.Stderr, "PushBatch failed:", err)
os.Exit(1)
}
fmt.Printf("sent %d synthetic Windows-shaped records, ingest accepted %d\n", len(records), resp.GetAccepted())
for _, rec := range records {
fmt.Printf(" [%s] %s (event_id=%s provider=%s)\n",
rec.GetSeverity(), rec.GetMessage(), rec.GetAttributes()["winevt.event_id"], rec.GetAttributes()["winevt.provider"])
}
}
func loadTLSConfig(caFile, certFile, keyFile string) (*tls.Config, error) {
caPEM, err := os.ReadFile(caFile)
if err != nil {
return nil, fmt.Errorf("reading CA cert %s: %w", caFile, err)
}
caPool := x509.NewCertPool()
if !caPool.AppendCertsFromPEM(caPEM) {
return nil, fmt.Errorf("no valid certificates found in %s", caFile)
}
cert, err := tls.LoadX509KeyPair(certFile, keyFile)
if err != nil {
return nil, fmt.Errorf("loading client cert/key: %w", err)
}
return &tls.Config{
RootCAs: caPool,
Certificates: []tls.Certificate{cert},
}, nil
}
// A handful of realistic, well-known Windows Event Log entries (real
// EventIDs/providers/channels), cycled through if --count exceeds the
// list length.
func syntheticWindowsRecords(count int) []*logsv1.LogRecord {
events := []struct {
eventID string
provider string
channel string
level logsv1.Severity
message string
}{
{"4625", "Microsoft-Windows-Security-Auditing", "Security", logsv1.Severity_SEVERITY_WARN, "An account failed to log on."},
{"7036", "Service Control Manager", "System", logsv1.Severity_SEVERITY_INFO, "The Windows Update service entered the running state."},
{"1000", "Application Error", "Application", logsv1.Severity_SEVERITY_ERROR, "Faulting application name: notepad.exe"},
{"4624", "Microsoft-Windows-Security-Auditing", "Security", logsv1.Severity_SEVERITY_INFO, "An account was successfully logged on."},
{"41", "Microsoft-Windows-Kernel-Power", "System", logsv1.Severity_SEVERITY_FATAL, "The system has rebooted without cleanly shutting down first."},
}
records := make([]*logsv1.LogRecord, 0, count)
for i := 0; i < count; i++ {
e := events[i%len(events)]
records = append(records, &logsv1.LogRecord{
TimestampUnixNano: time.Now().UnixNano(),
Host: "WIN-FIXTURE-01",
Service: "default",
Severity: e.level,
Message: e.message,
Attributes: map[string]string{
"winevt.event_id": e.eventID,
"winevt.provider": e.provider,
"winevt.channel": e.channel,
"winevt.computer": "WIN-FIXTURE-01",
"winevt.record_number": fmt.Sprintf("%d", 100000+i),
},
})
}
return records
}