Build the Phase 8 conformance corpus

The design argues the conformance suite is the specification and should be built before either implementation, since two hand-written implementations of one language diverge unless something shared pins them. This is that suite: 38 cases in /processing, a language-neutral top-level directory for the same reason /proto is one -- the Rust agent and the Go ingest tier both consume the definition and neither owns it. Nothing executes the cases, because neither implementation exists. A stdlib-only validator checks the corpus stays well-formed and runs in CI: known actions, addressable fields, compilable patterns, names matching filenames, and no case depending on record_id, which is withheld so the question of whether an ingest-side rule can see one stays open. The validator was checked against seven deliberately broken cases before being trusted, since "38/38 valid" means nothing from a validator that cannot fail. Writing the cases first has already paid for itself twice. It forced two determinism decisions the prose had left vague, both of which a conformance suite cannot avoid answering. Sampling is counter-based rather than random: random is statistically nicer and impossible to assert on. Windows are measured on record timestamps rather than wall-clock, which makes replay deterministic and, not incidentally, makes backfill behave correctly where a wall-clock window would not. And it made the missing aggregate_count answer concrete. The design does not say what that action emits, or what a query not expecting a synthetic record sees. Rather than invent one by writing cases, the validator rejects any case using it, so the design question has to be answered before the behaviour can be frozen by accident. The corpus includes the acceptance case from real measured data: the two processes that account for roughly 60% of a real workstation's journal volume, and the one kernel message worth keeping. Signed-off-by: John Coffey <[email protected]>
2026-09-04 20:19:51 -07:00
parent 8787c1d087
commit 7fabc6a067
43 changed files with 2360 additions and 3 deletions
@@ -0,0 +1,103 @@
+{
+  "name": "workstation_noise_reduction",
+  "description": "The v1 acceptance shape, from real measured data: Discord repeating a byte-identical version string and a polling daemon repeating a fixed message, against one kernel message worth keeping. See /docs/phase-8-processing-design.md.",
+  "rules": [
+    {
+      "match": [
+        {
+          "field": "message",
+          "op": "eq",
+          "value": "Discord 1.0.155"
+        }
+      ],
+      "actions": [
+        {
+          "action": "drop"
+        }
+      ]
+    },
+    {
+      "match": [
+        {
+          "field": "message",
+          "op": "eq",
+          "value": "Checking active tasks"
+        }
+      ],
+      "actions": [
+        {
+          "action": "suppress_duplicates",
+          "window_ms": 60000
+        }
+      ]
+    }
+  ],
+  "inputs": [
+    {
+      "timestamp_unix_nano": 0,
+      "host": "h1",
+      "service": "s1",
+      "severity": "SEVERITY_INFO",
+      "message": "Discord 1.0.155",
+      "attributes": {}
+    },
+    {
+      "timestamp_unix_nano": 1000000000,
+      "host": "h1",
+      "service": "s1",
+      "severity": "SEVERITY_INFO",
+      "message": "Checking active tasks",
+      "attributes": {}
+    },
+    {
+      "timestamp_unix_nano": 2000000000,
+      "host": "h1",
+      "service": "s1",
+      "severity": "SEVERITY_INFO",
+      "message": "Discord 1.0.155",
+      "attributes": {}
+    },
+    {
+      "timestamp_unix_nano": 3000000000,
+      "host": "h1",
+      "service": "s1",
+      "severity": "SEVERITY_INFO",
+      "message": "Checking active tasks",
+      "attributes": {}
+    },
+    {
+      "timestamp_unix_nano": 4000000000,
+      "host": "h1",
+      "service": "s1",
+      "severity": "SEVERITY_WARN",
+      "message": "[UFW BLOCK] IN=wlan0 SRC=172.16.16.1",
+      "attributes": {}
+    },
+    {
+      "timestamp_unix_nano": 5000000000,
+      "host": "h1",
+      "service": "s1",
+      "severity": "SEVERITY_INFO",
+      "message": "Discord 1.0.155",
+      "attributes": {}
+    }
+  ],
+  "expect": [
+    {
+      "timestamp_unix_nano": 1000000000,
+      "host": "h1",
+      "service": "s1",
+      "severity": "SEVERITY_INFO",
+      "message": "Checking active tasks",
+      "attributes": {}
+    },
+    {
+      "timestamp_unix_nano": 4000000000,
+      "host": "h1",
+      "service": "s1",
+      "severity": "SEVERITY_WARN",
+      "message": "[UFW BLOCK] IN=wlan0 SRC=172.16.16.1",
+      "attributes": {}
+    }
+  ]
+}