Build the Phase 8 conformance corpus

The design argues the conformance suite is the specification and should
be built before either implementation, since two hand-written
implementations of one language diverge unless something shared pins
them. This is that suite: 38 cases in /processing, a language-neutral
top-level directory for the same reason /proto is one -- the Rust agent
and the Go ingest tier both consume the definition and neither owns it.

Nothing executes the cases, because neither implementation exists. A
stdlib-only validator checks the corpus stays well-formed and runs in
CI: known actions, addressable fields, compilable patterns, names
matching filenames, and no case depending on record_id, which is
withheld so the question of whether an ingest-side rule can see one
stays open. The validator was checked against seven deliberately broken
cases before being trusted, since "38/38 valid" means nothing from a
validator that cannot fail.

Writing the cases first has already paid for itself twice.

It forced two determinism decisions the prose had left vague, both of
which a conformance suite cannot avoid answering. Sampling is
counter-based rather than random: random is statistically nicer and
impossible to assert on. Windows are measured on record timestamps
rather than wall-clock, which makes replay deterministic and, not
incidentally, makes backfill behave correctly where a wall-clock window
would not.

And it made the missing aggregate_count answer concrete. The design
does not say what that action emits, or what a query not expecting a
synthetic record sees. Rather than invent one by writing cases, the
validator rejects any case using it, so the design question has to be
answered before the behaviour can be frozen by accident.

The corpus includes the acceptance case from real measured data: the
two processes that account for roughly 60% of a real workstation's
journal volume, and the one kernel message worth keeping.

Signed-off-by: John Coffey <[email protected]>
This commit is contained in:
2026-09-04 20:19:51 -07:00
parent 8787c1d087
commit 7fabc6a067
43 changed files with 2360 additions and 3 deletions
+12 -3
View File
@@ -153,15 +153,24 @@ the same thing in both places" is the whole promise, and two
hand-written implementations will diverge — not maybe, eventually.
The deliverable that prevents it is a **language-neutral conformance
suite**: a directory of cases, each an input record, a rule set, and the
expected output record, in JSON. Both implementations run it in their own
CI. A case is added for every bug found in either.
suite**: a directory of cases, each a rule set, a sequence of input
records, and the records expected out, in JSON. Both implementations run
it in their own CI. A case is added for every bug found in either.
This is the same discipline `/hack`'s fixtures already apply to ingest
shapes, applied to semantics instead. It should be built **first**, not
last — the suite is the specification, and the prose above is a summary
of it.
**Built:** [`/processing`](../processing/README.md), 38 cases. Nothing
executes them yet, since neither implementation exists; a structural
validator runs in CI so the corpus cannot rot in the meantime. Writing
the cases first has already paid for itself — it forced two determinism
decisions that prose had left vague (see that README's "Two determinism
decisions the suite forces"), and it made the absence of an
`aggregate_count` answer concrete enough that the validator rejects any
case using it.
## Decision 4: distribution reuses the channel that exists
Fleet management already delivers desired state