c2f74b4e1aac4f59e1142bcbb7dd78d1275b3eb8
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 jcoffey-dev
|
c2f74b4e1a |
Put apply-then-verify in v1, and specify it
It was a defensible cut while a canary might have caught a bad rollout partway through. With no canary it is the only thing that recovers a host without an operator noticing, and "total evaluation should make it unreachable" is what every crash-loop was before it happened. Specified rather than named. A two-field file is written and fsynced before rules are applied: the version being attempted, and whether it is trying, good or quarantined. The rule set itself is never written, which is what keeps the crash-loop-not-strand property -- an agent that loses the file still boots clean and re-syncs. A version found still "trying" at boot is quarantined, the agent starts with no rules, and it reports the quarantined version so the failure is visible rather than merely survived. A different version clears the quarantine, because pushing new rules is the correction. Three failure modes decided instead of discovered. An agent that cannot write the file logs once and runs with total evaluation alone: degrading to "no rules" would punish every read-only deployment for a failure that has not happened, and the backstop is best-effort by construction rather than by accident. An agent killed for an unrelated reason quarantines a blameless rule set, which is a deliberate false positive -- the alternative is claiming to distinguish "died because of the rules" from "died while they happened to be loaded", which it cannot do honestly. And a rule set fatal on only some hosts quarantines per host, which is the closest thing to a canary this design has: the first host to hit it reports while the rest carry on. Also states that the conformance corpus cannot test any of this, so nobody tries. The corpus pins rule semantics -- records in, records out. Process death and file state across restarts are not expressible that way and need agent-side tests driving a real process through crash and restart. Signed-off-by: John Coffey <[email protected]> |
||
|
jcoffey-dev
|
02ebbc86a1 |
Settle Phase 8's regex and rollout questions
regex-lite on the agent, full regex at ingest, conformance corpus limited to the syntax both accept. The agent had no regex dependency at all and the full crate is a megabyte-plus against a binary whose pitch is that it is small and static. The rejected alternative, regex ingest-side only, would have given up redacting PII before it leaves the host -- the one capability that most needs to be on the agent, and most of the reason on-host processing exists at all. Checked rather than assumed: every pattern the corpus uses compiles and behaves under regex-lite, including named captures and replace_all replacing every occurrence, which is what mask needs and what a redaction stopping at the first hit would get wrong. Both engines also resolve alternation leftmost-first, confirmed by running the same pattern through regex-lite and Go's regexp and getting the same output. A case now pins that, since it is the kind of semantic two independent implementations can differ on silently. No canary gate. An edit reaches every matching agent at once, and this design had suggested that might have to block the feature. It does not, but the risk is accepted rather than waved away: total evaluation is the real mitigation, a fatal rule set crash-loops rather than strands because overrides are never persisted, apply-then-verify makes that self-healing, and applied_override_version already shows the blast radius. What is genuinely given up is the ability to stop a bad rollout partway through -- everything else shortens the outage without preventing the rule reaching every host first. That raises the stakes on apply-then-verify, which is still open. Without a canary, total evaluation stops being a nice property of a well-designed DSL and becomes the only thing between a bad rule and every host at once, so cutting the backstop from v1 is a harder call than it looked when it was written down. Signed-off-by: John Coffey <[email protected]> |
||
|
jcoffey-dev
|
7fabc6a067 |
Build the Phase 8 conformance corpus
The design argues the conformance suite is the specification and should be built before either implementation, since two hand-written implementations of one language diverge unless something shared pins them. This is that suite: 38 cases in /processing, a language-neutral top-level directory for the same reason /proto is one -- the Rust agent and the Go ingest tier both consume the definition and neither owns it. Nothing executes the cases, because neither implementation exists. A stdlib-only validator checks the corpus stays well-formed and runs in CI: known actions, addressable fields, compilable patterns, names matching filenames, and no case depending on record_id, which is withheld so the question of whether an ingest-side rule can see one stays open. The validator was checked against seven deliberately broken cases before being trusted, since "38/38 valid" means nothing from a validator that cannot fail. Writing the cases first has already paid for itself twice. It forced two determinism decisions the prose had left vague, both of which a conformance suite cannot avoid answering. Sampling is counter-based rather than random: random is statistically nicer and impossible to assert on. Windows are measured on record timestamps rather than wall-clock, which makes replay deterministic and, not incidentally, makes backfill behave correctly where a wall-clock window would not. And it made the missing aggregate_count answer concrete. The design does not say what that action emits, or what a query not expecting a synthetic record sees. Rather than invent one by writing cases, the validator rejects any case using it, so the design question has to be answered before the behaviour can be frozen by accident. The corpus includes the acceptance case from real measured data: the two processes that account for roughly 60% of a real workstation's journal volume, and the one kernel message worth keeping. Signed-off-by: John Coffey <[email protected]> |
||
|
jcoffey-dev
|
7cc2fd8c78 |
Draft the Phase 8 processing design
Starts from the distribution channel and the safety invariant it protects, and derives the language from them, rather than designing a rule language and asking later how to ship it. Four decisions proposed. A rule is a matcher plus ordered typed actions, with no expressions and nothing resembling eval -- less expressive than Cribl on purpose, and the only shape that can be pushed to ten thousand hosts and audited by reading it. Total evaluation is the primary safety guarantee, with apply-then-verify as a backstop. One spec with two implementations means a language-neutral conformance suite is the specification and should be built first, not last. Distribution reuses DesiredOverride, following extra_file_paths as the precedent for a repeated field. It also corrects something #21 got wrong. That change said a fatal rule would strand an agent the way a corrupted ingest endpoint does. Reading apply_override's actual semantics, overrides live only in the running process's memory and are never written to disk, so a restarted agent boots clean and re-syncs -- a fatal rule set crash-loops rather than strands, and the agent keeps checking in, so it stays correctable. That is a much better failure mode, and it was acquired by accident: the "don't persist" choice was made to avoid filesystem writes on read-only images, not for safety. This design promotes it to a constraint, since persisting overrides later would silently convert every crash-loop into a strand. positioning.md is corrected to match rather than left disagreeing. Five open questions are left open rather than answered to look decisive, the sharpest being that there is no staged rollout today: an edit reaches every matching agent at once, which for executable rules is the difference between breaking one host and breaking all of them. The v1 acceptance test is real data, not a fixture: two processes on the maintainer's own workstation account for 308 of 325 journal entries in five minutes, and a suppress rule should remove about 60% of that host's volume. Signed-off-by: John Coffey <[email protected]> |