jcoffey-dev
ec4b860ba8
The last unanswered action, and the only one whose output is not the input with edits -- it emits a record that never existed, which is why it was deferred twice. It emits the window's first record unchanged, tagged with cairnobs.aggregated, cairnobs.count, and the observed window bounds. That follows the convention the agent already uses for heartbeat and host-metrics records rather than inventing a second synthetic-record mechanism, and keeping the first record intact means a reader sees a real example of what was collapsed instead of an invented summary. It tags even when the count is one. Emitting a bare record there would be tidier and would make cairnobs.count present only sometimes, so summing it silently breaks on quiet windows. window_last is the last record that actually contributed, never window_start + window_ms, because a window flushed early must not claim an end that never happened. Specifying it surfaced a problem the other nine actions do not have. Windows are measured on record time, so a window can only be closed by a later record arriving. suppress_duplicates never has anything pending; aggregate_count holds state, so a matching stream that goes quiet leaves its aggregate unemitted indefinitely -- data loss dressed as latency. Emission therefore has a second trigger, end of stream, which the corpus defines as an implicit flush after the last input and which production gets from the batch flush. The cost is stated rather than hidden: window_ms becomes a maximum, not a guarantee, and one burst can produce more than one aggregate. And it has a consequence nobody should meet in production first: stats count undercounts aggregated data silently, so every panel and alert counting rows changes meaning the moment a rule aggregates the data behind it. Nothing here fixes that. The correct idiom is summing cairnobs.count; teaching the query layer to do it automatically is a Phase 2 change to the IR, recorded as the open question this decision leaves in its place rather than quietly inherited. Six cases added, corpus at 45. The validator's unspecified-action guard stays in place with an empty set, still rejecting anything added to it. Signed-off-by: John Coffey <[email protected]>
74 lines
1.7 KiB
JSON
74 lines
1.7 KiB
JSON
{
|
|
"name": "aggregate_count_honours_key_fields",
|
|
"description": "With key_fields, each key gets its own window and its own aggregate. Emission order at end of stream follows the order the windows were opened.",
|
|
"rules": [
|
|
{
|
|
"match": [],
|
|
"actions": [
|
|
{
|
|
"action": "aggregate_count",
|
|
"window_ms": 5000,
|
|
"key_fields": [
|
|
"host",
|
|
"message"
|
|
]
|
|
}
|
|
]
|
|
}
|
|
],
|
|
"inputs": [
|
|
{
|
|
"timestamp_unix_nano": 0,
|
|
"host": "h1",
|
|
"service": "s1",
|
|
"severity": "SEVERITY_INFO",
|
|
"message": "z",
|
|
"attributes": {}
|
|
},
|
|
{
|
|
"timestamp_unix_nano": 100000000,
|
|
"host": "h2",
|
|
"service": "s1",
|
|
"severity": "SEVERITY_INFO",
|
|
"message": "z",
|
|
"attributes": {}
|
|
},
|
|
{
|
|
"timestamp_unix_nano": 200000000,
|
|
"host": "h1",
|
|
"service": "s1",
|
|
"severity": "SEVERITY_INFO",
|
|
"message": "z",
|
|
"attributes": {}
|
|
}
|
|
],
|
|
"expect": [
|
|
{
|
|
"timestamp_unix_nano": 0,
|
|
"host": "h1",
|
|
"service": "s1",
|
|
"severity": "SEVERITY_INFO",
|
|
"message": "z",
|
|
"attributes": {
|
|
"cairnobs.aggregated": "true",
|
|
"cairnobs.count": "2",
|
|
"cairnobs.window_start_unix_nano": "0",
|
|
"cairnobs.window_last_unix_nano": "200000000"
|
|
}
|
|
},
|
|
{
|
|
"timestamp_unix_nano": 100000000,
|
|
"host": "h2",
|
|
"service": "s1",
|
|
"severity": "SEVERITY_INFO",
|
|
"message": "z",
|
|
"attributes": {
|
|
"cairnobs.aggregated": "true",
|
|
"cairnobs.count": "1",
|
|
"cairnobs.window_start_unix_nano": "100000000",
|
|
"cairnobs.window_last_unix_nano": "100000000"
|
|
}
|
|
}
|
|
]
|
|
}
|