Log tracer: a JSON line format #462

Merged
jcoffey-dev merged 1 commits from feat/456-json-log-format into main 2026-11-18 22:27:09 +00:00
Owner

Fixes #456 (split out of inbuxa/inbuxa-admin#229, forum t/308).

A Log tracer gets a format setting: text (the default, unchanged) or json, one object per line:

{"timestamp":"2026-11-18T10:00:00Z","level":"info","event":"smtp.connection-start","description":"...","data":{"remoteIp":"192.0.2.1","remotePort":25}}

data holds the event's keys and its span's as typed values (nested events as objects, lists as arrays), with the same encoding webhooks use. ansi and multiline apply to text only. Spec: MON-43 in docs/spec/features/monitoring.md.

Reading back: x:Log/get and x:Log/query (with every #455 filter) read both formats, and a file holding both, in order. A JSON entry's details is its data as a JSON object; inbuxa-admin #246 (merged) shows those fields as written.

Stored format: x:Tracer and x:Bootstrap move to version 3 together, since the bootstrap holds a tracer and more fields follow it there. Tracers saved before read back as text. An older server ignores the format and writes text.

Schema: new LogFormat enum and the format field after "Rotate frequency" on the Log file form. Hash written with expr-schema.py's URL-safe encoder.

Checks run locally:

  • cargo check --all-targets, cargo fmt --check, all fork checks and their unit tests.
  • Unit tests in common, jmap, registry and trc: the JSON line, JSON and mixed files through every filter, the tracer's output read back exactly, and the stored format at the old and new versions.
  • system::tracer_reload gains a step: a running tracer switched to JSON writes a JSON line next, and earlier lines stay text. That step passes. The test's existing path-change part fails on this machine on main as well (all 2000 stream events reach the old file before the reload runs, so seq-01999-end never reaches the new one). It's a timing assumption from before this PR; I ran the new step with those two assertions skipped locally (not committed).

Not included: the Stdout tracer keeps text only.

Fixes #456 (split out of inbuxa/inbuxa-admin#229, forum t/308). A Log tracer gets a `format` setting: `text` (the default, unchanged) or `json`, one object per line: ```json {"timestamp":"2026-11-18T10:00:00Z","level":"info","event":"smtp.connection-start","description":"...","data":{"remoteIp":"192.0.2.1","remotePort":25}} ``` `data` holds the event's keys and its span's as typed values (nested events as objects, lists as arrays), with the same encoding webhooks use. `ansi` and `multiline` apply to text only. Spec: MON-43 in `docs/spec/features/monitoring.md`. **Reading back:** `x:Log/get` and `x:Log/query` (with every #455 filter) read both formats, and a file holding both, in order. A JSON entry's `details` is its `data` as a JSON object; inbuxa-admin #246 (merged) shows those fields as written. **Stored format:** `x:Tracer` and `x:Bootstrap` move to version 3 together, since the bootstrap holds a tracer and more fields follow it there. Tracers saved before read back as `text`. An older server ignores the format and writes text. **Schema:** new `LogFormat` enum and the `format` field after "Rotate frequency" on the Log file form. Hash written with `expr-schema.py`'s URL-safe encoder. **Checks run locally:** - `cargo check --all-targets`, `cargo fmt --check`, all fork checks and their unit tests. - Unit tests in common, jmap, registry and trc: the JSON line, JSON and mixed files through every filter, the tracer's output read back exactly, and the stored format at the old and new versions. - `system::tracer_reload` gains a step: a running tracer switched to JSON writes a JSON line next, and earlier lines stay text. That step passes. The test's existing path-change part fails on this machine **on main as well** (all 2000 stream events reach the old file before the reload runs, so `seq-01999-end` never reaches the new one). It's a timing assumption from before this PR; I ran the new step with those two assertions skipped locally (not committed). Not included: the Stdout tracer keeps text only.
jcoffey-dev added 1 commit 2026-11-18 22:14:55 +00:00
Log tracer: a JSON line format
ci / fork-checks (pull_request) Skipped
ci / build (pull_request) Skipped
github/ci (branch) GitHub Actions
ci / github (pull_request) Successful in 12m6s
cd8ae1c0d2
The Log tracer wrote text only: `<time> <LEVEL> <description> (<event id>)
key = value, ...`, with quotes inside strings left unescaped, so neither a
log collector (Loki, Vector, Kubernetes) nor the console's Logs page could
read the fields exactly. Requested in community t/308.

A Log tracer now has a `format` (monitoring spec MON-43): `text`, the
default and unchanged, or `json`, one object per line:
{"timestamp", "level", "event", "description", "data"}, where `timestamp`
is RFC 3339, `level` lowercase, and `data` holds the event's keys and its
span's as typed values, nested events as objects and lists as arrays (the
same encoding as webhooks). `ansi` and `multiline` apply to text only.
Changing the format restarts the tracer like any other setting, after the
old one has written what it queued.

x:Log/get and x:Log/query, with all of #455's filters, read both formats,
and a file holding both (a tracer switched over) reads in order. A JSON
entry's `details` is its `data` as a JSON object ("" when it has none);
inbuxa-admin #246 shows those fields as written.

The format is stored at x:Tracer and x:Bootstrap version 3 (the bootstrap
holds a tracer too, and more fields follow it there, so both move
together); tracers saved before read back as text. A server from before
this reads a newer tracer's fields up to its own and ignores the format,
writing text.

The served schema has the LogFormat enum and the field, after Rotate
frequency on the Log file form, so inbuxa Admin offers it.

Test: unit tests for the line (one typed object, escapes kept), for
reading JSON and mixed files through every filter, for a tracer's output
reading back exactly, and for the stored format (a tracer and a bootstrap
saved at the old versions read back as text; saved now, as JSON). The
tracer reload system test switches a running tracer to JSON and checks
the next line is JSON while the earlier ones stay text.

Fixes #456.
jcoffey-dev merged commit 1853af13c2 into main 2026-11-18 22:27:09 +00:00
jcoffey-dev deleted branch feat/456-json-log-format 2026-11-18 22:27:09 +00:00
Sign in to join this conversation.