Registry writes apply to the running settings without ReloadSettings #39
Assignees
jcoffey-dev (John Coffey)
Clear assignees
jcoffey-dev (John Coffey)
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: inbuxa/inbuxa-server#39
Reference in New Issue
Block a user
Depends on #38 (reload behavior). This branch is built on it, so until #38 merges the diff shows both.
Problem
A registry write only took effect after an explicit
x:Action ReloadSettings. The one exception was Directory/Authentication writes (crates/jmap/src/registry/set.rs, DIR-17).In the rehearsal, a new
MtaDeliverySchedulestayed unknown to the queue ("Queue strategy not found") until someone reloaded. Any client that forgot the reload left the server running stale settings.Change: automatic reload after a write
When it reloads. After an
x:<Object>/setthat created, updated or destroyed at least one object the running settings are built from, the server reloads those settings. It then broadcastsRegistryChange::Reload, the same eventReloadSettingssends, so every node applies it. Other nodes pick it up about a second later, as they do for ReloadSettings; the cluster test below shows it at once.Which object types reload what (
common::cache::reload::write_reload_target):Why it's safe
New set response field
x:settingsReloadappears only when the write called for a reload:The description comes from the same helper as #38's
ReloadSettingsrefusal.For the admin UI change
The server now reloads by itself after these saves, so a
ReloadSettingscall after each save is redundant. It's still harmless: one more reload.The UI can instead read
x:settingsReloadfrom the set response. Ifappliedis false, show the description, since the object was saved but isn't in effect.ReloadSettingsis still needed for:Test harness
Every reload sends the queue a
ReloadSettingsevent.read_event,try_read_eventandassert_no_eventsin the SMTP test harness now skip those events.expect_reload_settingsstill waits for one.Tests
system::auto_reload::settings_reload_tests(new, RocksDb):MtaVirtualQueueand anMtaDeliverySchedulecreated over JMAP are in the running core with no ReloadSettings. The schedule is gone again once destroyed.applied: falsewith the error.x:settingsReload.main, the schedule is missing after the create (has_schedulefails).cluster::broadcast::cluster_tests(3 nodes,STORE=PostgreSql COORDINATOR=Nats): now creates a schedule on node 0 without a reload, and checks all three nodes have it within 5 s. They had it at once. The same check onmainfails ("node 0 didn't pick up the new delivery schedule").A 3-node rehearsal found that saving an MtaDeliverySchedule left it unknown to the queue ("Queue strategy not found") until someone ran x:Action ReloadSettings; only Directory and Authentication writes reloaded (DIR-17). The admin UI has to remember a separate reload after every save, and a script or API client that doesn't gets a server running stale settings. x:<Object>/set now reloads the running settings when it created, updated or destroyed an object they are built from, and broadcasts the same RegistryChange::Reload over the coordinator as ReloadSettings, so every node applies it: - Settings objects (MTA, spam filter, listeners, tracers, Sieve system scripts, cluster roles, directories, ...: the object types the core, telemetry, listener and directory builders read) get a full reload. - Certificates, lookup stores and blocked/allowed IPs get their own targeted reloads. - Accounts, domains, roles and other data read as needed, stores (they take a restart) and applications (their own reload action) get none. Full reloads are coalesced: a write waits for a reload that started after it was stored and joins one if it can, so a burst of writes, or a request with many objects, costs one or two reloads, not one each. The write itself is never undone. When the reload is refused (build errors in objects that were working, the rule from the previous commit), the set response says so in a new x:settingsReload field, {"applied": false, "description": "Saved, but the running settings were not reloaded. <object>: <error>"}; {"applied": true} otherwise. The field is absent when the write needs no reload. The description helper is shared with ReloadSettings' refusal. Each reload sends the queue a ReloadSettings event, so the SMTP test harness's read_event, try_read_event and assert_no_events now pass over those; expect_reload_settings still waits for one. system::auto_reload::settings_reload_tests (new): an MtaVirtualQueue and an MtaDeliverySchedule created over JMAP are in the running settings with no ReloadSettings, and gone once destroyed; eight concurrent creates all land; a write whose reload fails is stored and reported applied: false with the error; a domain write carries no x:settingsReload. On main the new schedule is missing. The cluster broadcast test (three nodes, PostgreSQL + NATS) now checks that every node has a schedule created on node 0 without a reload.