Listener that can't bind: retry, then report through readiness (optional exit) #485

Closed
opened 2026-11-19 17:09:33 +00:00 by jcoffey-dev · 1 comment
Owner

Follow-up to #436. When a configured listener can't bind, today the server logs one error and carries on without it. Nothing outside the log shows it, and nothing tries again. #436 was a different cause (the listener was never tried), but it showed how a half-up node looks fine to anything checking only the process.

Decided (John, 2026-11-19): retry, then report, don't exit by default.

  • Retry address-in-use. EADDRINUSE is retried for about 30 s with backoff. It's usually the previous process still holding the port after a restart. Permission errors and bad addresses are reported at once, without retrying.
  • Readiness. /healthz/ready answers not-ready while any configured listener isn't bound, naming it. Liveness is unchanged, so Kubernetes, load balancers and Docker health checks take the node out without restarting it in a loop.
  • Visible. The failure stays an error in the log and also reaches the console as a warning (listener, address, reason), so it's seen without a probe.
  • Optional exit. INBUXA_LISTENER_FAILURE=exit makes the server exit non-zero once retries are used up, for setups that would rather have systemd restart it. The default is report.
  • Not in scope. Ports re-opened by the legacy-protocols switch (LP-5) keep reporting as they do now, never exiting.

Done when

  • A listener whose port is taken at start binds once the port frees within the retry window.
  • A listener that never binds: error logged, /healthz/ready fails naming it, the console shows it, and the process stays up (default) or exits non-zero (exit).
  • Tests for the retry and the readiness answer.
Follow-up to #436. When a configured listener can't bind, today the server logs one error and carries on without it. Nothing outside the log shows it, and nothing tries again. #436 was a different cause (the listener was never tried), but it showed how a half-up node looks fine to anything checking only the process. **Decided (John, 2026-11-19): retry, then report, don't exit by default.** - **Retry address-in-use.** `EADDRINUSE` is retried for about 30 s with backoff. It's usually the previous process still holding the port after a restart. Permission errors and bad addresses are reported at once, without retrying. - **Readiness.** `/healthz/ready` answers not-ready while any configured listener isn't bound, naming it. Liveness is unchanged, so Kubernetes, load balancers and Docker health checks take the node out without restarting it in a loop. - **Visible.** The failure stays an error in the log and also reaches the console as a warning (listener, address, reason), so it's seen without a probe. - **Optional exit.** `INBUXA_LISTENER_FAILURE=exit` makes the server exit non-zero once retries are used up, for setups that would rather have systemd restart it. The default is `report`. - **Not in scope.** Ports re-opened by the legacy-protocols switch (LP-5) keep reporting as they do now, never exiting. **Done when** - A listener whose port is taken at start binds once the port frees within the retry window. - A listener that never binds: error logged, `/healthz/ready` fails naming it, the console shows it, and the process stays up (default) or exits non-zero (`exit`). - Tests for the retry and the readiness answer.
jcoffey-dev added this to the 27.1 milestone 2026-11-19 17:09:33 +00:00
jcoffey-dev added the
priority
p2
enhancement
labels 2026-11-19 17:09:33 +00:00
jcoffey-dev self-assigned this 2026-11-19 17:09:33 +00:00
Author
Owner

Done in #487 (server) and inbuxa/inbuxa-admin#252 (dashboard). A port in use is retried for 30 s. A listener that stays shut makes /healthz/ready answer 503 naming it, appears on the dashboard as "listeners not open" (every node, single server too) and in x:ClusterNode.unboundListeners. INBUXA_LISTENER_FAILURE=exit stops the server with code 1 instead. Checked live with a held port in all three cases. Ships in 27.1-rc.34. Docs: inbuxa/inbuxa.org#178, merged with that release.

Done in #487 (server) and inbuxa/inbuxa-admin#252 (dashboard). A port in use is retried for 30 s. A listener that stays shut makes `/healthz/ready` answer 503 naming it, appears on the dashboard as "listeners not open" (every node, single server too) and in `x:ClusterNode.unboundListeners`. `INBUXA_LISTENER_FAILURE=exit` stops the server with code 1 instead. Checked live with a held port in all three cases. Ships in 27.1-rc.34. Docs: inbuxa/inbuxa.org#178, merged with that release.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: inbuxa/inbuxa-server#485