S5 · size S · part of inbuxa for Kubernetes (tracking issue in inbuxa-charts)
Decides whether the 2026.x to 27.1 store upgrade can be rolling or needs every server pod stopped first.
Done when
Answer recorded with evidence, feeding K-S3 and K-C9
**S5** · size S · part of inbuxa for Kubernetes (tracking issue in inbuxa-charts)
Decides whether the 2026.x to 27.1 store upgrade can be rolling or needs every server pod stopped first.
**Done when**
- Answer recorded with evidence, feeding K-S3 and K-C9
jcoffey-dev
added this to the 27.1 milestone 2026-11-18 05:14:37 +00:00
Answer (from the code, 2026-11-17): no. The 2026.x to 27.1 crossing can't be rolling.
2026.x is store layout 1; 27.1 is layout 3 (crates/migration/src/layout.rs).
Step 1→2 is rolling-safe. Step 2→3 is not (SL-18: old nodes would keep writing the moved data, and an old binary can't open a RocksDB store with column families it doesn't know).
27.1 handles this itself: one node takes the layout lease (SL-3); a step that isn't rolling-safe waits until no other node shows as Active (SL-6), and a stopped node stops showing as Active 3 minutes after its last heartbeat. 2026.x refuses a store with a newer mark (SL-4).
What this means on Kubernetes: a normal rolling update stalls. The new pod waits forever for the old pods to stop, and the old pods aren't removed because the new one never becomes ready. So for the crossing the chart must:
Scale every server Deployment to 0 (pre-upgrade hook).
Run one Job on the new image that does the layout upgrade and exits (K-S3). It will wait up to ~3 min for the old heartbeats to expire.
Then roll out the new Deployments.
Mail is down for roughly 3 minutes plus the step's run time. Later 27.1.x upgrades are plain rolling updates unless a future step is marked not rolling-safe; the hook should read that from the server rather than hard-code it (worth a field in K-S3's output).
**Answer (from the code, 2026-11-17): no. The 2026.x to 27.1 crossing can't be rolling.**
- 2026.x is store layout 1; 27.1 is layout 3 (`crates/migration/src/layout.rs`).
- Step 1→2 is rolling-safe. Step 2→3 is not (SL-18: old nodes would keep writing the moved data, and an old binary can't open a RocksDB store with column families it doesn't know).
- 27.1 handles this itself: one node takes the layout lease (SL-3); a step that isn't rolling-safe waits until no other node shows as Active (SL-6), and a stopped node stops showing as Active 3 minutes after its last heartbeat. 2026.x refuses a store with a newer mark (SL-4).
**What this means on Kubernetes:** a normal rolling update stalls. The new pod waits forever for the old pods to stop, and the old pods aren't removed because the new one never becomes ready. So for the crossing the chart must:
1. Scale every server Deployment to 0 (pre-upgrade hook).
2. Run one Job on the new image that does the layout upgrade and exits (K-S3). It will wait up to ~3 min for the old heartbeats to expire.
3. Then roll out the new Deployments.
Mail is down for roughly 3 minutes plus the step's run time. Later 27.1.x upgrades are plain rolling updates unless a future step is marked not rolling-safe; the hook should read that from the server rather than hard-code it (worth a field in K-S3's output).
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
S5 · size S · part of inbuxa for Kubernetes (tracking issue in inbuxa-charts)
Decides whether the 2026.x to 27.1 store upgrade can be rolling or needs every server pod stopped first.
Done when
Answer (from the code, 2026-11-17): no. The 2026.x to 27.1 crossing can't be rolling.
crates/migration/src/layout.rs).What this means on Kubernetes: a normal rolling update stalls. The new pod waits forever for the old pods to stop, and the old pods aren't removed because the new one never becomes ready. So for the crossing the chart must:
Mail is down for roughly 3 minutes plus the step's run time. Later 27.1.x upgrades are plain rolling updates unless a future step is marked not rolling-safe; the hook should read that from the server rather than hard-code it (worth a field in K-S3's output).
Folded into the checklist on inbuxa/inbuxa-charts#14 to keep the issue list short. The details stay here; reopen this when work on it starts.