phoenix-oss/tenant-demo
1
0
Fork 0
generated from phoenix-oss/tenant-tpl
Code Issues 2 Pull requests 4 Projects Releases Packages Wiki Activity Actions

[Task] Node-drain safety: sales demo: ubuntu deployment single-replica; 7 VMs #19

New issue
Open
opened 2026-11-11 11:10:36 +00:00 by angel.nunez · 0 comments
angel.nunez commented 2026-11-11 11:10:36 +00:00
Owner
Copy link

Part of phoenix/openshift-config#369 — cross-team preparation for fully automated cluster updates: workloads must tolerate one-at-a-time node drains.

Goal

Make the workloads deployed by this tenant (demo on cluster ai-3) safe for node drains during automated OpenShift platform and operator updates, per the cross-team preparation requirements in phoenix/openshift-config#369.

Findings (live cluster audit, 2026-11-11)

demo @ ai-3

Single-replica workloads (downtime on every drain, no PDB possible until replicated) (1)

  • Deployment/ubuntu-deployment

Virtual machines (need live-migration policy or accepted restarts) (1)

  • 7 VMI(s)/7 VM(s) — VMs: fortigate-ksd, strongswan, ubuntu-vm-1, ubuntu-vm-2, ubuntu-vm-3, windows-vm-standard, windows-vm-standard-dev — need live-migration policy or explicit restart acceptance

Required work (per the epic's requirements)

  • PodDisruptionBudgets for every multi-replica, customer-facing or platform-critical workload — and never one that allows 0 disruptions while pods are healthy (that stalls the drain and fires MCDDrainError after an hour)
  • Multi-replica for customer-facing / platform-critical services — no single-replica services on production clusters; for per-customer instances, either scale the shared components or document the accepted per-drain restart
  • Topology spread constraints / pod anti-affinity so replicas never share one node
  • Stateful workloads: replicas and failover within drain timeouts (volume detach/attach behaviour)
  • Node-pinned workloads (nodeSelector, hostNetwork/hostPort, local PVs): migration path or explicit acceptance of restarts
  • Virtual machines: live-migration policy or explicit acceptance of restarts during drains

Deliverable

Manifest changes in this repo (PDBs, replicas, spread constraints, VM migration settings) — or a documented per-workload acceptance where downtime during a drain is acceptable. Reference this epic and the PRs from the fix branches.

Notes

Related: openshift-config#387 — host-model VMs on basel cannot live-migrate (platform-side follow-up).

Part of phoenix/openshift-config#369 — cross-team preparation for fully automated cluster updates: workloads must tolerate one-at-a-time node drains. ## Goal Make the workloads deployed by this tenant (`demo` on cluster `ai-3`) safe for node drains during automated OpenShift platform and operator updates, per the cross-team preparation requirements in phoenix/openshift-config#369. ## Findings (live cluster audit, 2026-11-11) ### demo @ ai-3 **Single-replica workloads (downtime on every drain, no PDB possible until replicated)** (1) - `Deployment/ubuntu-deployment` **Virtual machines (need live-migration policy or accepted restarts)** (1) - `7 VMI(s)/7 VM(s)` — VMs: fortigate-ksd, strongswan, ubuntu-vm-1, ubuntu-vm-2, ubuntu-vm-3, windows-vm-standard, windows-vm-standard-dev — need live-migration policy or explicit restart acceptance ## Required work (per the epic's requirements) - [ ] PodDisruptionBudgets for every multi-replica, customer-facing or platform-critical workload — and never one that allows 0 disruptions while pods are healthy (that stalls the drain and fires `MCDDrainError` after an hour) - [ ] Multi-replica for customer-facing / platform-critical services — no single-replica services on production clusters; for per-customer instances, either scale the shared components or document the accepted per-drain restart - [ ] Topology spread constraints / pod anti-affinity so replicas never share one node - [ ] Stateful workloads: replicas and failover within drain timeouts (volume detach/attach behaviour) - [ ] Node-pinned workloads (nodeSelector, hostNetwork/hostPort, local PVs): migration path or explicit acceptance of restarts - [ ] Virtual machines: live-migration policy or explicit acceptance of restarts during drains ## Deliverable Manifest changes in this repo (PDBs, replicas, spread constraints, VM migration settings) — or a documented per-workload acceptance where downtime during a drain is acceptable. Reference this epic and the PRs from the fix branches. ## Notes Related: [openshift-config#387](https://git.kvant.cloud/phoenix/openshift-config/issues/387) — host-model VMs on basel cannot live-migrate (platform-side follow-up).
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
renovate/nginx-25.x
renovate/helmrelease-2.x
renovate/quay.io-containerdisks-ubuntu-26.x
renovate/ubuntu-26.x
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Blocks
You do not have permission to read 1 dependency
Reference
phoenix-oss/tenant-demo#19
No description provided.