Moving Pod to Another Node

Posted on | Edited on | In

Sometimes there is a need to shift pod to another node in order to rebalance the k8s node load, such as context switch, CPU LA, network traffic, etc.

For pods controlled by deployment or statefulset, just delete the pod will usually end up with recreating it in the same node again, this can be fixed with cordon:

1
2
3
4
5
6
7
8
# make the node unschedulable for new pods
kubectl cordon <pod host node name>

# delete the target pod, it will be recreated  in another node
kubectl delete pod <pod name>

# revert node status
kubectl uncordon <pod host node name>

But please note that sometimes the deleted pod will not be rescheduled to the idle node due to its big resource request, from the document:

Although actual memory or CPU resource usage on nodes is very low, the scheduler still refuses to place a Pod on a node if the capacity check fails. This protects against a resource shortage on a node when resource usage later increases, for example, during a daily peak in request rate.

In this case, you can move some less-comsumed pods rather than the original one.

BTW, if you are working on GCP, you can check the node capacity and requested resource from the UI.

0%