A pod stays Pending with volume node affinity conflict
1 node(s) had volume node affinity conflict, 3 node(s) didn't match Pod's node affinity/selectorShort answer
The scheduler found that the pod's volume can only be reached from some nodes, and none of those nodes are allowed for the pod. The usual cause is a StorageClass that binds volumes immediately, in a zone the pod cannot use. Use WaitForFirstConsumer for new volumes, and align node selectors with the volume's node affinity.
Fix it fast
1Find out which cause you have
Read the pod's events to see which nodes the scheduler rejected, then read the volume's node affinity to see where it can be used.
kubectl describe pod redis-0 | tail -n 15 kubectl get pv -o custom-columns=NAME:.metadata.name,CLAIM:.spec.claimRef.name,AFFINITY:.spec.nodeAffinity.required2Match what you see, then jump to its fix
- 1
The volume's affinity names one zone or node, and the pod's other constraints exclude itThe StorageClass binds volumes before the pod is scheduledFix - 2
The event says the nodes did not match the pod's node affinity or selector, and the volume's affinity lists other nodesThe pod's selector or affinity excludes every node the volume can useFix - 3
A local volume's affinity names one node, and that node is unschedulableThe node that holds the volume is cordoned or goneFix
- 1
What does the error look like?
1 node(s) had volume node affinity conflict, 3 node(s) didn't match Pod's node affinity/selectorSTATUS Pending
Cause 1 of 3
The StorageClass binds volumes before the pod is scheduled
1Check
With the default Immediate mode, the volume is provisioned when the claim is created, without knowing where the pod will run, which can leave the pod unschedulable.
kubectl get storageclass -o custom-columns=NAME:.metadata.name,MODE:.volumeBindingMode2Fix
Use a StorageClass with volumeBindingMode WaitForFirstConsumer for new volumes, so provisioning follows the pod's scheduling constraints. Existing bound volumes stay where they are, so delete the pending claim and its volume if the data is disposable.
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: standard-wait
provisioner: example.com/example # use your cluster's provisioner
volumeBindingMode: WaitForFirstConsumer3Verify
A new claim stays Pending until a pod uses it, then the pod schedules.
kubectl get pvc,pod -wConfirmed by kubernetes.io
Cause 2 of 3
The pod's selector or affinity excludes every node the volume can use
1Check
Pods that use a volume are only scheduled to nodes selected by its node affinity, and the pod's own constraints must overlap with it.
kubectl get statefulset redis -o jsonpath='{.spec.template.spec.nodeSelector}{"\n"}{.spec.template.spec.affinity}{"\n"}'
kubectl get nodes --show-labels2Fix
Change the selector or node affinity in the StatefulSet so it includes nodes in the volume's zone. When provisioning is meant to stay in certain zones, set allowedTopologies in the StorageClass.
allowedTopologies:
- matchLabelExpressions:
- key: topology.kubernetes.io/zone
values:
- us-central-1a
- us-central-1b3Verify
The pod leaves Pending.
kubectl get pod redis-0 -o wideConfirmed by kubernetes.io
Cause 3 of 3
The node that holds the volume is cordoned or gone
1Check
For local volumes, node affinity must be set explicitly and ties the pod to that node. If it was drained, the pod waits.
kubectl get nodes2Fix
Make the node schedulable again, or recreate the volume elsewhere if the node is lost and the data can be restored.
kubectl uncordon <node name>3Verify
The node shows Ready without SchedulingDisabled, and the pod is scheduled.
kubectl get nodes && kubectl get pod redis-0Confirmed by kubernetes.io
How do I stop it coming back?
- Set WaitForFirstConsumer on every StorageClass backed by zonal disks.
- Spread StatefulSet replicas with the same zones your volumes use.
- After node maintenance, run uncordon, and do not leave drained nodes behind.
Sources
How was this page made?
I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.
Which stack is behind this page?
Projects of mine built on the same technologies:
Related fixes
Servers
Argo CD behind an nginx Ingress loops with too many redirects
307 Temporary Redirect on every request to the Argo CD host
The browser and Argo CD disagree about who handles TLS.
3 causes, each with a fixServers
A cert-manager Certificate stays at READY False with Let's Encrypt
Issuing certificate as Secret does not exist
A Certificate that stays not ready is waiting on a chain: Issuer, CertificateRequest, Order, Challenge.
3 causes, each with a fixServers
Kubernetes in Docker Desktop will not start or stays on starting
kubectl shows no contexts, or errors against the docker-desktop context
When the Docker Desktop cluster hangs on starting, the usual causes are too few resources, a broken cluster state, or the kubeadm provisioner failing.
4 causes, each with a fixServers
EKS managed node group ends in Create failed because nodes never joined
NodeCreationFailure
Nodes that do not register within 15 minutes cause NodeCreationFailure and a Create failed status.
4 causes, each with a fixServers
An Elasticsearch pod keeps restarting with Back-off restarting failed container
CrashLoopBackOff
Back-off restarting failed container only says the container keeps dying.
3 causes, each with a fixServers
helm install fails: must either provide a name or specify --generate-name
Error: must either provide a name or specify --generate-name
helm install expects a release name followed by a chart.
3 causes, each with a fix