EKS managed node group ends in Create failed because nodes never joined
Instances failed to join the Kubernetes clusterShort answer
Nodes that do not register within 15 minutes cause NodeCreationFailure and a Create failed status. AWS lists the common causes: missing node IAM role policies, no outbound path to the cluster, a cluster endpoint that nodes cannot reach, and a VPC without DNS options.
Fix it fast
1Find out which cause you have
Read the health issues of the node group. The code and message narrow the cause before you change anything.
aws eks describe-nodegroup --cluster-name <cluster> --nodegroup-name <nodegroup> --query nodegroup.health2Match what you see, then jump to its fix
- 1
NodeCreationFailure; AWS names insufficient node IAM role permissions as a common causeThe node IAM role is missing required policiesFix - 2
NodeCreationFailure on nodes in a public subnet without a public IP, or in a private subnet without a route outThe nodes cannot reach the cluster API serverFix - 3
The console shows Instances failed to join the Kubernetes clusterThe cluster endpoint access settings exclude the nodesFix - 4
The kubelet log contains: node "" not foundThe VPC has no DNS option values, so nodes have no private DNS entryFix
- 1
What does the error look like?
Instances failed to join the Kubernetes clusterNodeCreationFailureCreate failed
Cause 1 of 4
The node IAM role is missing required policies
1Check
The node role needs AmazonEKSWorkerNodePolicy and AmazonEC2ContainerRegistryPullOnly. List what is attached.
aws iam list-attached-role-policies --role-name <node-role-name>2Fix
Attach the two managed policies to the node role. The VPC CNI policy AmazonEKS_CNI_Policy belongs on the role used by the aws-node service account, which AWS recommends over putting it on the node role.
aws iam attach-role-policy \
--policy-arn arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy \
--role-name <node-role-name>
aws iam attach-role-policy \
--policy-arn arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryPullOnly \
--role-name <node-role-name>3Verify
Recreate the node group, or wait for the replacement nodes, and check its health.
aws eks describe-nodegroup --cluster-name <cluster> --nodegroup-name <nodegroup> --query nodegroup.statusConfirmed by docs.aws.amazon.com
Cause 2 of 4
The nodes cannot reach the cluster API server
1Check
Nodes must either reach the internet through a public IP address (and a security group that allows it), or meet the requirements for private clusters with limited internet access. Check the subnet's route table and whether nodes get a public IP.
aws ec2 describe-route-tables --filters Name=association.subnet-id,Values=<subnet-id>2Fix
Give nodes a route out (public IP in a public subnet, or a NAT path from a private subnet), or build the private-cluster endpoints AWS documents.
# public subnet: make new instances receive a public IPv4 address
aws ec2 modify-subnet-attribute --subnet-id <subnet-id> --map-public-ip-on-launch3Verify
Create the node group again and wait for the nodes to register.
kubectl get nodesConfirmed by docs.aws.amazon.com
Cause 3 of 4
The cluster endpoint access settings exclude the nodes
1Check
Make sure that either private endpoint access is enabled for the cluster, or the CIDR blocks allowed on the public endpoint include the addresses the nodes use to reach it.
aws eks describe-cluster --name <cluster> --query cluster.resourcesVpcConfig2Fix
Enable private access, or add the nodes' outbound addresses to the public access CIDRs.
aws eks update-cluster-config --name <cluster> --resources-vpc-config endpointPublicAccess=false,endpointPrivateAccess=true3Verify
Wait for the update to finish, then create the node group again.
aws eks describe-cluster --name <cluster> --query cluster.resourcesVpcConfig.endpointPrivateAccessConfirmed by docs.aws.amazon.com
Cause 4 of 4
The VPC has no DNS option values, so nodes have no private DNS entry
1Check
The VPC needs domain-name and domain-name-servers set in its DHCP options. The defaults are <region>.compute.internal and AmazonProvidedDNS.
aws ec2 describe-dhcp-options2Fix
Create a DHCP options set with those values and associate it with the VPC.
# DHCP options set values (default):
# domain-name = <region>.compute.internal
# domain-name-servers = AmazonProvidedDNS3Verify
Launch a new node and watch it register.
kubectl get nodes -wConfirmed by docs.aws.amazon.com
How do I stop it coming back?
- Create node roles from the AWS-documented policy set and keep them in infrastructure code.
- Decide the cluster endpoint access mode before adding nodes, and test one node group before scaling out.
- Keep spare IP addresses in the node subnets; node creation also fails when a subnet runs out.
Sources
How was this page made?
I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.
Which stack is behind this page?
Projects of mine built on the same technologies:
- Dispatch ProA real-time logistics platform: Flutter apps, Spring Boot services and a Kafka event pipeline on AWS.
- RaheebA consumer app shipped to both stores from one Flutter codebase on a .NET and Azure back end.
- Loade DriverThe driver side of a delivery platform: live jobs, routing and earnings, past 5K Google Play downloads.
Related fixes
Servers
Pod stuck in ContainerCreating: unable to attach or mount EBS volumes
failed to provision volume with StorageClass
The kubelet waited for the volume and gave up.
3 causes, each with a fixServers
Status code 460 on an AWS Application Load Balancer
elb_status_code 460 in ALB access logs
A 460 is generated by the load balancer, not by your app.
3 causes, each with a fixServers
Application Load Balancer returns 504 for requests to EC2 instances
HTTPCode_ELB_504_Count
A 504 from an Application Load Balancer means it could not get an answer from a target in time.
4 causes, each with a fixServers
Apache has a certificate, but https:// does not open
No line with :443 in the output of ss -ltnp
HTTPS needs four things at once: mod_ssl loaded, Apache listening on 443, a virtual host on 443 with SSLEngine on, and a firewall that lets 443 in.
4 causes, each with a fixServers
Argo CD behind an nginx Ingress loops with too many redirects
307 Temporary Redirect on every request to the Argo CD host
The browser and Argo CD disagree about who handles TLS.
3 causes, each with a fixServers
aws emr --steps fails: Error parsing parameter, Expected '=', received 'EOF'
Error parsing parameter '--steps': Expected: '=', received: 'EOF' for input:
The AWS CLI tried to read your option as shorthand key=value pairs and ran out of text where it expected an equals sign.
3 causes, each with a fix