Maaz Zindani

Loading the index…

↑↓ move↵ openesc close4072 pages
Node.js, PHP and APIs

AWS SDK for PHP cannot get credentials from the instance profile metadata server

ErrorError retrieving credentials from the instance profile metadata server.

Short answer

The SDK tries the EC2 instance metadata service when it has no valid credentials from anywhere earlier in its chain. The error means that last step also failed: the code is not running on EC2, no IAM role is attached, or a container cannot reach the metadata service. Give the SDK credentials, attach a role, or raise the hop limit.

  • Applies to AWS SDK for PHP 3.x, PHP 8.3, Amazon Linux 2023
  • Updated
  • By Maaz Zindani, Senior Software Engineer

Fix it fast

  1. 1Find out which cause you have

    From the same machine or container that runs the PHP code, ask the metadata service for the role name. Any answer, even an error, tells you which cause applies.

    TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
      && curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/iam/security-credentials/
  2. 2Match what you see, then jump to its fix

    1. 1The curl above hangs or fails because there is no metadata service, on a laptop, another host or CIThe code runs outside EC2 and nothing else supplied credentialsFix
    2. 2On the instance, the role-name request returns 404 Not FoundThe EC2 instance has no IAM role attachedFix
    3. 3The first-step command works on the host but times out inside Docker or a podA container cannot reach the metadata service (hop limit 1)Fix
    4. 4The request returns 403 Forbidden, or the SDK never tries because AWS_EC2_METADATA_DISABLED is trueThe metadata service is turned offFix

What does the error look like?

  • Error retrieving credentials from the instance profile metadata server.

Cause 1 of 4

The code runs outside EC2 and nothing else supplied credentials

1Check

AWS documents that an SDK without valid credentials falls back to the instance metadata service. Off EC2 that last step cannot work, so the real gap is earlier in the chain.

env | grep -E '^AWS_'

2Fix

Provide credentials the normal way: environment variables, a shared credentials profile, or key and secret in the client config for local use. Do not hard-code keys in source.

export AWS_ACCESS_KEY_ID=your-key-id
export AWS_SECRET_ACCESS_KEY=your-secret
export AWS_DEFAULT_REGION=eu-west-1

3Verify

Create the client again and make one cheap call, such as listing buckets.

php -r 'require "vendor/autoload.php"; $c = new Aws\S3\S3Client(["region"=>"eu-west-1","version"=>"latest"]); print_r(array_column($c->listBuckets()["Buckets"], "Name"));'

Confirmed by docs.aws.amazon.com

Did this fix it?

Cause 2 of 4

The EC2 instance has no IAM role attached

1Check

Credentials come from the iam/security-credentials path in the metadata service. A 404 there means the instance has nothing to return. Check the attached profile from the AWS CLI on any machine with access.

aws ec2 describe-instances --instance-ids i-0123456789abcdef0 --query 'Reservations[].Instances[].IamInstanceProfile'

2Fix

Attach an IAM role through its instance profile (console: Actions, Security, Modify IAM role), then repeat the first command, which should now print the role name. An instance can hold one role.

aws ec2 associate-iam-instance-profile --instance-id i-0123456789abcdef0 --iam-instance-profile Name=my-app-profile

3Verify

The first-step command now lists the role name.

curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/iam/security-credentials/

Confirmed by docs.aws.amazon.com

Did this fix it?

Cause 3 of 4

A container cannot reach the metadata service (hop limit 1)

1Check

SDKs use IMDSv2, whose token reply travels one network hop more from inside a container. With a hop limit of 1 the call may get no response at all.

aws ec2 describe-instances --instance-ids i-0123456789abcdef0 --query 'Reservations[].Instances[].MetadataOptions'

2Fix

Raise the response hop limit to 2, or pass settings such as credentials to the container directly.

aws ec2 modify-instance-metadata-options --instance-id i-0123456789abcdef0 --http-put-response-hop-limit 2 --http-endpoint enabled

3Verify

Run the first-step command again inside the container.

docker exec my-container curl -s -m 3 -X PUT http://169.254.169.254/latest/api/token -H "X-aws-ec2-metadata-token-ttl-seconds: 60"

Confirmed by docs.aws.amazon.com

Did this fix it?

Cause 4 of 4

The metadata service is turned off

1Check

The metadata service can be disabled per instance, and the SDK setting AWS_EC2_METADATA_DISABLED=true stops it from using the service for credentials.

env | grep AWS_EC2_METADATA_DISABLED
aws ec2 describe-instances --instance-ids i-0123456789abcdef0 --query 'Reservations[].Instances[].MetadataOptions.HttpEndpoint'

2Fix

Remove the environment variable, or enable the endpoint on the instance if the role is meant to be used.

unset AWS_EC2_METADATA_DISABLED
aws ec2 modify-instance-metadata-options --instance-id i-0123456789abcdef0 --http-endpoint enabled

3Verify

The first-step command returns the role name.

curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/iam/security-credentials/

Confirmed by docs.aws.amazon.com

Did this fix it?

How do I stop it coming back?

  • On EC2, use an instance role and let the SDK find it; do not copy access keys onto the server.
  • Cache credentials and clients instead of creating a new client for every request, since metadata calls are throttled.
  • When a container image runs on EC2, set the hop limit to 2 in the launch template.
  • Locally, set the AWS_ environment variables or a profile so the SDK never has to wait for the metadata service.

Sources

How was this page made?

I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.

Which stack is behind this page?

Projects of mine built on the same technologies:

Related fixes

Back end

CodeIgniter 4 cannot write its cache files

CodeIgniter\Cache\Exceptions\CacheException: Cache unable to write to /var/www/app/writable/cache/

CodeIgniter 4's file cache needs a cache directory that the web server can really write to.

2 causes, each with a fix

Back end

CodeIgniter 3 loads the model but the property is undefined

The model name you are loading is the name of a resource that is already being used: user

CodeIgniter 3 expects a model file named like its class, with the first letter capitalised, for example User_model.php holding class User_model.

2 causes, each with a fix

Next step

Want this handled for good?

I look after servers, apps and releases end to end. Tell me what keeps breaking.