Maaz Zindani

Loading the index…

↑↓ move↵ openesc close4072 pages
AI and LLM integration

Amazon Bedrock refuses a Claude model ID with on-demand throughput

ErrorInvocation of model ID ... with on-demand throughput isn't supported

Short answer

Some Bedrock models, including newer Claude versions, are not served from a single Region by their bare model ID. You must call them through an inference profile: put the profile ID or ARN in modelId. List the profiles in your Region, use that ID, and make sure your IAM and Region policies allow every Region the profile routes to.

  • Applies to Amazon Bedrock Runtime, AWS CLI v2, boto3 1.x
  • Updated
  • By Maaz Zindani, Senior Software Engineer

Fix it fast

  1. 1Find out which cause you have

    List the system-defined inference profiles for your Region and look for your model. If it appears there, call that ID.

    aws bedrock list-inference-profiles --region us-east-1 --type-equals SYSTEM_DEFINED \
      --query "inferenceProfileSummaries[?contains(inferenceProfileId, 'claude')].[inferenceProfileId,inferenceProfileName]" \
      --output table
  2. 2Match what you see, then jump to its fix

    1. 1The modelId in your code looks like anthropic.claude-... with no region prefixThe call uses the bare model ID instead of an inference profile IDFix
    2. 2The profile is not found, or is not available from the Region you callThe profile ID is for a different geography than your source RegionFix
    3. 3The profile ID is right but the call is deniedA policy blocks one of the Regions the profile routes toFix

What does the error look like?

  • Invocation of model ID ... with on-demand throughput isn't supported

Cause 1 of 3

The call uses the bare model ID instead of an inference profile ID

1Check

AWS documents that for a model served only through an inference profile you specify a cross-Region profile. Cross-Region profile IDs carry a geography prefix such as us. in front of the model ID.

grep -rn "anthropic.claude" .   # find every modelId in your code

2Fix

Replace the model ID with the ID (or ARN) of the inference profile from the list above, in InvokeModel and Converse alike.

import boto3
client = boto3.client("bedrock-runtime", region_name="us-east-1")
resp = client.converse(
    modelId="us.anthropic.claude-3-haiku-20240307-v1:0",  # example profile ID; use one from list-inference-profiles
    messages=[{"role": "user", "content": [{"text": "Hello"}]}],
)

3Verify

The call returns a reply instead of a validation error.

print(resp['output']['message']['content'][0]['text'])

Confirmed by docs.aws.amazon.com

Did this fix it?

Cause 2 of 3

The profile ID is for a different geography than your source Region

1Check

Profiles are defined by the Regions they support, and available profiles differ by the source Region you call from. Ask Bedrock which profiles exist where you call.

aws bedrock get-inference-profile --region us-east-1 --inference-profile-identifier us.anthropic.claude-3-haiku-20240307-v1:0 --query 'models[].modelArn'

2Fix

Take the ID from list-inference-profiles run in the same Region your client uses, or call from a Region the profile supports.

aws bedrock list-inference-profiles --region eu-west-1 --type-equals SYSTEM_DEFINED

3Verify

get-inference-profile returns model ARNs for that Region.

aws bedrock get-inference-profile --region eu-west-1 --inference-profile-identifier <profile-id>

Confirmed by docs.aws.amazon.com

Did this fix it?

Cause 3 of 3

A policy blocks one of the Regions the profile routes to

1Check

A request to a cross-Region profile can be routed to any destination Region in it. AWS states that if any destination Region is blocked in a service control policy, the request fails even when the others are allowed.

aws bedrock get-inference-profile --region us-east-1 --inference-profile-identifier <profile-id> --query 'models[].modelArn'

2Fix

Allow the Bedrock inference actions (for example bedrock:InvokeModel*) in all destination Regions of the profile in your SCPs and IAM policies.

# IAM or SCP: allow in every destination Region listed above
"Action": ["bedrock:InvokeModel*"]

3Verify

Repeat the call that failed.

print(client.converse(modelId='<profile-id>', messages=[{'role': 'user', 'content': [{'text': 'Hello'}]}])['output'])

Confirmed by docs.aws.amazon.com

Did this fix it?

How do I stop it coming back?

  • Keep the profile ID in config, not hard-coded, so a model upgrade is one edit.
  • Read each model's page in the Bedrock docs for its inference profile IDs before you wire it in.
  • Remember that inference profiles do not work with Provisioned Throughput.

Sources

How was this page made?

I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.

Which stack is behind this page?

Projects of mine built on the same technologies:

Related fixes

AI

LangChain says it could not import the chromadb package

ImportError: Could not import chromadb python package. Please install it with `pip install chromadb`.

The message comes from LangChain's old Chroma class, which cannot import chromadb in the Python that runs your code.

2 causes, each with a fix

Next step

Want this handled for good?

I look after servers, apps and releases end to end. Tell me what keeps breaking.