aws emr --steps fails: Error parsing parameter, Expected '=', received 'EOF'
Error parsing parameter '--steps': Expected: '=', received: 'EOF' for input:Short answer
The AWS CLI tried to read your option as shorthand key=value pairs and ran out of text where it expected an equals sign. Usual reasons: a space between pairs, a comma inside an Args value, or JSON whose quotes the shell changed. Remove the spaces, or put the steps in a JSON file and pass file://step.json.
Fix it fast
1Find out which cause you have
Move the steps into a JSON file and validate it. If the file is valid and the command works with file://, the problem was how the value was typed on the command line.
python3 -m json.tool step.json aws emr add-steps --cluster-id j-XXXXXXXX --steps file://./step.json2Match what you see, then jump to its fix
- 1
The step was typed with spaces after commas or inside unquoted valuesWhite space inside the shorthand valueFix - 2
The step is a Streaming or Spark step with -files a,b or a comma-separated list inside one argumentAn Args value contains a comma, so shorthand cannot hold itFix - 3
Inline JSON that works in one terminal fails in another, especially on WindowsThe shell changed the quotes around inline JSONFix
- 1
What does the error look like?
Error parsing parameter '--steps': Expected: '=', received: 'EOF' for input:
Cause 1 of 3
White space inside the shorthand value
1Check
Shorthand is a comma-separated list of key=value pairs with no white space between pairs. A space after a comma, or an unquoted value that contains a space (such as a step name), splits the argument and leaves a key with no equals sign.
echo 'Type=CUSTOM_JAR, Name=My step,Jar=s3://amzn-s3-demo-bucket/mytest.jar'2Fix
Remove the spaces between pairs and wrap any value that contains a space in quotes, as the AWS CLI reference does for Name='Streaming Program'.
aws emr add-steps \
--cluster-id j-XXXXXXXX \
--steps Type=CUSTOM_JAR,Name='My step',ActionOnFailure=CONTINUE,Jar=s3://amzn-s3-demo-bucket/mytest.jar,Args=arg1,arg23Verify
The command prints a StepIds list instead of a parse error.
aws emr list-steps --cluster-id j-XXXXXXXXConfirmed by docs.aws.amazon.com
Cause 2 of 3
An Args value contains a comma, so shorthand cannot hold it
1Check
In shorthand, commas separate list items, so a comma inside a single argument is read as a break. The CLI reference shows a Streaming step with several files as a JSON-only example.
grep -n 'Args' step.json2Fix
Write that step as JSON, with each option and each value as its own list item, and pass the file.
[
{
"Name": "JSON Streaming Step",
"Type": "STREAMING",
"ActionOnFailure": "CONTINUE",
"Args": [
"-files",
"s3://amzn-s3-demo-bucket/mapper.py,s3://amzn-s3-demo-bucket/reducer.py",
"-mapper", "mapper.py",
"-reducer", "reducer.py",
"-input", "s3://amzn-s3-demo-bucket/input",
"-output", "s3://amzn-s3-demo-bucket/output"
]
}
]3Verify
Run add-steps with the file.
aws emr add-steps --cluster-id j-XXXXXXXX --steps file://./multiplefiles.jsonConfirmed by docs.aws.amazon.com
Cause 3 of 3
The shell changed the quotes around inline JSON
1Check
Linux and macOS want single quotes around JSON. The Windows command prompt wants double quotes with each inner double quote escaped by a backslash. PowerShell has its own rules. If the quotes get stripped, the value no longer looks like JSON and the shorthand parser fails.
aws emr add-steps --cluster-id j-XXXXXXXX --steps '[{"Type":"CUSTOM_JAR","Jar":"s3://amzn-s3-demo-bucket/mytest.jar"}]'2Fix
Skip the quoting problem: save the JSON to a file and load it with file://. A relative path such as file://./step.json works, and a path with spaces needs quoting for your terminal.
aws emr add-steps --cluster-id j-XXXXXXXX --steps file://./step.json3Verify
StepIds is printed.
aws emr list-steps --cluster-id j-XXXXXXXXConfirmed by docs.aws.amazon.com
How do I stop it coming back?
- Keep EMR steps in version-controlled JSON files and always pass them with file://.
- Validate the JSON file in CI before it is used.
- Use shorthand only for short steps with plain values.
- Run the same file from every terminal your team uses, instead of retyping quotes.
Sources
How was this page made?
I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.
Which stack is behind this page?
Projects of mine built on the same technologies:
- Dispatch ProA real-time logistics platform: Flutter apps, Spring Boot services and a Kafka event pipeline on AWS.
- RaheebA consumer app shipped to both stores from one Flutter codebase on a .NET and Azure back end.
- Loade DriverThe driver side of a delivery platform: live jobs, routing and earnings, past 5K Google Play downloads.
Related fixes
Servers
Status code 460 on an AWS Application Load Balancer
elb_status_code 460 in ALB access logs
A 460 is generated by the load balancer, not by your app.
3 causes, each with a fixServers
Application Load Balancer returns 504 for requests to EC2 instances
HTTPCode_ELB_504_Count
A 504 from an Application Load Balancer means it could not get an answer from a target in time.
4 causes, each with a fixServers
Apache has a certificate, but https:// does not open
No line with :443 in the output of ss -ltnp
HTTPS needs four things at once: mod_ssl loaded, Apache listening on 443, a virtual host on 443 with SSLEngine on, and a firewall that lets 443 in.
4 causes, each with a fixServers
apt-get does not exist on this EC2 instance
sudo: apt-get: command not found
apt-get belongs to Debian and Ubuntu.
2 causes, each with a fixServers
AWS CLI says you must specify a region
You must specify a region. You can also configure your region by running "aws configure".
The AWS CLI sends every request to one region and has none to use.
4 causes, each with a fixServers
docker login to Amazon ECR fails with: Cannot perform an interactive login from a non TTY device
Cannot perform an interactive login from a non TTY device
Docker asked for a password because none reached it, and a pipe or script has no keyboard to answer.
3 causes, each with a fix