Error parsing parameter, amazon aws emr

Question

I'm trying to create a step by Linux console:

aws emr add-steps --cluster-id j-XXXXXXXXXX --steps Type=CUSTOM_JAR,Name="S3DistCp step",Jar=/home/hadoop/lib/emr-s3distcp-1.0.jar,\ 
Args=["--s3Endpoint,s3-eu-west-1.amazonaws.com","--src,s3://folder-name/logs/j-XXXXXXXXXX/node/","--dest,hdfs:///output","--srcPattern,.*[a-zA-Z,]+"]

I jump the following error

Error parsing parameter '--steps': Expected: ',', received: '+' for input

How I can fix it?

I'm looking for a solution to upload multiple files to S3 and S3DistCp the Hive gather for Amazon EMR. Is there any other way?

I have another question: Now I am creating an SSH tunnel to connect to Hive, how I could connect with PHP?

At the moment I have solved the error by removing "src Pattern", however gives me another error, I include image below

Image error

This is the error that appears

INFO Synchronously wait child process to complete : hadoop jar /var/lib/aws/emr/step-runner/hadoop- 
INFO waitProcessCompletion ended with exit code 1 : hadoop jar
/var/lib/aws/emr/step-runner/hadoop-
INFO total process run time: 2 seconds
2016-07-12T14:26:48.744Z INFO Step created jobs:
2016-07-12T14:26:48.744Z WARN Step failed with exitCode 1 and took 2 seconds

Thx!!!

the error parsing is weird, did you try to put your options in a json file and call the command with the Json file instead just to see if it helps — Frederic Henri, Jul 12 '16 at 12:05
Hello Frederic, I've got to work removing "src Pattern", now I get another error that is not really as specified in amazon not much (I ascend above the image) — David, Jul 12 '16 at 12:27

score 1 · Answer 1 · answered Jun 04 '17 at 00:00

Try the JSON configuration

[
    {
        "Name":"S3DistCp step",
        "Args":["s3-dist-cp","--s3Endpoint=s3.amazonaws.com","--src=s3://mybucket/logs/j-3GYXXXXXX9IOJ/node/","--dest=hdfs:///output","--srcPattern=.*[a-zA-Z,]+"],
        "ActionOnFailure":"CONTINUE",
        "Type":"CUSTOM_JAR",
        "Jar":"command-runner.jar"        
    }
]

aws emr add-steps --cluster-id j-3GYXXXXXX9IOK --steps file://./myStep.json

http://docs.aws.amazon.com/emr/latest/ReleaseGuide/UsingEMR_s3distcp.html#UsingEMR_s3distcp.step

Error parsing parameter, amazon aws emr

1 Answers1