Elastic Map Reduce Error

Question

I am getting an error when using Elastic Map Reduce and I am not sure what it means because it is not very descriptive. I want to know specifically what kind of JSONDecodeError I am getting. "12" is not descriptive. This is the output. I am using the MRjob class

writing to step-0-mapper_part-00000
writing to step-0-mapper_part-00001
Counters from step 1:
  Undecodable input:
    JSONDecodeError: 12
  Unencodable output:
   TypeError: 2
writing to /tmp/emr_irene.ec2-user.20130716.171313.288877/step-0-mapper-sorted
> sort /tmp/emr_irene.ec2-user.20130716.171313.288877/step-0-mapper_part-00000 /tmp/emr_irene.ec2-     user.20130716.171313.288877/step-0-mapper_part-00001
writing to step-0-reducer_part-00000
Counters from step 1:
 Undecodable input:
 JSONDecodeError: 12
 Unencodable output:
   TypeError: 2

score 1 · Answer 1 · answered Jul 17 '14 at 17:24

1

The number 12 in this case is the number of json encode errors you had. To find out specifically why they failed the best way I know how is by reducing your input set and triangulating the issue by rerunning the job.

answered Jul 17 '14 at 17:24

Justin Bozonier

7,584
9
44
46

score 0 · Answer 2 · edited Jan 18 '17 at 19:55

0

If you are working on Amazon AWS, you can run following script and you will get more information about your error:

python -m mrjob.tools.emr.fetch_logs --find-failure YOUR_JOB_ID

edited Jan 18 '17 at 19:55

thor

21,418
31
87
173

answered Jan 18 '17 at 19:32

ugur

400
6
20

Elastic Map Reduce Error

2 Answers2