0

I'm trying to create a simple dataFlow pipeline with a single Activity of ShellCommandActivity type. I've attached the configuration of the activity and ec2 resource.

When I execute this the Ec2Resource sits in the WAITING_ON_DEPENDENCIES state then after sometime changes to TIMEDOUT. The ShellCommandActivity is always in the CANCELED state. I see the instance launch and very quicky changes to the terminated stated.

I've specified a s3 log file url, but that never gets updated.

Can anyone give me any pointers? Also is there any guidance out there on debugging this?

Thanks!!

enter image description here enter image description here

webber
  • 1,834
  • 5
  • 24
  • 56

1 Answers1

1

You are currently forcing your instance to shut down after 1 minute which gives the TIMEOUT status if it can't execute in that time. Try increasing it to 50 minutes.

Also make sure you are using an AMI that runs Amazon Linux and that you are using full absolute paths in your scripts.

S3 log files are written as:

s3://bucket/folder/

gevert
  • 682
  • 6
  • 8