0

I have 3 python scripts perfectly working individually on my EC2 machine, now i want all 3 of them to run one after the other automatically using AWS Step Functions. How can i possibly do that? I have done my part on the research going through almost all the official AWS Documentations, but couldn't find a thing that could help me out.

Sarthak
  • 1
  • 1

1 Answers1

0

The scripts must be adapted to work with AWS Step functions, as describe in FAQ:

Q: How does AWS Step Functions work with Amazon EC2 and other compute resources?

All work in your state machine is done by tasks. A task may be an Activity, which can consist of any code in any language. Activities can be hosted on Amazon EC2, Amazon ECS, mobile devices—basically any computer that can communicate with the AWS Step Functions API. Activities long-poll Step Functions using API calls to request work, receive input data, do the work, and return a result.

An example in Ruby is on this page: Example Activity Worker in Ruby:

For Python you can try to use boto3

FelixEnescu
  • 4,664
  • 2
  • 33
  • 34
  • Thank you! But I have already read all of the above in AWS documents. The problem still remains intact as in how can i write an activity function in python? And also, the three programs are just a simple output programs that write the output into a csv file in AWS s3. – Sarthak Jun 20 '18 at 06:17
  • May be use simple Lambda: the first do the work, write the file to S3 then create a SNS event that will fire the next function and so on. – FelixEnescu Jun 20 '18 at 07:17
  • Yes, I've tried using Step Functions with AWS Lambda, and it works perfectly fine. But the issue here is the python programs are mostly on data analytics, and they couldn't work on AWS Lambda. So, all I'm left with is use EC2 for executing the programs and then run them all together as required on Step Functions. – Sarthak Jun 21 '18 at 08:09
  • Could you arrive at any solution on the same? I am trying to solve a similar problem. – Aakash Basu May 12 '20 at 06:51