-2

I have an EC2 instance that schedule many tasks (using crontab). some of them are executed every 1 min, 5 min, and so on..

I want to move all cron tasks into AWS service.

I am trying to figure which AWS service can give me the best solution.

I found 2 services that can schedule cron like tasks:

  1. AWS Data Pipeline
  2. AWS Lambda

which of them can give me the best solution?

Chacko
  • 1,506
  • 1
  • 20
  • 42
Eyal Ch
  • 9,552
  • 5
  • 44
  • 54

1 Answers1

2

I don't know how you want to define "best" but if you have many tasks, each one will require a separate pipeline, and that will cost you around $1 each.

Lambda on the other hand will probably be much less - you get 1M requests free, and they're $0.20 / million after that. You will also get charged based on the time & memory each task takes to run. There are some limits (5 min is the max time I think) so you'll have to take that into consideration.

But overall, I think Lambda will be much cheaper to run.

chris
  • 36,094
  • 53
  • 157
  • 237
  • but if i want to run the task on one of my EC2 machine can i do it only with Data pipeline? (with task runner installed on my machine)? what about version control? if i run on lambda then i need to update my libraries, but with data pipeline on running on my EC2 i dont need (i already have version control on my EC2 for my application that is running there) – Eyal Ch May 15 '16 at 17:10