Are there any open source scheduling tools for AWS Redshift?

Question

I have few sql statements (insert, update, delete, truncate) to be executed in a transaction, every 5 mins. I cannot use AWS Data Pipeline since minimum scheduling interval is 15 mins for pipeline. Are there any open source tools I can use? Can I use talend open studio to achieve this?

Via a cron job and a SQL file? – jzonthemtn Sep 13 '16 at 20:41 — jzonthemtn, Sep 13 '16 at 20:41

score 0 · Answer 1 · answered Sep 13 '16 at 16:14

0

You can try to overcome the limitation of the 15 min schedule interval by creating 3 pipelines and starting them 5 min apart.

Another solution would be to use OnDemand pipeline. Then you can create a CloudWatch event (with schedule) to trigger lambda function. Inside the lambda you can start the pipeline execution.

answered Sep 13 '16 at 16:14

luk75

44
2

I am not sure if it is possible to perform insert, update, delete, truncate on redshift tables with pipeline, is it?\ – Rahul Gupta Sep 13 '16 at 16:22

Are there any open source scheduling tools for AWS Redshift?

1 Answers1