0

I have a situation where data is getting loaded into staging tables of Aurora MySQL from AWS S3 using Lambda function on a daily basis. Now, I need to schedule a job in Aurora DB which will trigger a SQL Procedure (having some transformation logic) to load the data from staging tables to target tables.

Can this be achieved in AWS Aurora Database? Does it needs any help from any other AWS Services?

Nehal
  • 1
  • 2

1 Answers1

1

AFAIK you can't schedule db jobs from within the DB. You would need to use other AWS services.

Some options for scheduled tasks:

  • Trigger a cron job via Cloudwatch events. See this page for information on integrating with AWS Batch.

  • If using ECS, then scheduled tasks are an option

  • You've also got the option of running cron directly on an EC2 instance, but this seems flaky, and you'd be better off using AWS IMO.

You have a few options for the target of a cron job. It could trigger a Lambda function to modify the db (probably the easiest approach), but is likely to involve low level / Raw SQL operations.

Alternatively, you could launch a Docker container on ECS. This means you can use frameworks, or other libraries to give you abstractions for the DB update.

Matthew Hegarty
  • 3,791
  • 2
  • 27
  • 42