0

I have the follow schema enter image description here (using AWS-IoTCore and AWS-Lambdas) that starts upon some MQTT event. At the end of the main process a flag success is saved in the database. This process takes around 5 min. I would like to call a Lambda function 24hs after the beginning of the process to check if the flag exists in the database. How can I set this timeout or delay in the Lambda function without stepfunctions?

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
IgorAlves
  • 5,086
  • 10
  • 52
  • 83
  • Why do you want to do this without step functions? I apologize if my question is irrelevant. – jellycsc May 31 '20 at 14:25
  • Does this answer your question? [Invoke AWS Lambda function only once, at a single specified future time](https://stackoverflow.com/questions/47880716/invoke-aws-lambda-function-only-once-at-a-single-specified-future-time) – Paradigm May 31 '20 at 15:00
  • If it wasn't 24 hours( <=15 minutes) it would be easy with SQS Delay queues. You can still do it with sqs delayed queue by adding an extra logic into your second lambda but that would be an another complexity problem. – Ersoy May 31 '20 at 15:04
  • May be cronschedule can help here! You can create cloudwatch events that accepts cron schedule! so you can set cloudwatch events trigger to lambda schedule to run after 24 hours with your lambda as destination arn – error404 May 31 '20 at 15:36
  • @Paradigm, no I will have multiple processes running at different times. Each one has its own serialNumber that identifies the subject. I must to count 24hs after each process. Ex: proc1-started 13hs, I should run the check 13+24hs. The poc2 starts 14hs. I must run 14+24hs. I mean I know the deltaT but not the initial time. I cant setup the cron-Lambda to a fixed time. There maybe a day that we will not need this check. This is why cron is not a solution for this scenario. – IgorAlves May 31 '20 at 15:40
  • @jellycsc, In fact the step function is an elegant solution for this scenario. Here you have the solution using wait in step functions https://www.youtube.com/watch?v=0fDjptXVcsc at 8:05. I was just wondering if I could setup this same `wait` approach directly in the lambda without the step functions. As I said before cron seems not be a solution. – IgorAlves May 31 '20 at 15:46
  • I'd probably just wait exactly 24 hours if I was asked to do this without StepFunctions. I wish this was easy with Lambda. Possibly schedule the Lambda to run every ten minutes with Amazon CloudWatch events. Eventually it will be close the right time though it may not be exactly 24 hours after the start. It's sloppy timing, possibly good enough and omits StepFunctions. StepFunctions executions costs money. Lambda executions cost money too. – starpebble May 31 '20 at 22:33

2 Answers2

2

There are two possible paths to "wait 24 hours" until execution of an AWS Lambda function.

Using a 'wait' step in AWS Step Functions

AWS Step Functions is ideal for orchestrating multi-step Lambda functions. There is a Wait state that can be triggered, and the flow could then trigger another Lambda function. Step Functions will track each individual request and you can see each execution and their current state. Sounds good!

Schedule your own check

Use Amazon CloudWatch Events to trigger an AWS Lambda function at regular intervals. The Lambda function could query the database to locate records for processes that did not complete successfully. However, since your code needs to find "non-successful" processes, each process would first need to be added to the database (eg at the start of the 'Process' function).

Alternate approach: Only track failures

The architecture in the above diagram stores 'success' in the database, which is then checked 24 hours later. Based on this diagram, it appears as though the only purpose of the database is to track successful processes.

Instead, I would recommend:

  • Changing the top line to only store 'failed' processes, and the time of their failure (assuming failure can be detected)
  • Trigger an AWS Lambda function at regular intervals that will:
    • Check the database
    • Retrieve failed processes that are older that 24 hours
    • Submit them for re-processing
    • Delete the records from the database, or mark them as 'retried' to avoid future retrievals

Basically, the database is used to track failures, rather than successes. The "wait 24 hours" step is replaced with a regular database check for failed processes.

I'm not sure why the systems wants to wait 24 hours before sending a 'success' email, but I presume that is an intentional part of your design.

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
0

You can use CloudWatch Events (or EventBridge) to schedule a Lambda function to run once, at a set time.

To create a CloudWatch Event rule, you can use the PutRule API operation to create the rule and then add the Lambda function as the target using PutTargets. The Lambda function should also allow CloudWatch Events to invoke the function in its function policy.

I also tested this out with the following cron expression: cron(0 0 1 1 ? 2021)

This would trigger the target Lambda function once at Fri, 01 Jan 2021 00:00:00 GMT. Also note, function may be invoked more than once due to CW invoking the function asynchronously. After the Lambda function has been invoked, you can delete the rule from the same function for clean-up.

Paradigm
  • 1,876
  • 1
  • 12
  • 16