0

I am currently designing a serverless system where I want to store objects that will be used when the user defines. For example the user can say "this object will do x in 3 days." The object gets stored in the DB, and then 3 days later, an action will happen with that object.

I want this to be as real time as possible. Time driven events architecture this answer suggests using a priority queue, which I think is a great idea. But in a serverless architecture, how can I pull objects off that priority queue only after the time that the user set passes? The only way I can think to do this now is to poll the queue every so often, but it seems like it would be better if the priority queue could pop itself and take action if the object at the front of the queue is expired.

This seems like it could work https://aws.amazon.com/blogs/mt/build-scheduler-as-a-service-amazon-cloudwatch-events-amazon-eventbridge-aws-lambda/ but I worry that it's overkill or that it's not a perfect fit that will run into scaling issues.

bronzeson
  • 1
  • Look at the temporal.io. It allows modeling your use case as code based workflow. – Maxim Fateev Nov 18 '21 at 04:53
  • Have a look at the EventBridge Scheduler, which allow you to programatically/dynamically create scheduled events. https://docs.aws.amazon.com/scheduler/latest/UserGuide/what-is-scheduler.html – joyrexus Jul 20 '23 at 19:42

1 Answers1

0

I would suggest using the TTL feature of AWS DynamoDB. ref - https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/TTL.html

When you store the object, add a record to DynamoDB with the details of the object and the epoch time (now + user-provided time). Once the time you give expires, DynamoDB will delete the record.

Create a DynamoDB stream that triggers a lambda function that can now process the record info it receives in the event.

ref - https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/time-to-live-ttl-streams.html

kumar harshit
  • 487
  • 6
  • 12
  • This seems like it could be a really good option. But I don't want to delete the record from the database at the end of the day. I just want to update it. So the TTL would delete, then the object would be processed, updated, and returned to the DB. Is that an inefficient process? – bronzeson Nov 18 '21 at 16:45
  • Well. I wouldn't say it's inefficient. DynamoDB is serverless and you mainly pay for storage and capacity units in your case. you'd be consuming almost the same resources whether you use the same table or a different table. It's also not uncommon to have a 'pending transactions' table and a 'completed transactions' table. – kumar harshit Nov 19 '21 at 04:53